Bangle.js and Edge Impulse for Machine Learning

INTRODUCTION

In this tutorial you will learn how to get started with Machine Learning on your Bangle.js watch. Specifically you will build and train a model learning to recognize different movements of your watch hand. The steps include how to collect data, how to use Edge Impulse for the machine learning part and how to finally upload the learned model back to the watch and utilise it there.

Prerequisites

Hardware

Software

Preparation

Collect gesture samples

This part will guide you how to use your watch to collect multiple samples for one gesture type at a time.

  1. Pair your computer with the watch using Espruino Web IDE
  2. Paste the below Gesture collection code into the right side in Espruino Web IDE (adapted from this code)
    • the code will create a text file in the watch memory
  3. Name the event you are going to collect samples for by changing the line event="left";
    • use e.g. event="left"; for twitching your watch hand left and later on event="right"; for the opposite direction
    • upload the code to RAM. Do not upload this code to flash or storage, you might in worst case need to reset the watch completely.
  4. Perform the gesture
    • repeat the gesture many times, the more the merrier!
      • wait a second between each
    • the gesture collecting code will append each sample to the .CSV-file
    • a graph will also be shown on your watch screen
  5. Repeat steps 3-4 above, remember to change event="<gesture>"; where <gesture> is the hand movement you will collect
  6. The devil is in the details, do not e.g. remove the seemingly insignificant semi-colon ; !

Gesture collecting code:

// ******* Gesture collecting code ********
name="Gesture";
event="left";

var fname = 1;

function gotGesture(d) {  
  var f = require("Storage").open(event + "." + fname + ".csv", "a");

  print("timestamp, x, y, z");
  f.write("timestamp, x, y, z\n");
  for (var j=0;j<d.length;j+=3) {
       print(j +", ", d[j] + ", " + d[j+1] + ", " + d[j+2] );
       f.write(j + ", " + d[j] + ", " + d[j+1] + ", " + d[j+2] +"\n" );
  }

  g.clear();
  g.setColor(1,1,1);
  var my = g.getHeight()/2;
  var sy = my/128;
  var sx = g.getWidth()/(50*3);
  g.drawLine(0,my,g.getWidth(),my);
  for (var i=0;i<d.length-3;i+=3) {
    for (var c=0;c<3;c++) {
      g.setColor(c==0,c==1,c==2);
      g.drawLine(i*sx, my+d[i+c]*sy, (i+3)*sx, my+d[i+c+3]*sy);
    }
  }
  g.flip(1);
}

Bangle.on('gesture',gotGesture);

Transfer .CSV-files from Bangle.js to your computer

This part will guide you how to transfer the .CSV-files from your watch to your computer.

Split .CSV-files using Python

This part will guide you how to split the .CSV-files you've downloaded from your watch into separate .CSV-files. The reason for this is that Edge Impulse requires one .CSV-file per sample.

  1. Copy the below Python code (shamelessly copied from Stackoverflow) into your favourite Python editor.
  2. Replace the path on the second line (starting with PATENTS = ...) with the full path and filename for the first file you want to split. I.e. the file you downloaded in previous steps.
  3. Run the code in your Python editor
    • The program will search for the string 'timestamp, x, y, z' in the original file and for each time (= sample) it finds, create a new file.
    • If you don't use Python, you'd need to split the file for each sample using some other method, manual or automatic. Remember that the samples aren't all of the same size so the amount of rows will vary.
    • You should now have several .CSV-files in the folder you chose. The files will be named like left.1.csv (StorageFile)-15.csv where -15 at the end is a running number.
  4. Repeat steps 2-3 above for each file you downloaded from your watch.
import re
PATENTS = 'C:/temp/left.1.csv (StorageFile)'

def split_file(filename):
    # Open file to read
    with open(filename, "r") as r:

        # Counter
        n=0

        # Start reading file line by line
        for i, line in enumerate(r):

            # If line match with template -- <?xml --increase counter n
            if re.match(r'timestamp, x, y, z', line):
                n+=1

                # This "if" can be deleted, without it will start naming from 1
                # or you can keep it. It depends where is "re" will find at
                # first time the template. In my case it was first line
                if i == 0:
                    n = 0               

            # Write lines to file    
            with open("{}-{}.csv".format(PATENTS, n), "a") as f:
                f.write(line)

split_file(PATENTS)

Use Edge Impulse for machine learning

In this part you will learn how to upload the sample files you've created earlier, create a machine learning model, train and finally analyse it. This tutorial will only cover the essential steps needed for Bangle.js. To learn more about Edge Impulse, see e.g. getting started and continuous motion recognition.

Log in and create a project

Upload sample data

Upload sample data

Create an impulse

An impulse takes raw data, uses signal processing to extract features, and then uses a learning block to classify new data. These steps will create an impulse.

Create impulse

Generate features

Feature explorer

Train the neural network

Here you will train the neural network and analyse its performance.

Training performance

Download the trained model

Here you will download the trained model to your computer.

DEPLOYMENT

Transfer the trained model to Bangle.js from your computer

This part will guide you how to transfer the model file from your computer to Bangle.js.

Upload file

Test the gestures on Bangle.js!

Finally you will be able to test how well the trained model performs in real life! Just a few steps left.

Bangle.on('aiGesture',(gesture,raw)=>print(gesture,raw));

Bangle.on('aiGesture',(gesture)=>{
  E.showMessage(gesture);
  setTimeout(()=>g.clear(), 1000);
});

FINAL COMMENTS

First of all, hopefully you with this short tutorial were successful in training and recognising gesture events from your Bangle.js. Hopefully it also inspires you to try to improve the performance, e.g. by collecting more samples, by collecting more event types or by tweaking the different parameters and settings in Edge Impulse.

This page is auto-generated from GitHub. If you see any mistakes or have suggestions, please let us know.