../../_images/cartesiam_logo.png

Multispeed Fan Classifier

../../_images/fan_preview_text.jpg ../../_images/banner.png

I. Objectives of the tutorial

In this project, we will create a smart device that will read vibration patterns from a 3-speed USB fan, and be able to detect the operating mode of the fan, by classifying the vibration patterns into categories such as:

  • turned off
  • running at speed 1 (low)
  • running at speed 2 (medium)
  • running at speed 3 (high)
  • anomaly: air flow obstructed
  • anomaly: fan tilted
  • anomaly: tapping on fan

This device will integrate and use Cartesiam’s NanoEdge AI Library for Classification, which will be selected via NanoEdge AI Studio.

Video: Vimeo link.


II. Requirements

1. Hardware

The required hardware for this project are:


2. Software

The required software for this project are:


III. Making a data logger

1. Putting the hardware together

Here we will create a data logger to acquire vibrational data and log it via serial port (USB).

Below is the hardware setup to reproduce, to connect the development board to the accelerometer via I2C:

../../_images/hardware.png

Connect the acceletometer to the board as follow (LIS3DH to L432KC):

  • Vin to 3V3
  • GND to GND
  • SCL to D1/TX
  • SDA to D0/RX

2. Data logging

i. Setting up the project

  1. Install mbed CLI.

    Make sure your python path, mbed-cli path, and GNU toolchain path are all correctly set. For more information check the other tutorials.

  2. Create your main project directory.

    $ mkdir /absolute/path/to/myproject/
    
  3. Clone the multispeed_fan_tutorial repository into this directory.

    $ cd /absolute/path/to/myproject/
    $ git clone https://github.com/cartesiam/multispeed_fan_tutorial.git
    
  4. Import mbed-os and make sure its path is correctly set.

    $ mbed import mbed-os
    
    $ mbed config -G MBED_OS_DIR /absolute/path/to/myproject/mbed-os
    [mbed] /absolute/path/to/myproject/mbed-os now set as global MBED_OS_DIR
    
  5. At the root of your project /absolute/path/to/myproject/, create a file named mbed_app.json, and copy paste the following contents in that file. It will make compilation much faster, and enable support for printing floats.

    {
       "target_overrides": {
          "*": {
             "target.printf_lib": "minimal-printf",
             "platform.minimal-printf-enable-floating-point": true
          }
       },
       "requires": ["bare-metal"]
    }
    

Note

You will have 2 sub-directories in your main project directory:

  • mbed-os containing the mbed-os repository
  • multispeed_fan_tutorial containing this tutorial’s repository

ii. Preparing the data logger

  1. Connect the NUCLEO-L432KC development board and your fan to your computer.

  2. Put the accelerometer / board on your fan. We stick it using blu-tack, but glue also works (even tape if desperate).

    ../../_images/blu-tack.jpg
  3. Compile and flash your code on the MCU. For data logging, we use the -DDATA_LOGGING compiler flag.

    $ cd /absolute/path/to/myproject/
    $ mbed compile -t GCC_ARM -m nucleo_l432kc -DDATA_LOGGING -f
    

    Important

    Take some time to read the code provided for data logging in the file main.cpp from the myproject/multispeed_fan_tutorial/src/ directory.

    In a nutshell, we read raw accelerometer data and place the acceleration samples (on the x, y, and z axes), one after the other until we get a line with length BUFFER_SIZE, then repeat the process. Then, we print the data to the serial.

    while(1) {
       /* we fill the accelerometer buffer */
       for (uint16_t i = 0; i < BUFFER_SIZE; i++) {
          if (lis3dh.data_ready()) { // New data is available
                lis3dh.read_data(&lis3dh_xyz[0]);
                acc_buffer[NB_AXES * i] = lis3dh_xyz[0];
                acc_buffer[(NB_AXES * i) + 1] = lis3dh_xyz[1];
                acc_buffer[(NB_AXES * i) + 2] = lis3dh_xyz[2];
          } else {
             i--; // New data not ready
          }
       }
       /* we print values to serial one line at a time */
       for (uint16_t isample = 0; isample < NB_AXES * BUFFER_SIZE - 1; isample++) {
          printf("%.4f ", acc_buffer[isample]);
       }
       printf("%.4f\n", acc_buffer[NB_AXES * BUFFER_SIZE] - 1);
    
    }
    
  4. Make sure the data logger is working.

    [Linux]:

    • Open a terminal and use the following command:

      $ ls /dev/ttyACM*
      
    • Copy the device name given by the command above (in our case /dev/ttyACM0) and use it in the following command:

      $ stty -F /dev/ttyACM0 115200 raw -clocal -echo icrnl
      
    • Check the serial output:

      $ cat /dev/ttyACM0
      

    [Windows]:

    • Open Tera Term and configure it.

      • In Setup > Terminal, increase terminal size (for example) to 85 x 25, and use: New Line, Receive NF and Transmit CR.
      • In Setup > Font > Font, increase your font size to (example) size 18.
      • In Setup > Serial port, select the correct Port and use a Speed of 115200.
      • In Setup > Save setup, save this setup to the default directory under the default name TERATERM.INI.

    You should now see a bunch of lines with many numerical values appearing in real time. This is your raw accelerometer data. If needed, unplug/replug the board, or change USB ports, or quit and relaunch your terminal / Tera Term.


iii. Sampling methodology

In this project, we chose the following parameters:

  • Sampling frequency: 1.6 kHz (LIS3DH_DR_LP_1R6KHZ in the following code). The higher the frequency, the higher the chances to get many interesting features in the signal snapshot we capture.

  • Range: 4 g (LIS3DH_FS_4G in the following code).

    LIS3DH lis3dh (lis3dh_i2c, LIS3DH_G_CHIP_ADDR, LIS3DH_DR_LP_1R6KHZ, LIS3DH_FS_4G);
    
  • Accelerometer buffer size: 512 samples per axis, with 3 axes.

    #define BUFFER_SIZE     512
    #define NB_AXES         3
    

This results in an effective signal length of 512/1600 = 0.32 seconds.

Important

It means that each line in our input files (that we will create using our data logger) will represent an independent signal example; it will be a 0.32 second snapshot of the accelerometer temporal vibration data, as measured on our fan.


IV. Using NanoEdge AI Studio to find the best library

The goal of this section is to use NanoEdge AI Studio to find the static library that is most relevant to our embedded application.

Please refer to the NanoEdge AI Studio documentation for additional details.

1. Creating a project

Close your terminal / Tera Term, and launch NanoEdge AI Studio.

In the main window, create a new Classification project.

  • Select Target: NUCLEO-L432KC Cortex M4 under STMicroelectronics
  • Choose the Max RAM: here 32kB
  • Select the sensor type: 3-axis accelerometer
../../_images/new_project.png

2. Importing signal examples for each class

In the Studio, step number 2, we will create and import all our input “datasets”. There will be one file for each class, each containing 100 signal examples of the fan behavior associated to that class.

For each Class, we need to do the following:

  • Choose signals,

    ../../_images/choose_signal.png
  • Click the From serial (USB) tab,

  • The correct Port should already be selected (refresh if needed),

  • Leave Baudrate at 115200,

  • Set Max lines to 100,

  • Click the red Record icon.

    ../../_images/signal_import.png
  • Validate import after the 100 lines have been recorded.

  • Then rename the class to a something meaningful,

    ../../_images/rename.png
  • And repeat the process, by clicking Add class.

    ../../_images/add_class1.png

Repeat the process to capture all the fan behaviors we want to classify:

  1. Fan is turned off: just record signals when nothing is happening.

  2. Fan is running at lowest speed (speed 1)

  3. Fan is running at intermediate speed (speed 2)

  4. Fan is running at highest speed (speed 3)

  5. Fan’s airflow is obstructed by a piece of paper, while running at speed 1

  6. Fan is horizontally tilted by 90°, while running at speed 1

    ../../_images/fan_tilted.jpg
  7. User is tapping on the fan with fingers, while running at speed 1

Note

While recording signals, try to have your setup as still as possible, because the accelerometer will pick up any background / parasite vibration, and possibly “pollute” your log.

Here are all the classes imported:

../../_images/all_classes_imported.png

We are now ready to start the benchmark.


3. Benchmarking the library

Click START under Step 3: Optimize and Benchmark, and select all 7 classes.

../../_images/select_all_classes.png

Then, validate and start the benchmark.

The Studio will automatically find the best bundle of signal processing, machine learning model, and hyperparametrization given the input signals provided, and learn the appropriate knowledge.

Wait until the benchmark is complete.
It may take a while. In our case, it took 92 minutes using 14 CPU cores.
Take a break, grab a coffee (or tea), and/or read the NanoEdge AI Studio documentation regarding benchmark and performance indicators.

The graph below summarizes the performances of the best library found during benchmark. Each dot corresponds to a signal example as imported in the files created in the previous step. In our case, each signal has been attributed to the correct class, with a certainty percentage of almost 100% every time (results are nearly perfect).

../../_images/benchmark_results.png

Click Step 4: Emulator to start testing the library using fresh data.


4. Emulating the Library

On the Emulator screen, click Initialize Emulator.

Select the Serial data tab. Make sure you fan and electronic board are plugged in, and that the correct Port and Baudrate (115200) are selected.

Then, click the red Record button to start collecting signals.

Finally, play around with your fan, going from one state to another, and trying to cover all 7 classes. See how the Emulator reacts, and if the results conform to your expectations. You should be able to properly reproduce and detect all classes with reasonable certainty percentages.

../../_images/emulator_results.png

Here, we logged 100 signals, with the following repartition: fan_off (10), speed_1 (20), speed_2 (16), speed_3 (22), fan_obstructed (6), fan_tilted (15) and user_tapping (11). The class currently being detected on the image is speed_3, with a certainty of 100%.

When you’re happy with the results, you can move on to the next step (Step 5: Deploy).

Note

Otherwise, if Emulator results are not as good as expected, go back to Step 2: Signals, re-record signals in problematic classes (something must have gone wrong during data logging), and start a new benchmark.

For more informations about this step, check Using the Emulator.

5. Downloading the library

Leave all options unchecked (multi-library, compilation flags…), click Compile, and choose Development version.

You will obtain a .zip file containing:

  • the static library itself (libneai.a),
  • the NanoEdge AI header file (NanoEdgeAI.h), containing functions and variable definitions,
  • the knowledge header file (knowledge.h), containing the model’s knowledge.

V. Embedding the library on the final device

1. Copying library, knowledge, and headers

Go back to your project directory:

$ cd /absolute/path/to/myproject/multispeed_fan_tutorial/

Then, copy:

  • knowledge.h into the directory inc/
  • NanoEdgeAI.h into the directory inc/
  • libneai.a into the directory lib/

2. Compiling the code on the MCU

Go back to your mbed project directory:

$ cd /absolute/path/to/myproject/

Then, compile and flash your code on the MCU. We now use the -DLIBRARY_MODE compiler flag, because we want to run the classification features of the library (instead of just doing data logging).

$ mbed compile -t GCC_ARM -m nucleo_l432kc -DLIBRARY_MODE -f

Important

Take some time to read the code provided for classification in the file main.cpp from the myproject/multispeed_fan_tutorial/src/ directory.

In a nutshell, we first initialize the knowledge, then run the NanoEdge AI Classifier, and print the output to the serial.

/* we initialize the knowledge */
NanoEdgeAI_knowledge_init(knowledge);

while(1) {
   /* we get fresh accelerometer data */
   fill_acc_array();
   /* we classify the input signal and get a class id as output */
   class_number = NanoEdgeAI_classifier(acc_buffer, output_class_buffer, 0);
   /* we print the result to the serial */
   printf("Class detected: %s (Certainty: %d%%)\n",
      id2class[class_number], // the name of the class
      (uint16_t) (output_class_buffer[class_number - 1] * 100)); // the associated probability %
}

3. Testing our final device

  1. Place the electronic board / accelerometer on the fan (we stick it using blu-tack).
  2. Plug the fan and the electronic board to your computer.
  3. Open a terminal and watch your serial port (use cat /dev/ttyACM*) on Linux, or Tera Term on Windows).
  4. Play with the fan, trying to replicate all the different classes, by changing the speed, tilting the fan, obstructing it with a piece of paper, and tapping on it with your finger.

You should be able to see all the different regimes in your terminal output, after allowing some time for stabilization:

../../_images/fan_testing.png
Congratulations.
You just built a Multi-speed Fan Classifier, using NanoEdge AI Studio and a simple data logger, without using any particular skill in AI, Machine Learning, or Data Science.
Feel free to alter the code, and use your data-logger to build any smart device you can think of!
Sky’s the limit!

Resources

Documentation
All NanoEdge AI Studio documentation is available here.
Tutorials
Step-by-step tutorials, to use NanoEdge AI Studio to build a smart device from A to Z:

Useful links: