Introduction
Supervised learning • Inference engines at the edge • Edge AI components • Edge AI applications • Data acquisition
Practicals: Try out model training environments based on CPU and GPU
Data Planning & Pre-processing
CRISP-DM Methodology & MLOps for Edge AI • Use case of audio feature pre-processing • Split audio file for training • Conversion of audio segments to spectrogram • Training data preparation
Practicals: Use SoX (Sound eXchange) software to trim, filter, and plot Spectrograms • View Spectrograms before training • Organize spectrograms into folders for training • Work with Environment Sound Classification (ESC) dataset
Creating Neural Network (NN) Models for Edge
Review of different kinds of neural networks • Convolutional Neural Network (CNN) using 2D and 1D convolution • Simple Recurrent Neural Network model • Transfer Learning • Chaining NN models
Practicals: Code (using Keras) simple Convolutional and Recurrent neural network using small datasets • Write code to use pretrained models (such as MobileNet) for transfer learning
Time Series
Terminology – univariate, multivariate, regression, classification • Pre-processing using Pandas • Time Series as supervised learning problem • Keras Time Series Generator API • Time Series classification using CNN • RNN for Time Series Data • Time Series Database
Practicals: Set up time series dataset as supervised learning task • Perform CNN based fingerprint analysis of sections of time series data
Edge AI Hardware
Constrained Inference platforms • Inference server • Linux based SBC • 32 bit Microcontroller • NN Accelerator • GPU • FPGA
Practicals: Take picture, play and record audio using SBC • Read sensor data using serial port from SBC and Microcontroller
Model Formats
Model Formats • Open Neural Network Exchange Format (ONNX) • TensorFlow Lite (TFLite) • Viewing Model Graph • Model Format Conversion • Model Quantization
Practicals: Train and convert Scikit-Learn (ML) model to ONNX • Train and convert Keras Models to ONNX and TFLite
Model Inference
Inferencing steps • Input Tensor Shape • ONNX runtime, TFLite Interpreter Python methods • TFLite C++ Classes
Practicals: Convert Scikit-learn model to ONNX format. Perform Inference of NN model using ONNX/TFLite formats in Python environment. Infer TFLite model using C++ classes
TinyML
TinyML Implementation frameworks • TFLite Micro • Converting Keras Model • Setting up and using TFLite Micro Interpreter
Practicals: Quantize Fully Connected TFLite Model • Convert Model to C array for storing in MPU • Read sensor value and execute model using TinyML (TFLite Micro) on Cortex-M4 device
Neural Network Accelerator
Neural Network Accelerator platforms • Model compiler and workload partitioning • Executing Model on Accelerator • Working with multiple accelerators
Practicals: Quantize and compile CNN Model for NN Accelerator • Run compiled model on accelerator and compare execution time with SBC
Object Detection
MobileNet architecture •Object Detection using MobileNet based SSD • Object Detector Output • Decoding and boxing detected output
Practicals: Use object detection code for determining if a car has been parked beyond a stipulated time
Monitor Model Inference and Performance
Accessing Model Inference using MQTT • Model inference using REST API • Monitoring Model Drift
Practicals: Communicate model inference output using MQTT and Flask webserver
Deploy Model to Edge Device
Enumerate Model Inference Dependencies • Create Dockerfile for Model Inference Container • Push Model Inference Container to Edge Device
Practicals: Create ONNX/TFLite inference Dockerfiles • Use Dockerfile to create Docker container and test it on Edge device
Edge AI use cases
Edge AI application planning template • Discussion on creating Edge AI solution for different use cases
