Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,15 +1,47 @@
---
title: Overview
title: Introduction to TinyML
weight: 2

### FIXED, DO NOT MODIFY
layout: learningpathall
---

TinyML represents a significant shift in machine learning deployment.

Unlike traditional machine learning, which typically depends on cloud-based servers or high-powered hardware, TinyML is tailored to function on devices with limited resources, constrained memory, low power, and less processing capabilities.

TinyML has gained popularity because it enables AI applications to operate in real-time, directly on the device, with minimal latency, enhanced privacy, and the ability to work offline.

This shift opens up new possibilities for creating smarter and more efficient embedded systems.

## Module Overview
This session delves into TinyML, which applies machine learning to devices with limited resources like microcontrollers. This module serves as a starting point for learning how cutting-edge AI technologies may be put on even the smallest of devices, making Edge AI more accessible and efficient.

Additionally, we'll cover the necessary setup on your host machine and target device to facilitate cross-compilation and ensure smooth integration across all devices.
This Learning Path is about TinyML, applying machine learning to devices with limited resources like microcontrollers. It serves as a starting point for learning how cutting-edge AI technologies may be put on even the smallest of devices, making Edge AI more accessible and efficient.

You will learn how to setup on your host machine and target device to facilitate compilation and ensure smooth integration across all devices.

## Examples of Arm-based devices and applications

There are many devices you can use for TinyML projects. Some of them are listed below.

### Raspberry Pi 4 and 5

Raspberry Pi single-board computers are excellent for prototyping TinyML projects. They are commonly used for prototyping machine learning projects at the edge, such as in object detection and voice recognition for home automation.

### NXP i.MX RT microcontrollers

NXP i.MX RT microcontrollers are low-power microcontrollers that can handle complex TinyML tasks while maintaining energy efficiency, making them ideal for applications like wearable healthcare devices and environmental sensors.

### STM32 microcontrollers

STM32 microcontrollers are used in industrial IoT applications for predictive maintenance. These microcontrollers are energy-efficient and capable of running TinyML models for real-time anomaly detection in factory machinery.

### Arduino Nano 33 BLE Sense

The Arduino Nano, equipped with a suite of sensors, supports TinyML and is ideal for small-scale IoT applications, such as detecting environmental changes and movement patterns.

### Edge Impulse

In addition to hardware, there are software platforms that can help you build TinyML applications.

## Introduction to TinyML
TinyML represents a significant shift in how we approach machine learning deployment. Unlike traditional machine learning, which typically depends on cloud-based servers or high-powered hardware, TinyML is tailored to function on devices with limited resources, such as constrained memory, power, and processing capabilities. TinyML has quickly gained popularity because it enables AI applications to operate in real-time, directly on the device, with minimal latency, enhanced privacy, and the ability to work offline. This shift opens up new possibilities for creating smarter and more efficient embedded systems.
Edge Impulse platform offers a suite of tools for developers to build and deploy TinyML applications on Arm-based devices. It supports devices like Raspberry Pi, Arduino, and STMicroelectronics boards.
Original file line number Diff line number Diff line change
@@ -1,26 +1,24 @@
---
title: Introduction to TinyML on Arm using PyTorch v2.x and Executorch
title: Introduction to TinyML on Arm using PyTorch and ExecuTorch

minutes_to_complete: 40

who_is_this_for: This learning module is tailored for developers, engineers, and data scientists who are new to TinyML and interested in exploring its potential for edge AI. If you have an interest in deploying machine learning models on low-power, resource-constrained devices, this course will help you get started using PyTorch v2.x and Executorch on Arm-based platforms.
who_is_this_for: This is an introductory topic for developers, engineers, and data scientists who are new to TinyML and interested in exploring its potential for edge AI. You will learn how to get started using PyTorch and ExecuTorch for TinyML.

learning_objectives:
- Identify TinyML and how it's different from the AI you might be used to.
- Understand the benefits of deploying AI models on Arm-based edge devices.
- Select Arm-based devices for TinyML.
- Identify real-world use cases demonstrating the impact of TinyML in various industries.
- Identify real-world use cases demonstrating the impact of TinyML.
- Install and configure a TinyML development environment.
- Set up a cross-compilation environment on your host machine.
- Perform best practices for ensuring optimal performance on constrained edge devices.


prerequisites:
- Basic knowledge of machine learning concepts.
- Understanding of IoT and embedded systems (helpful but not required).
- A Linux host machine or VM running Ubuntu 20.04 or higher, or an AWS account to use [Arm Virtual Hardware](https://www.arm.com/products/development-tools/simulation/virtual-hardware)
- Target device, phyisical or using the or Corstone-300 FVP, preferably Cortex-M boards but you can use Cortex-A7 boards as well.

- Understanding of IoT and embedded systems.
- A Linux host machine or VM running Ubuntu 22.04 or higher.

author_primary: Dominica Abena O. Amanfo

Expand All @@ -35,15 +33,15 @@ operatingsystems:
- Linux

tools_software_languages:
- Corstone 300 FVP
- Grove - Vision AI Module V2
- Arm Virtual Hardware
- Fixed Virtual Platform
- Python
- PyTorch v2.x
- Executorch
- PyTorch
- ExecuTorch
- Arm Compute Library
- GCC
- Edge Impulse
- Nodejs
- Node.js

### FIXED, DO NOT MODIFY
# ================================================================================
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,15 +1,58 @@
---
title: Benefits of TinyML for Edge Computing on Arm Devices
weight: 4
title: Benefits and application of TinyML for Edge Computing
weight: 3

### FIXED, DO NOT MODIFY
layout: learningpathall
---

The advantages of TinyML for edge computing on Arm devices are vast, particularly when paired with Arm's architecture, which is widely used in IoT, mobile devices, and edge AI deployments. Here are some key benefits:
## Benefits and applications

The advantages of TinyML match up well with the Arm architecture, which is widely used in IoT, mobile devices, and edge AI deployments.

Here are some key benefits of TinyML on Arm:

- Power Efficiency: TinyML models are designed to be extremely power-efficient, making them ideal for battery-operated devices like sensors, wearables, and drones.

- Low Latency: Because the AI processing happens on-device, there's no need to send data to the cloud, reducing latency and enabling real-time decision-making.

- Data Privacy: With on-device computation, sensitive data remains local, providing enhanced privacy and security. This is particularly crucial in healthcare and personal devices.

- Cost-Effective: Arm devices, which are cost-effective and scalable, can now handle sophisticated machine learning tasks, reducing the need for expensive hardware or cloud services.

- Scalability: With billions of Arm devices in the market, TinyML is well-suited for scaling across industries, enabling widespread adoption of AI at the edge.

TinyML is being deployed across multiple industries, enhancing everyday experiences and enabling groundbreaking solutions.

Here are a few examples of TinyML applications:

### Healthcare - Wearable Heart Rate Monitors

- Arm-based microcontrollers like those in Fitbit devices run TinyML models to monitor vital signs such as heart rate, detect arrhythmias, and provide real-time feedback.

- **Example**: Fitbit Charge 5 smart band uses the Arm 32-bit Cortex-M4 processor.

### Agriculture - Smart Irrigation Systems

- Arm-powered microcontrollers in smart sensors help monitor soil moisture and control water usage. TinyML models process environmental data locally to optimize water distribution.

- **Example**: OpenAg uses Arm Cortex-M processors to run machine learning models on edge devices, optimizing irrigation based on real-time data.

### Home Automation - Smart Cameras

- Arm-based processors in smart cameras can detect objects and people, triggering alerts or actions without needing to send data to the cloud, saving bandwidth and improving privacy.

- **Example**: Arlo smart cameras, powered by Arm Cortex processors, perform object detection at the edge, enhancing performance and energy efficiency.

### Industrial IoT - Predictive Maintenance in Factories

- Arm-powered industrial sensors analyze vibration patterns in machinery, running TinyML models to predict when maintenance is needed and prevent breakdowns.

- **Example**: Siemens utilizes Arm Cortex-A processors in industrial sensors for real-time data analysis, detecting faults before they cause significant downtime. They rely on Arm-based processors for their Industrial Edge computing solutions.

### Wildlife Conservation - Smart Camera Traps

- Arm-based smart camera traps can identify animal movements or detect poachers using TinyML models. These energy-efficient devices can operate in remote areas without relying on external power sources.

- **Example**: Conservation X Labs uses Arm Cortex-M microcontrollers to power camera traps, helping monitor endangered species in the wild.

Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,17 @@
# User change
title: "Build a Simple PyTorch Model"

weight: 9 # 1 is first, 2 is second, etc.
weight: 7 # 1 is first, 2 is second, etc.

# Do not modify these elements
layout: "learningpathall"
---

With our Environment ready, we will create a simple program to test our setup. This example will define a simple feedforward neural network for a classification task. The model consists of 2 linear layers with ReLU activation in between. Create a file called simple_nn.py with the following code:
With our Environment ready, you can create a simple program to test the setup.

This example defines a small feedforward neural network for a classification task. The model consists of 2 linear layers with ReLU activation in between.

Use a text editor to create a file named `simple_nn.py` with the following code:

```python
import torch
Expand Down Expand Up @@ -55,45 +59,55 @@ with open("simple_nn.pte", "wb") as file:
print("Model successfully exported to simple_nn.pte")
```

Run it from your terminal:
Run the model from the Linux command line:

```console
python3 simple_nn.py
```

If everything runs successfully, the output will be:
```bash { output_lines = "1" }
The output is:

```output
Model successfully exported to simple_nn.pte
```
Finally, the model is saved as a .pte file, which is the format used by ExecuTorch for deploying models to the edge.

Now, we will run the ExecuTorch version, first run:
The model is saved as a .pte file, which is the format used by ExecuTorch for deploying models to the edge.

Run the ExecuTorch version, first build the executable:

```console
# Clean and configure the build system
rm -rf cmake-out && mkdir cmake-out && cd cmake-out && cmake ..
(rm -rf cmake-out && mkdir cmake-out && cd cmake-out && cmake ..)

# Build the executor_runner target
cmake --build cmake-out --target executor_runner -j9
cmake --build cmake-out --target executor_runner -j$(nproc)
```

You should see an output similar to:
```bash { output_lines = "1" }
You see the build output and it ends with:

```output
[100%] Linking CXX executable executor_runner
[100%] Built target executor_runner
```

Now, run the executor_runner with the Model:
When the build is complete, run the executor_runner with the model as an argument:

```console
./cmake-out/executor_runner --model_path simple_nn.pte
```

Expected Output: Since the model is a simple feedforward model, you can expect a tensor of shape [1, 2]

```bash { output_lines = "1-3" }
Input tensor shape: [1, 10]
Output tensor shape: [1, 2]
Inference output: tensor([[0.5432, -0.3145]]) #will vary due to random initialization
Since the model is a simple feedforward model, you see a tensor of shape [1, 2]

```output
I 00:00:00.006598 executorch:executor_runner.cpp:73] Model file simple_nn.pte is loaded.
I 00:00:00.006628 executorch:executor_runner.cpp:82] Using method forward
I 00:00:00.006635 executorch:executor_runner.cpp:129] Setting up planned buffer 0, size 320.
I 00:00:00.007225 executorch:executor_runner.cpp:152] Method loaded.
I 00:00:00.007237 executorch:executor_runner.cpp:162] Inputs prepared.
I 00:00:00.012885 executorch:executor_runner.cpp:171] Model executed successfully.
I 00:00:00.012896 executorch:executor_runner.cpp:175] 1 outputs:
Output 0: tensor(sizes=[1, 2], [-0.105369, -0.178723])
```

If the model execution completes successfully, you’ll see confirmation messages similar to those above, indicating successful loading, inference, and output tensor shapes.
When the model execution completes successfully, you’ll see confirmation messages similar to those above, indicating successful loading, inference, and output tensor shapes.

Loading