diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 67d666c1c8..4377fcae6c 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,5 +1,5 @@ -Before submitting a pull request for a new Learning Path, please review [Create a Learning Path](https://learn.arm.com//learning-paths/cross-platform/_example-learning-path/) +Before submitting a pull request for a new Learning Path, please review [Create a Learning Path](https://learn.arm.com/learning-paths/cross-platform/_example-learning-path/) - [ ] I have reviewed Create a Learning Path Please do not include any confidential information in your contribution. This includes confidential microarchitecture details and unannounced product information. diff --git a/.wordlist.txt b/.wordlist.txt index 8c457510e9..eb990b7b46 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -4454,4 +4454,9 @@ Zenoh's instrumentable subprocesses CPzfYHdpQ -iso \ No newline at end of file +iso +Arcee's +commandlinetools +ppl +rollout +sdkmanager \ No newline at end of file diff --git a/content/install-guides/aperf.md b/content/install-guides/aperf.md index ba8e0df7bd..0ead65808e 100644 --- a/content/install-guides/aperf.md +++ b/content/install-guides/aperf.md @@ -50,26 +50,26 @@ Visit the [releases page](https://github.com/aws/aperf/releases/) to see a list You can also download a release from the command line: ```bash { target="ubuntu:latest" } -wget https://github.com/aws/aperf/releases/download/v0.1.12-alpha/aperf-v0.1.12-alpha-aarch64.tar.gz +wget https://github.com/aws/aperf/releases/download/v0.1.15-alpha/aperf-v0.1.15-alpha-aarch64.tar.gz ``` Extract the release: ```bash { target="ubuntu:latest" } -tar xvfz aperf-v0.1.12-alpha-aarch64.tar.gz +tar xvfz aperf-v0.1.15-alpha-aarch64.tar.gz ``` Add the path to `aperf` in your `.bashrc` file. ```console -echo 'export PATH="$PATH:$HOME/aperf-v0.1.12-alpha-aarch64"' >> ~/.bashrc +echo 'export PATH="$PATH:$HOME/aperf-v0.1.15-alpha-aarch64"' >> ~/.bashrc source ~/.bashrc ``` Alternatively, you can copy the `aperf` executable to a directory already in your search path. ```bash { target="ubuntu:latest" } -sudo cp aperf-v0.1.12-alpha-aarch64/aperf /usr/local/bin +sudo cp aperf-v0.1.15-alpha-aarch64/aperf /usr/local/bin ``` Confirm `aperf` is installed by printing the version: diff --git a/content/install-guides/go.md b/content/install-guides/go.md index 5794a4d06a..9665cca74d 100644 --- a/content/install-guides/go.md +++ b/content/install-guides/go.md @@ -47,13 +47,13 @@ The easiest way to install Go for Ubuntu on Arm is to download a release, extrac Download a Go release: ```bash { target="ubuntu:latest" } -wget https://go.dev/dl/go1.23.1.linux-arm64.tar.gz +wget https://go.dev/dl/go1.24.5.linux-arm64.tar.gz ``` Extract the release to `/usr/local/go`: ```bash { target="ubuntu:latest" } -sudo tar -C /usr/local -xzf ./go1.23.1.linux-arm64.tar.gz +sudo tar -C /usr/local -xzf ./go1.24.5.linux-arm64.tar.gz ``` Add the path to `go` in your `.bashrc` file. @@ -74,7 +74,7 @@ go version The output should print the version: ```output -go version go1.23.1 linux/arm64 +go version go1.24.5 linux/arm64 ``` You are ready to use the Go programming language on your Arm machine running Ubuntu. diff --git a/content/install-guides/multipass.md b/content/install-guides/multipass.md index e70f0d9063..d1ef439ccc 100644 --- a/content/install-guides/multipass.md +++ b/content/install-guides/multipass.md @@ -19,7 +19,7 @@ minutes_to_complete: 30 author: Jason Andrews ### Link to official documentation -official_docs: https://multipass.run/docs +official_docs: https://documentation.ubuntu.com/multipass/en/latest/ test_images: - ubuntu:latest @@ -53,7 +53,7 @@ Multipass uses the terms virtual machine and instance synonymously. Download Multipass for macOS. ```console -wget https://github.com/canonical/multipass/releases/download/v1.14.1-rc1/multipass-1.14.1-rc1+mac.14+gf2381bfe9.mac-Darwin.pkg +wget https://github.com/canonical/multipass/releases/download/v1.16.0/multipass-1.16.0+mac-Darwin.pkg ``` ### How do I install Multipass on macOS? @@ -61,7 +61,7 @@ wget https://github.com/canonical/multipass/releases/download/v1.14.1-rc1/multip Install the download using the package command. ```console -sudo installer -pkg multipass-1.14.1-rc1+mac.14+gf2381bfe9.mac-Darwin.pkg -target / +sudo installer -pkg multipass-1.16.0+mac-Darwin.pkg -target / ``` The getting started instructions below use the command line interface. If you prefer to use the graphical interface start it from the macOS Launchpad, the initial screen is shown below. You can use the UI to create, start, and stop virtual machines. @@ -130,11 +130,7 @@ If you need to install `snapd` run: sudo apt install snapd -y ``` -LXD is also required for Multipass. -```bash -sudo snap install lxd -``` {{% notice Note %}} You can select from three Multipass releases: stable, beta, or edge. The default version is stable. diff --git a/content/install-guides/nomachine.md b/content/install-guides/nomachine.md index 2b0f1104f0..8d13ddde77 100644 --- a/content/install-guides/nomachine.md +++ b/content/install-guides/nomachine.md @@ -66,8 +66,8 @@ Use a text editor to copy and paste this script into a file on the remote machin #!/bin/bash # install NoMachine for remote desktop -wget https://download.nomachine.com/download/8.4/Arm/nomachine_8.4.2_1_arm64.deb -sudo dpkg -i nomachine_8.4.2_1_arm64.deb +wget https://download.nomachine.com/download/9.0/Arm/nomachine_9.0.188_11_arm64.deb +sudo dpkg -i nomachine_9.0.188_11_arm64.deb if [ $? != 0 ]; then exit 1 fi diff --git a/content/install-guides/openvscode-server.md b/content/install-guides/openvscode-server.md index a0882e20e5..6405293b2a 100644 --- a/content/install-guides/openvscode-server.md +++ b/content/install-guides/openvscode-server.md @@ -53,7 +53,7 @@ Download a release of OpenVSCode Server from the [GitHub release area](https://g For example, use `wget` to download. ```bash -wget https://github.com/gitpod-io/openvscode-server/releases/download/openvscode-server-v1.90.0/openvscode-server-v1.90.0-linux-arm64.tar.gz +wget https://github.com/gitpod-io/openvscode-server/releases/download/openvscode-server-v1.101.2/openvscode-server-v1.101.2-linux-arm64.tar.gz ``` ## How do I install OpenVSCode Server? @@ -61,7 +61,7 @@ wget https://github.com/gitpod-io/openvscode-server/releases/download/openvscode Install the download by extracting the file ```bash -tar xvfz openvscode-server-v1.90.0-linux-arm64.tar.gz +tar xvfz openvscode-server-v1.101.2-linux-arm64.tar.gz ``` ## How do I start OpenVSCode Server? @@ -69,7 +69,7 @@ tar xvfz openvscode-server-v1.90.0-linux-arm64.tar.gz To start OpenVSCode Server run: ```bash -./openvscode-server-v1.90.0-linux-arm64/bin/openvscode-server +./openvscode-server-v1.101.2-linux-arm64/bin/openvscode-server ``` The server will print a URL to access VS Code in a browser. The URL is localhost URL. If your machine is a remote system or a Linux subsystem there are two options to connect using your local browser. @@ -112,7 +112,7 @@ On ChromeOS you can use the Linux configuration settings to automatically do por There are command line options to change the port, the token, and other configuration options. To see the options run: ```bash -./openvscode-server-v1.90.0-linux-arm64/bin/openvscode-server --help +./openvscode-server-v1.101.2-linux-arm64/bin/openvscode-server --help ``` If you are running all on a local machine the token can be eliminated using the `--without-connection-token` option. diff --git a/content/install-guides/powershell.md b/content/install-guides/powershell.md index 10e418d92d..3da5f48aa1 100644 --- a/content/install-guides/powershell.md +++ b/content/install-guides/powershell.md @@ -52,7 +52,7 @@ You can download a release file for the Arm architecture from GitHub and install ```bash { target="ubuntu:latest" } # Download the powershell '.tar.gz' archive -curl -L -o /tmp/powershell.tar.gz https://github.com/PowerShell/PowerShell/releases/download/v7.4.1/powershell-7.4.1-linux-arm64.tar.gz +curl -L -o /tmp/powershell.tar.gz https://github.com/PowerShell/PowerShell/releases/download/v7.5.2/powershell-7.5.2-linux-arm64.tar.gz # Create the target folder where powershell will be placed sudo mkdir -p /opt/microsoft/powershell/7 @@ -88,7 +88,7 @@ pwsh --version The version is printed: ```output -PowerShell 7.4.1 +PowerShell 7.5.2 ``` You are now ready to use PowerShell on your Arm Linux computer. diff --git a/content/install-guides/swift.md b/content/install-guides/swift.md index 8e4e8d7ae9..ec2b3dfa03 100644 --- a/content/install-guides/swift.md +++ b/content/install-guides/swift.md @@ -68,26 +68,26 @@ sudo apt-get -y install \ ## How do I download and install Swift? -This guide uses Swift version 6.0.1 on Ubuntu 24.04. +This guide uses Swift version 6.1.2 on Ubuntu 24.04. You can get more information about other versions and platforms from [Download Swift](https://www.swift.org/download/). Download Swift for Arm Linux: ```bash -wget https://download.swift.org/swift-6.0.1-release/ubuntu2404-aarch64/swift-6.0.1-RELEASE/swift-6.0.1-RELEASE-ubuntu24.04-aarch64.tar.gz +wget https://download.swift.org/swift-6.1.2-release/ubuntu2404-aarch64/swift-6.1.2-RELEASE/swift-6.1.2-RELEASE-ubuntu24.04-aarch64.tar.gz ``` Extract the archive: ```bash -sudo tar -xf swift-6.0.1-RELEASE-ubuntu24.04-aarch64.tar.gz -C /usr/local +sudo tar -xf swift-6.1.2-RELEASE-ubuntu24.04-aarch64.tar.gz -C /usr/local ``` Add the `bin/` directory to your search path: ```bash -echo 'export PATH="$PATH:/usr/local/swift-6.0.1-RELEASE-ubuntu24.04-aarch64/usr/bin"' >> ~/.bashrc +echo 'export PATH="$PATH:/usr/local/swift-6.1.2-RELEASE-ubuntu24.04-aarch64/usr/bin"' >> ~/.bashrc source ~/.bashrc ``` @@ -102,7 +102,7 @@ swift --version The expected output is: ```output -Swift version 6.0.1 (swift-6.0.1-RELEASE) +Swift version 6.1.2 (swift-6.1.2-RELEASE) Target: aarch64-unknown-linux-gnu ``` @@ -116,7 +116,7 @@ print("Hello from Swift on Arm Linux!") Compile and run the program: -```bash +```console swift hello.swift ``` @@ -128,7 +128,7 @@ Hello from Swift on Arm Linux! You can also compile and run the program using: -```bash +```console swiftc hello.swift -o hello ./hello ``` diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md b/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md index 3366b68d39..94c5d71714 100644 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md @@ -88,6 +88,10 @@ This avoids interruptions in later steps when you run `docker compose up` or `do Run the following command in your project directory: ```bash +cd docker +export TIMEOUT=120 +export CONF_FILE=/home/ubuntu/openadkit_demo.autoware/docker/etc/simulation/config/fail_static_obstacle_avoidance.param.yaml +export COMMON_FILE=/home/ubuntu/openadkit_demo.autoware/docker/etc/simulation/config/common.param.yaml docker compose -f docker-compose.yml pull ``` @@ -309,9 +313,8 @@ export COMMON_FILE=$SCRIPT_DIR/etc/simulation/config/common.param.yaml export NGROK_AUTHTOKEN="" export NGROK_URL="" export TIMEOUT=300 - # Launch the container -docker compose -f docker/docker-compose-2ins.yml run --rm planning-control bash +docker compose -f docker-compose-2ins.yml run --rm planning-control bash ``` Once inside the container shell, activate the ROS 2 environment and start publishing to the /hello topic: @@ -330,6 +333,14 @@ This confirms that DDS communication from the planning node is received on the s Same with Publisher side, you need to set the required environment variables and launch the Simulator container. +Navigate to the directory with the Docker Compose file: + +```bash +cd $HOME/openadkit_demo.autoware/docker +``` + +Launch the application: + ```bash export SCRIPT_DIR=/home/ubuntu/openadkit_demo.autoware/docker export CONF_FILE=$SCRIPT_DIR/etc/simulation/config/fail_static_obstacle_avoidance.param.yaml @@ -339,7 +350,7 @@ export NGROK_URL="" export TIMEOUT=300 # Launch the container -docker compose -f docker/docker-compose-2ins.yml run --rm simulator bash +docker compose -f docker-compose-2ins.yml run --rm simulator bash ``` Once inside the container shell, activate the ROS 2 environment and start publishing to the /hello topic: diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md b/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md index fbdd7c7c06..12d69659c5 100644 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md @@ -36,8 +36,8 @@ On each instance, copy the appropriate launch script into the `openadkit_demo.au export NGROK_URL=$NGROK_URL # Start planning-control - echo "Running planning v1.." - TIMEOUT=120 CONF_FILE=$CONF_FILE_PASS docker compose -f "$SCRIPT_DIR/docker-compose-2ins.yml" up planning-control -d + echo "Running planning v1.." + TIMEOUT=120 CONF_FILE=$CONF_FILE_PASS docker compose -f "$SCRIPT_DIR/docker-compose-2ins.yml" up planning-control -d {{< /tab >}} {{< tab header="Visualizer & Simulator" language="bash">}} @@ -85,8 +85,12 @@ On the Simulation and Visualization node, execute: ./opad_sim_vis.sh ``` -Once both machines are running their respective launch scripts, the Visualizer will generate a web-accessible interface using the machine’s public IP address. -You can open this link in a browser to observe the demo behavior. + +Once both machines are running their respective launch scripts, the Visualizer will generate a web-accessible interface on: + +http://[Visualizer public IP address]:6080/vnc.html + +You can open this link in a browser to observe the demo behavior, which will closely resemble the output from the [previous learning path](http://learn.arm.com/learning-paths/automotive/openadkit1_container/4_run_openadkit/). ![img3 alt-text#center](split_aws_run.gif "Figure 4: Simulation") diff --git a/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/1-get-started.md b/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/1-get-started.md index 624344cb32..2c2c7ce60b 100644 --- a/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/1-get-started.md +++ b/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/1-get-started.md @@ -21,7 +21,7 @@ This section walks you through the required tools and two supported setup option ## Download and explore the code examples -To get started, begin by [downloading the code examples](https://gitlab.arm.com/learning-cde-examples/code-examples/-/archive/main/code-examples-main.tar.gz?path=learning-paths/cross-platform/multiplying-matrices-with-sme2). +To get started, begin by [downloading the code examples](https://gitlab.arm.com/learning-code-examples/code-examples/-/archive/main/code-examples-main.tar.gz?path=learning-paths/cross-platform/multiplying-matrices-with-sme2). Now extract the archive, and change directory to: ``code-examples/learning-paths/cross-platform/multiplying-matrices-with-sme2.`` diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/1_intro-zenoh.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/1_intro-zenoh.md index 65d30cc1ea..5e1595325a 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/1_intro-zenoh.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/1_intro-zenoh.md @@ -8,55 +8,43 @@ layout: learningpathall ## The Need for Scalable Communication in Robotics and Edge Computing -Modern robotics and industrial IoT (IIoT) systems are evolving rapidly—from indoor collaborative arms on assembly lines to fleets of outdoor autonomous delivery robots. +Modern robotics and industrial IoT systems are evolving rapidly, from indoor collaborative arms on assembly lines to fleets of outdoor autonomous delivery robots. These applications must operate in real-time, often across multiple compute nodes, over networks that may span factory LANs, 5G cellular links, or even cloud data centers. Such systems require fast, reliable, and scalable data communication between nodes. -This includes not just broadcasting sensor updates or actuator commands (i.e., pub/sub), but also performing state queries, storing values for later use, and even distributed computation across nodes. A modern protocol must be: +This includes not just broadcasting sensor updates or actuator commands (pub/sub), but also performing state queries, storing values for later use, and even distributed computation across nodes. + +A modern protocol must be: * Low-latency: Immediate delivery of time-critical messages. * High-throughput: Efficient data flow across many devices. * Decentralized: No reliance on central brokers or fixed infrastructure. * Flexible: Able to run on lightweight edge devices, across WANs, or inside cloud-native environments. -Traditional communication stacks such as DDS ([Data Distribution Service](https://en.wikipedia.org/wiki/Data_Distribution_Service)) serve as the backbone for middleware like ROS 2. However, DDS struggles in multi-network or wireless environments where multicast is unavailable, or NAT traversal is needed. +Traditional communication stacks such as Data Distribution Service (DDS) serve as the backbone for middleware like ROS 2. However, DDS struggles in multi-network or wireless environments where multicast is unavailable, or NAT traversal is needed. These constraints can severely impact deployment and performance at the edge. -## Zenoh: An Open-Source Pub/Sub Protocol for the Industrial Edge +## Zenoh: an open-source pub/sub protocol for the Industrial Edge -[Eclipse Zenoh](https://zenoh.io/) is a modern, [open-source](https://github.com/eclipse-zenoh/zenoh) data-centric communication protocol that goes beyond traditional pub/sub. Designed specifically for edge computing, IIoT, robotics, and autonomous systems, Zenoh unifies: +[Eclipse Zenoh](https://zenoh.io/) is a modern, open-source, data-centric communication protocol that goes beyond traditional pub/sub. Designed specifically for edge computing, industrial IoT, robotics, and autonomous systems, Zenoh unifies: -* Data in motion through a powerful and efficient pub/sub model. -* Data at rest via geo-distributed storage plugins. -* Data in use with direct query support. -* On-demand computation via queryable nodes that can generate data dynamically. +- Data in motion through a powerful and efficient pub/sub model. +- Data at rest via geo-distributed storage plugins. +- Data in use with direct query support. +- On-demand computation via queryable nodes that can generate data dynamically. Unlike most traditional stacks, Zenoh is fully decentralized and designed to operate across cloud-to-edge-to-thing topologies, making it ideal for industrial robotics, autonomous systems, and smart environments. + It supports heterogeneous platforms, is implemented in Rust for performance and safety, and also offers bindings for Python, enabling rapid prototyping. Zenoh is particularly effective in wireless, 5G, or cross-network deployments where multicast and DDS fall short. -Its routing engine avoids excessive discovery traffic, conserves bandwidth, and supports seamless bridging between legacy ROS 2/DDS apps and modern, optimized Zenoh networks using zenoh-bridge-dds. - -In this learning path, you’ll use Zenoh to build and validate a multi-node distributed communication system across multiple Arm-based platforms, gaining hands-on experience with data exchange and coordination between edge devices. - -To make the upcoming demo more intuitive and easy to follow, we’ll demonstrate the setup using two physical Cortex-A Linux devices. - -I’ll be using Raspberry Pi boards in this learning path, but you’re free to substitute them with any Cortex-A devices that support network connectivity with Linux-based OS installed, depending on your development setup. - -In real-world ROS 2 deployment scenarios, developers typically conduct validation and performance testing across systems with more than two nodes. -To simulate such environments, using [Arm virtual hardware](https://www.arm.com/products/development-tools/simulation/virtual-hardware) is also a common and efficient approach. - -This will help you quickly validate your architecture choices and communication patterns when designing distributed applications. - -* Raspberry Pi, -* Linux-based Cortex-A, or -* Arm Virtual Hardware (AVH). +Its routing engine avoids excessive discovery traffic, conserves bandwidth, and supports seamless bridging between legacy ROS 2/DDS apps and modern, optimized Zenoh networks using Zenoh-Bridge-DDS, a bridge between DDS systems and Zenoh networks. -After this learning path, you will: -* Understand the core architecture and data flow principles behind Eclipse Zenoh, including its support for pub/sub, querying, and queryable edge functions. -* Build and run distributed Zenoh examples across multiple Arm-based nodes—using Raspberry Pi or AVH to simulate scalable deployment environments. -* Rebuild and extend the Zenoh queryable example to simulate edge-side logic. +In this Learning Path, you’ll use Zenoh to build and validate a multi-node distributed communication system across multiple Arm-based platforms, gaining hands-on experience with data exchange and coordination between edge devices. -By the end of this learning path, you’ll have deployed a fully functional, scalable, and latency-aware Zenoh system. +The examples use Raspberry Pi boards, but you’re free to substitute any Cortex-A or Neoverse devices that support network connectivity running Linux. -You can also check [here](https://zenoh.io/docs/getting-started/first-app) to find some simple examples. +Upon completion, you will be able to: +- Understand the core architecture and data flow principles behind Eclipse Zenoh, including its support for pub/sub, querying, and queryable edge functions. +- Build and run distributed Zenoh examples across multiple Arm-based nodes using Raspberry Pi or other Arm Linux devices. +- Rebuild and extend the Zenoh queryable example to simulate edge-side logic. diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/2_zenoh-install.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/2_zenoh-install.md index 5a750202ec..da1a28cbe4 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/2_zenoh-install.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/2_zenoh-install.md @@ -8,56 +8,59 @@ layout: learningpathall ## Setting Up Zenoh on Arm Devices -The following instructions are verified on both Raspberry Pi 4/5 and Arm Virtual Hardware, but you can implement them on any Cortex-A Linux device. +The following instructions have been verified on both Raspberry Pi 4 and 5 devices, but you can implement them on any Arm Linux device. Before building Zenoh, make sure your system has the necessary development tools and runtime libraries. -### Install the Rust build environment +### Install the Rust development environment -First, we need to install the [Rust](https://www.rust-lang.org/) build environment, since the core of Zenoh is totally developed using Rust to keep it safe and efficient. - -Follow this [installation guide](https://learn.arm.com/install-guides/rust/) to install Rust and Cargo on Arm Linux, a build system for Rust. Or, simply use the following commands. +First, install the [Rust](https://www.rust-lang.org/) environment, since the core of Zenoh is developed using Rust to keep it safe and efficient. ```bash -curl https://sh.rustup.rs -sSf | sh +sudo apt update +sudo apt install -y curl gcc +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y ``` -Follow the prompts to complete the installation. If successful, you’ll see: +Near the end of the installation you will see the success message: ```output Rust is installed now. Great! ``` -For more details, refer to [Rust’s official install guide.](https://doc.rust-lang.org/cargo/getting-started/installation.html#install-rust-and-cargo) +Make sure to source the environment to add Rust to your shell environment: -### Install ROS 2 +```bash +source "$HOME/.cargo/env" +``` -[Robot Operating System](https://www.ros.org/) is a set of software libraries and tools that help you build robot applications. From drivers to state-of-the-art algorithms, and with powerful developer tools, ROS has what you need for your next robotics project. And it's all open source. +You can learn more using the [Rust install guide](/install-guides/rust/) for Arm Linux. -Since ROS was started in 2007, a lot has changed in the robotics and ROS community. The goal of the [ROS 2](https://docs.ros.org/en/rolling/index.html) project is to adapt to these changes, leveraging what is great about ROS 1 and improving what isn’t. +### Install ROS 2 -Here is the quick [installation guide](https://learn.arm.com/install-guides/ros2/) about how to install ROS 2 in Arm platform. +[Robot Operating System](https://www.ros.org/) is a set of software libraries and tools that help you build robot applications. ROS provides everything from drivers to state-of-the-art algorithms, as well as developer tools. It is completely open-source. + +Follow the [ROS2 installation guide](/install-guides/ros2/) to install ROS 2 on your Arm platforms. ### Download and build the Zenoh source -Now, we can clone the Zenoh. +Clone the Zenoh repository: ```bash -cd ~ +cd $HOME git clone https://github.com/eclipse-zenoh/zenoh.git ``` -After that, simply use cargo to build the source. +After cloning, use cargo to build the source: ```bash cd zenoh cargo build --release --all-targets -j $(nproc) ``` -This will take several minutes depending on your device. Once the installation is complete, you should see: +This process will take several minutes depending on your device. Once the installation is complete, you should see: ```output -cargo build --release --all-targets -j $(nproc) Updating crates.io index Downloaded humantime v2.2.0 Downloaded spin v0.10.0 @@ -101,7 +104,26 @@ This may become a hard error in the future; see plugins/zenoh-plugin-storage-manager/tests/operations.rs:91:55 +``` + +Edit the file `./plugins/zenoh-plugin-storage-manager/tests/operations.rs` and comment the line shown below and add the second line to the file: + +```rust +//use zenoh_plugin_trait::Plugin; +use crate::path::to::Plugin; +``` + +Run the build again: + +```bash +cargo clean && cargo build +``` + With Zenoh successfully compiled, you’re ready to explore how nodes communicate using the Zenoh runtime. diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/3_zenoh-multinode.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/3_zenoh-multinode.md index 8d21f0bfc0..06e17d6012 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/3_zenoh-multinode.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/3_zenoh-multinode.md @@ -6,65 +6,110 @@ weight: 4 layout: learningpathall --- -## Deploying Zenoh on Multiple Raspberry Pi Devices Using Docker +## Deploying Zenoh on Multiple Raspberry Pi Devices -After building Zenoh and its core examples, your next step is to deploy them across multiple Arm-based devices. +After building Zenoh and its core examples, the next step is to deploy them across multiple Arm-based devices. -In this session, you’ll use Raspberry Pi boards to simulate a scalable, distributed environment—but the same workflow applies to any Arm Linux system, including Arm cloud instances and Arm Virtual Hardware. +If you’ve already installed Zenoh on an Arm Cortex-A or Neoverse platform as shown in the previous section, you can simply copy the compiled binaries from `~/zenoh/target/release/` to each of your Raspberry Pi devices. -You’ll learn how to use Docker to deploy the environment on physical devices, and how to duplicate virtual instances using snapshot cloning on Arm Virtual Hardware. +However, to streamline deployment across multiple devices and ensure repeatability, this section demonstrates how to package Zenoh into a Docker image for batch rollout and scalable testing. -This setup lets you simulate `real-world`, `cross-node communication`, making it ideal for validating Zenoh's performance in robotics and industrial IoT use cases. +This containerized approach not only simplifies deployment on Raspberry Pi, but also integrates seamlessly with Arm cloud platforms such as AWS Graviton Arm Cortex-A Linux or Arm Virtual Hardware—enabling a consistent cloud-to-edge development and validation workflow. -### Install Docker on Raspberry Pi +In this session, you’ll use Raspberry Pi boards to simulate a scalable distributed environment. The same workflow applies to any Arm Linux system, including cloud instances and virtual hardware. + +This setup allows you to simulate real-world, cross-node communication scenarios, making it ideal for evaluating Zenoh’s performance in robotics and industrial IoT applications. -To simplify this process and ensure consistency, you’ll use Docker to containerize your Zenoh and ROS 2 environment. +### Install Docker on Raspberry Pi +To simplify this process and ensure consistency, you can use Docker to containerize your Zenoh and ROS 2 environment. This lets you quickly replicate the same runtime on any device without needing to rebuild from source. This enables multi-node testing and real-world distributed communication scenarios. -First, install the docker environment on each of Raspberry Pi if you don't have that. +First, install Docker on each of Raspberry Pi. ```bash -curl -sSL https://get.docker.com | sh -sudo usermod -aG docker pi +curl -fsSL get.docker.com -o get-docker.sh && sh get-docker.sh +sudo usermod -aG docker $USER ; newgrp docker ``` -Log out and back in, or run newgrp docker to activate Docker group permissions. +### Create a ROS2 + Zenoh Docker Image -### Create a ROS 2 + DDS Docker Image +To ensure compatibility with ROS-related tools, create a `Dockerfile` based on `ros:galactic `, and use the official Rust installation method to build Zenoh, as shown below. -In a working directory, create a `Dockerfile` with the following content to create the ROS 2 / DDS docker image. +This Dockerfile uses a multi-stage build process based on the ros:galactic environment. +In the first stage, it installs Rust and compiles the Zenoh binaries directly from source. +In the second stage, it installs essential ROS 2 demo tools and copies the Zenoh executables into the final runtime image. ```bash -FROM ros:galactic -RUN apt-get update -RUN apt-get install -y ros-galactic-demo-nodes-cpp ros-galactic-rmw-cyclonedds-cpp ros-galactic-turtlesim +# Stage 1: Build Zenoh using ROS base with Rust toolchain +FROM ros:galactic AS builder + +RUN apt-get update && apt-get install -y \ + curl \ + git \ + build-essential \ + pkg-config \ + clang \ + libssl-dev \ + cmake + +RUN curl -sSf https://sh.rustup.rs -o rustup-init.sh && \ + chmod +x rustup-init.sh && \ + ./rustup-init.sh -y --no-modify-path && \ + rm rustup-init.sh + +ENV PATH="/root/.cargo/bin:${PATH}" + +WORKDIR /root +RUN git clone https://github.com/eclipse-zenoh/zenoh.git +WORKDIR /root/zenoh +RUN cargo build --release --all-targets -j $(nproc) + +# Stage 2: Runtime with ROS + Zenoh binary only +FROM ros:galactic AS runtime + +RUN apt-get update && apt-get install -y \ + ros-galactic-demo-nodes-cpp \ + ros-galactic-rmw-cyclonedds-cpp \ + ros-galactic-turtlesim + +COPY --from=builder /root/zenoh/target/release /root/zenoh/target/release + ENV RMW_IMPLEMENTATION=rmw_cyclonedds_cpp -CMD bash + +WORKDIR /root/zenoh/target/release + +CMD ["/bin/bash"] ``` Under the directory where the above Dockerfile exists, run the following command to generate the docker image. ```bash -$ docker build -t zenoh-node . +docker build -t zenoh-node . ``` -After this has been done, the created ROS 2 docker image can be seen by the following command. +After this has been done, the created ROS 2 Docker image can be seen by the following command. ```bash -$ docker images | grep zenoh-node +docker images | grep zenoh-node ``` ```output -zenoh-node latest b7a9c27cf8a8 About a minute ago 962MB +REPOSITORY TAG IMAGE ID CREATED SIZE +zenoh-node latest 2300ea78d043 30 minutes ago 3.73GB ``` -### Transfer the Docker Image on Another RPi +Alternatively, if you’d like to skip the build process, a pre-built Docker image is available on Docker Hub. +You can pull it directly using: + +```bash +docker pull odinlmshen/zenoh-node +``` -You now need to transfer the Docker image to your second device. Choose one of the following methods: +### Transfer the Docker image to the other Raspberry Pi -You have two options: +There are two options to transfer the Docker image to your second device. Choose one of the following methods. Option 1: Save and copy via file @@ -74,31 +119,24 @@ scp zenoh-node.tar pi@:/home/pi/ ``` On the target device: + ```bash docker load < zenoh-node.tar ``` -Option 2: Push to a container registry (e.g., DockerHub or GHCR). +Option 2: Push the image to a container registry such as Docker Hub You can also push the image to Docker Hub or GitHub Container Registry and pull it on the second device. ### Run the Docker Image -Once the image is successfully loaded into second device, you can run the container by +Once the image is successfully loaded on the second device, you can run the container to start the Zenoh environment. ```bash docker run -it --network=host zenoh-node ``` -Now, all the Zenoh example binaries are now available within this container, allowing you to test pub/sub and query flows across devices. - -### Another Duplicate Setting Option on Arm Virtual Hardware - -If you have [Corellium](https://www.corellium.com/) account, you can - -1. Set up and install Zenoh on a single AVH instance. -2. Use the [Clone](https://support.corellium.com/features/snapshots) function to duplicate the environment. -3. Optionally, you may optionally rename the device to avh* for easy device recognition by changing the setting in the `/etc/hostname` file. +The Zenoh example binaries are now available within this container, allowing you to test pub/sub and query flows across devices. ## Run Zenoh in Multi-Node Environment diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/4_zenoh-ex1-pubsub.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/4_zenoh-ex1-pubsub.md index fc0115618a..6075c7cf3e 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/4_zenoh-ex1-pubsub.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/4_zenoh-ex1-pubsub.md @@ -6,34 +6,36 @@ weight: 5 layout: learningpathall --- -## Example 1: Simple Pub/Sub +## Example 1: Simple pub/sub -This first test demonstrates real-time publish/subscribe model using two Raspberry Pi devices. +This first test demonstrates the real-time publish/subscribe model using two Raspberry Pi devices. -The following command is to initiate a subscriber for a key expression `demo/example/**`, i.e. a set of topics starting with the path `demo/example`. +The following command is to initiate a subscriber for a key expression `demo/example/**`, a set of topics starting with the path `demo/example`. -### Step 1: Run Subscriber +### Step 1: Run subscriber -Log in to Pi using any of the methods: +Run the subscriber example on one of the Raspberry Pi systems. ```bash cd ~/zenoh/target/release/examples ./z_sub ``` -### Step 2: Run Publisher +### Step 2: Run publisher -Then, log in to another machine Pi. +Then, log in to the other Raspberry Pi and run the publisher. ```bash cd ~/zenoh/target/release/examples ./z_pub ``` -The result will look like: +The result is shown below: + ![img1 alt-text#center](zenoh_ex1.gif "Figure 1: Simple Pub/Sub") -In the left-side window, I have logged into the device Pi4 and run the z_sub program. -It receives values with the key `demo/example/zenoh-rs-pub` continuously published by z_pub running on Pi in the right-side window. +The left-side window shows the `z_sub` program. + +It receives values with the key `demo/example/zenoh-rs-pub` continuously published by `z_pub` running in the right-side window. This basic example shows Zenoh's zero-config discovery and low-latency pub/sub across physical nodes. diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/5_zenoh-ex2-storagequery.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/5_zenoh-ex2-storagequery.md index a0cdea6cef..4de6612a67 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/5_zenoh-ex2-storagequery.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/5_zenoh-ex2-storagequery.md @@ -6,24 +6,23 @@ weight: 6 layout: learningpathall --- -## Example 2: Storage and Query +## Example 2: Storage and query -The second example adds Zenoh's data storage and querying capabilities—enabling nodes to retrieve historical values on demand. +The second example adds Zenoh's data storage and querying capabilities by enabling nodes to retrieve historical values on demand. -Building on the previous Pub/Sub example, you’ll now explore how Zenoh supports `persistent data storage` and `on-demand querying` -- a powerful feature for robotics and IIoT applications. +Building on the previous pub/sub example, you’ll now explore how Zenoh supports persistent data storage and on-demand querying, a powerful feature for robotics and IIoT applications. -In a typical warehouse or factory scenario, autonomous robots may periodically publish sensor data (e.g., position, temperature, battery level), and a central system—or another robot—may later need to query the latest state of each unit. +In a typical warehouse or factory scenario, autonomous robots may periodically publish sensor data such as position, temperature, battery level, and a central system or another robot—may later need to query the latest state of each unit. Unlike Pub/Sub, which requires live, real-time message exchange, Zenoh's storage and query model enables asynchronous access to data that was published earlier, even if the original publisher is no longer online. -In this example, you’ll run the zenohd daemon with in-memory storage and use z_put to publish data and z_get to retrieve it. +In this example, you’ll run the `zenohd` daemon with in-memory storage and use `z_put` to publish data and `z_get` to retrieve it. This is especially useful for distributed systems where nodes may intermittently connect or request snapshots of state from peers. +### Step 1: Start the Zenoh daemon with in-memory storage -### Step 1: Start the Zenoh Daemon with In-Memory Storage - -On one Raspberry Pi, launch the Zenoh daemon with a configuration that enables in-memory storage for keys under demo/example/**: +On one Raspberry Pi, launch the Zenoh daemon with a configuration that enables in-memory storage for keys in the `demo/example/**` directory. ```bash cd ~/zenoh/target/release/ @@ -33,11 +32,12 @@ cd ~/zenoh/target/release/ This starts the Zenoh daemon with in-memory storage support. You should see log messages indicating that the storage_manager plugin is loaded. -If port 7447 is already in use, either stop any previous Zenoh processes or configure a custom port using the listen.endpoints.router setting. -### Step 2: Publish Data +If port 7447 is already in use, either stop any previous Zenoh processes or configure a custom port using the `listen.endpoints.router` setting. + +### Step 2: Publish data -On 2nd device, use z_put to send a key-value pair that will be handled by the zenohd storage: +On 2nd Raspberry Pi device, use `z_put` to send a key-value pair that will be handled by the `zenohd` storage. ```bash cd ~/zenoh/target/release/examples @@ -46,10 +46,9 @@ cd ~/zenoh/target/release/examples This command stores the string `Hello from storage!` under the key demo/example/test1. +### Step 3: Query the data -### Step 3: Query the Data - -Back on first Raspberry Pi, you can now query the stored data from any Zenoh-connected node: +Back on first Raspberry Pi, you can now query the stored data from any Zenoh connected node. ```bash cd ~/zenoh/target/release/examples @@ -63,12 +62,13 @@ Sending Query 'demo/example/test1'... >> Received ('demo/example/test1': 'Hello from storage!') ``` -The result will look like: +The result is shown below: + ![img2 alt-text#center](zenoh_ex2.gif "Figure 2: Storage and Query") {{% notice tip %}} -If you have more than two Raspberry Pi devices, you can run the z_get command on a third RPi to validate that storage queries work seamlessly across a multi-node setup. +If you have more than two Raspberry Pi devices, you can run the `z_get` command on a third device to validate that storage queries work seamlessly across a multi-node setup. {{% /notice %}} -This example shows how Zenoh's Storage + Query model supports asynchronous data access and resilient state-sharing—critical capabilities in robotics and industrial IoT systems where network connectivity may be intermittent or system components loosely coupled. +This example shows how Zenoh's storage with query model supports asynchronous data access and resilient state-sharing—critical capabilities in robotics and industrial IoT systems where network connectivity may be intermittent or system components loosely coupled. diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/6_zenoh-ex3-queryable.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/6_zenoh-ex3-queryable.md index 13f15e0020..ed9d07ea5e 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/6_zenoh-ex3-queryable.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/6_zenoh-ex3-queryable.md @@ -6,41 +6,40 @@ weight: 7 layout: learningpathall --- -## Example 3: Computation on Query using Queryable +## Example 3: Computation on query -Next, you’ll explore Zenoh's queryable capability, which lets a node dynamically respond to data queries by executing a custom computation or data generation function in this example. +Next, you’ll explore Zenoh's queryable capability, which lets a node dynamically respond to data queries by executing a custom computation or data generation function. -Unlike zenohd which simply returns stored data, a queryable node can register to handle a specific key expression and generate responses at runtime. This is ideal for distributed computing at the edge, where lightweight devices—such as Raspberry Pi nodes—can respond to requests with calculated values (e.g., sensor fusion, AI inference results, or diagnostics). +Unlike `zenohd` which simply returns stored data, a queryable node can register to handle a specific key expression and generate responses at runtime. This is ideal for distributed computing at the edge, where lightweight devices, such as Raspberry Pi nodes, can respond to requests with calculated values. This enables sensor fusion, AI inference results, and diagnostics. -### Use Case: On-Demand Battery Health Estimation +### Use Case: On-Demand battery health estimation Imagine a robot fleet management system where the central planner queries each robot for its latest battery health score, which is not published continuously but calculated only when queried. This saves bandwidth and enables edge compute optimization using Zenoh's Queryable. -### Step 1: Launch a Queryable Node +### Step 1: Launch a queryable node -On one Raspberry Pi device, run the built-in Zenoh example to register a queryable handler. +On one Raspberry Pi device, run the `z_queryable` Zenoh example to register a queryable handler. ```bash cd ~/zenoh/target/release/examples ./z_queryable ``` -You'll see the output like: +You'll see the output: -``` -pi@raspberrypi:~/zenoh/target/release/examples$ ./z_queryable +```output Opening session... Declaring Queryable on 'demo/example/zenoh-rs-queryable'... Press CTRL-C to quit... ``` -The node is now ready to accept queries on the key demo/example/zenoh-rs-queryable and respond with a predefined message. +The node is now ready to accept queries on the key `demo/example/zenoh-rs-queryable` and respond with a predefined message. -### Step 2: Trigger a Query from Another Node +### Step 2: Trigger a query from another node -On another Raspberry Pi device, run: +On the other Raspberry Pi device, run the `z_get` example. ```bash cd ~/zenoh/target/release/examples @@ -49,25 +48,25 @@ cd ~/zenoh/target/release/examples You should see: -``` -./z_get -s demo/example/zenoh-rs-queryable +```output Opening session... Sending Query 'demo/example/zenoh-rs-queryable'... >> Received ('demo/example/zenoh-rs-queryable': 'Queryable from Rust!') ``` -The result will look like: +The result is shown below: + ![img3 alt-text#center](zenoh_ex3.gif "Figure 3: Computation on Query using Queryable") The value you receive comes not from storage, but from the computation inside the queryable handler. -### Real-World Application: Distributed Inference & Computation +### Real-world application: distributed inference and computation This model enables edge-based intelligence, such as: -- Executing custom logic in response to a query (e.g., “calculate load average”) -- Triggering ML inference (e.g., “classify image X on demand”) -- Decentralized diagnostics (e.g., “report actuator status”) +- Executing custom logic in response to a query such as “calculate load average” +- Triggering ML inference such as “classify image X on demand” +- Decentralized diagnostics such as “report actuator status” Queryable is a key feature for data-in-use scenarios, allowing fine-grained, on-demand compute inside your Zenoh-powered architecture. -Next, you’ll extend this Queryable pattern to perform parameterized computation — simulating edge diagnostics and adaptive inference. \ No newline at end of file +Next, you’ll extend this Queryable pattern to perform parameterized computation, simulating edge diagnostics and adaptive inference. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/7_zenoh-querycomp.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/7_zenoh-querycomp.md index a870fe5b54..1afd3c1103 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/7_zenoh-querycomp.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/7_zenoh-querycomp.md @@ -6,7 +6,7 @@ weight: 8 layout: learningpathall --- -## Parameterized Battery Health Estimation with Zenoh Queryable +## Parameterized battery health estimation with Zenoh queryable Finally, you’ll combine pub/sub, storage, and queryable components to simulate a distributed computation flow—demonstrating how Zenoh enables intelligent, coordinated edge systems. @@ -14,15 +14,15 @@ You’ll learn how to use Zenoh's Queryable API in Rust to build a parameterized This extends a previous example by supporting runtime query parameters like battery level and temperature. -## Use Case: Real-time Battery Health via Computation +## Use Case: Real-time battery health via computation In robotic fleet management, a central controller may need to assess each robot’s battery health on demand. Instead of streaming data continuously, robots expose a queryable endpoint that returns a real-time health score based on current battery level and temperature. -This saves bandwidth and enables lightweight edge-side decision-making.” +This saves bandwidth and enables lightweight edge-side decision-making. -### Step 1: Create a New Zenoh Rust Project +### Step 1: Create a new Zenoh rust project On any Raspberry Pi: @@ -31,20 +31,20 @@ cd ~/zenoh cargo new zenoh_battery_estimator ``` -Update following `dependencies` setting at ~/zenoh/zenoh_battery_estimator/Cargo.toml. +Update following dependencies setting by editing the file `$HOME/zenoh/zenoh_battery_estimator/Cargo.toml` -``` +```toml [dependencies] zenoh = { path = "../zenoh" } tokio = { version = "1", features = ["full"] } url = "2" ``` -### Step 2: Implement the Queryable Node +### Step 2: Implement the queryable node -Then, log in to another machine Pi. +Next, log in to the other Raspberry Pi. -Replace the contents of ~/zenoh/zenoh_battery_estimator/src/main.rs with: +Replace the contents of `$HOME/zenoh/zenoh_battery_estimator/src/main.rs` with the code below. ```rust use zenoh::{open, Config}; @@ -103,64 +103,69 @@ Inside the callback: - A health score is computed from these inputs. - The result is sent back to the querying client using query.reply(). -This design pattern enables efficient, on-demand data exchange with minimal bandwidth usage—ideal for edge computing scenarios where resources and connectivity are constrained. +This design pattern enables efficient, on-demand data exchange with minimal bandwidth usage. This is ideal for edge computing scenarios where resources and connectivity are constrained. The health score is calculated using the following logic: ```rust let health_score = 100 - (100 - battery) - ((temp.saturating_sub(25)) / 2); ``` + This formula estimates battery health as a percentage, considering both battery level and temperature: - battery: Current battery level (default 50%) - temp: Current temperature (default 25°C) The health estimation logic begins with the battery level as the baseline score. If the temperature rises above 25°C, the score is adjusted downward—specifically, for every 2°C above this threshold, the health is reduced by 1%. + To ensure the calculation remains safe even when the temperature is below 25°C, the code uses saturating_sub(25), which prevents the result from becoming negative and avoids potential underflow errors. For example, if battery = 88 and temp = 32, then: - Temperature offset = (32 - 25) / 2 = 3 - Health = 88 - 3 = 85% -### Step 3: Build and Run +### Step 3: Build and run ```bash -cd ~/zenoh/zenoh_battery_estimator +cd $HOME/zenoh/zenoh_battery_estimator cargo build --release ``` After the build process, you will see: -``` -cargo build --release +```output Compiling zenoh_battery_estimator v1.4.0 (/home/ubuntu/zenoh_v1.4/zenoh_battery_estimator) Finished `release` profile [optimized] target(s) in 1m 22s ``` -### Step 4: Query It with Parameters +### Step 4: Query it with parameters + +Run it on the Raspberry Pi you used for the build run: -Run it on the Raspberry Pi you just built. ```bash -cd ~/zenoh/target/release/ +cd $HOME/zenoh/target/release/ ./zenoh_battery_estimator ``` -You can reuse the built-in Zenoh z_get CLI in another Raspberry Pi. +You can reuse the built-in Zenoh `z_get` CLI from the other Raspberry Pi. ```bash cd ~/zenoh/target/release/examples ./z_get -s "robot/battery/estimate?level=88&temp=32" ``` -The result will look like: +The result is shown below: + ![img4 alt-text#center](zenoh_ex4.gif "Figure 4: Dynamic Queryable with Computation") -The excepted output will be -``` +The excepted output is: + +```output >> Received ('robot/battery/estimate': 'Estimated battery health: 85%') ``` -You’ve just built a responsive, parameterized edge compute service using Zenoh's Queryable API in Rust — a lightweight but powerful pattern for real-time intelligence at the edge. +You’ve just built a responsive, parameterized edge compute service using Zenoh's Queryable API in Rust. It's lightweight but a powerful pattern for real-time intelligence at the edge. This approach not only minimizes network overhead but also enables each device to process and respond to context-aware queries on demand. + It’s a strong foundation for building scalable, event-driven IoT systems that can adapt dynamically to operational needs. diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/_index.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/_index.md index 48fa9c5db9..833978a14d 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/_index.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/_index.md @@ -7,18 +7,16 @@ cascade: minutes_to_complete: 45 -who_is_this_for: This learning path is designed for robotics developers, industrial automation engineers, and IoT system architects building distributed, scalable, and low-latency applications. Whether you are using Robot Operating System (ROS), developing autonomous systems, or designing multi-node communication frameworks, this guide will show you how to leverage the Eclipse Zenoh protocol on Arm-based platforms — both in the cloud (AVH or EC2) and on physical devices like Raspberry Pi. +who_is_this_for: This Learning Path is for robotics developers, industrial automation engineers, and IoT system architects building distributed, scalable, and low-latency applications. Whether you are using Robot Operating System (ROS), developing autonomous systems, or designing multi-node communication frameworks, you can use Eclipse Zenoh on Arm-based platforms, both in the cloud and on local devices like Raspberry Pi. learning_objectives: - Understand Zenoh's architecture and its integration of pub/sub, storage, querying, and computation models. - Build and run Zenoh examples on both Arm servers and Raspberry Pi. - - Set up and deploy a multi-node Zenoh system using Arm-based hardware or virtual environments. + - Set up and deploy a multi-node Zenoh system. prerequisites: - - At least two [Raspberry Pi5 or Pi4](https://www.raspberrypi.com/products/raspberry-pi-5/) or other Cortex-A instances with a Linux-based OS installed. - - Basic understanding with the Linux command line. + - At least two local Cortex-A devices running Linux, such as Raspberry Pi 4 or Pi 5. You can also use Arm servers or cloud instances. - Experience with ROS 2 applications. - - Corellium account for virtual hardware testing. (Option) author: - Odin Shen diff --git a/content/learning-paths/embedded-and-microcontrollers/edge/_index.md b/content/learning-paths/embedded-and-microcontrollers/edge/_index.md index 982192d8a1..a0fa9edcf2 100644 --- a/content/learning-paths/embedded-and-microcontrollers/edge/_index.md +++ b/content/learning-paths/embedded-and-microcontrollers/edge/_index.md @@ -1,25 +1,21 @@ --- -title: Learn How to Run AI on Edge Devices- Arduino Nano RP2040 - -draft: true -cascade: - draft: true +title: Learn how to run AI on Edge devices using Arduino Nano RP2040 minutes_to_complete: 90 -who_is_this_for: This learning path is for beginners in Edge AI and TinyML, including developers, engineers, hobbyists, AI/ML enthusiasts, and researchers working with embedded AI and IoT. +who_is_this_for: This Learning Path is for beginners in Edge AI and TinyML, including developers, engineers, hobbyists, AI/ML enthusiasts, and researchers working with embedded AI and IoT. learning_objectives: - - Understand Edge AI and TinyML basics. - - Collect and preprocess audio data using Edge Impulse. - - Train and deploy an audio classification model on Arduino Nano RP2040. - - Interface with LEDs to switch them on and off. + - Understand the basics of Edge AI and TinyML. + - Collect and preprocess audio data using Edge Impulse. + - Train and deploy an audio classification model on the Arduino Nano RP2040. + - Control LEDs by turning them on and off based on model predictions. prerequisites: - - Explore this [learning path](https://learn.arm.com/learning-paths/embedded-and-microcontrollers/arduino-pico/) if you are an absolute beginner. - - An [Edge Impulse](https://edgeimpulse.com/) Studio account. - - The [Arduino IDE with the RP2040 board support package](https://learn.arm.com/install-guides/arduino-pico/) installed on your computer - - An Arduino Nano RP2040 Connect [board](https://store.arduino.cc/products/arduino-nano-rp2040-connect-with-headers?_gl=1*9t4cti*_up*MQ..*_ga*NTA1NTQwNzgxLjE3NDYwMjIyODk.*_ga_NEXN8H46L5*MTc0NjAyMjI4Ny4xLjEuMTc0NjAyMjMxOC4wLjAuMjA3MjA2NTUzMA..). + - Completion of [Embedded programming with Arduino on the Raspberry Pi Pico](https://learn.arm.com/learning-paths/embedded-and-microcontrollers/arduino-pico/) if you're an absolute beginner. + - An [Edge Impulse Studio](https://studio.edgeimpulse.com/signup) account. + - The [Arduino IDE](/install-guides/arduino-pico/) with the RP2040 board support package installed on your computer. + - An [Arduino Nano RP2040 Connect board](https://store.arduino.cc/products/arduino-nano-rp2040-connect-with-headers). author: Bright Edudzi Gershon Kordorwu ### Tags @@ -38,15 +34,15 @@ operatingsystems: further_reading: - resource: - title: TinyML Brings AI to Smallest Arm Devices + title: TinyML brings AI to smallest Arm devices link: https://newsroom.arm.com/blog/tinyml type: blog - resource: - title: What is edge AI? + title: What is Edge AI? link: https://docs.edgeimpulse.com/nordic/concepts/edge-ai/what-is-edge-ai type: blog - resource: - title: Edge Impulse for Beginners + title: Edge Impulse for beginners link: https://docs.edgeimpulse.com/docs/readme/for-beginners type: doc diff --git a/content/learning-paths/embedded-and-microcontrollers/edge/connect-and-set-up-arduino.md b/content/learning-paths/embedded-and-microcontrollers/edge/connect-and-set-up-arduino.md index 9232e6a543..f9bb54bc0e 100644 --- a/content/learning-paths/embedded-and-microcontrollers/edge/connect-and-set-up-arduino.md +++ b/content/learning-paths/embedded-and-microcontrollers/edge/connect-and-set-up-arduino.md @@ -1,69 +1,72 @@ --- -title: Board Connection and IDE setup +title: Board connection and IDE setup weight: 4 ### FIXED, DO NOT MODIFY layout: learningpathall --- -### Arduino Nano RP2040 +## Arduino Nano RP2040 -To get started with your first **TinyML project**, the **Arduino Nano RP2040 Connect** is a good option. Built by Arduino, it uses the **RP2040 microcontroller** and is fully supported by the Arduino core package. The board comes with built-in Wi-Fi, Bluetooth, and an onboard IMU—features that is useful for deploying machine learning models at the edge. +To get started with your first **TinyML project**, the **Arduino Nano RP2040 Connect** is a good option. Built by Arduino, it uses the **RP2040 microcontroller** and is fully supported by the Arduino core package. + + +The board comes with built-in Wi-Fi, Bluetooth, and an onboard IMU. These are features that are useful for deploying machine learning models at the edge. ![example image alt-text#center](images/nano.png "Arduino Nano RP2040") -Its compatibility with popular tools like Edge Impulse and the Arduino IDE makes it a suitable choice for TinyML applications. You can learn more about the Arduino Nano RP2040 Connect on the [official Arduino website](https://store.arduino.cc/products/arduino-nano-rp2040-connect-with-headers?_gl=1*1laabar*_up*MQ..*_ga*MTk1Nzk5OTUwMS4xNzQ2NTc2NTI4*_ga_NEXN8H46L5*czE3NDY1NzY1MjUkbzEkZzEkdDE3NDY1NzY5NTkkajAkbDAkaDE1MDk0MDg0ODc.). +Its compatibility with popular tools like Edge Impulse and the Arduino IDE makes it a suitable choice for TinyML applications. You can learn more about the Arduino Nano RP2040 Connect on the [Arduino website](https://store.arduino.cc/products/arduino-nano-rp2040-connect-with-headers?_gl=1*1laabar*_up*MQ..*_ga*MTk1Nzk5OTUwMS4xNzQ2NTc2NTI4*_ga_NEXN8H46L5*czE3NDY1NzY1MjUkbzEkZzEkdDE3NDY1NzY5NTkkajAkbDAkaDE1MDk0MDg0ODc.). + +## Connect and set up the Arduino Nano RP2040 Connect -## Put everything together +This guide will help you connect the Arduino Nano RP2040 Connect to your computer and set up the Arduino IDE for programming. -### Step 1: Connect the LED to the Arduino Nano RP2040 +## Gather the components -To visualize the output of the voice command model, you will use a simple LED circuit. +To visualize the output of the voice command model, you'll use a simple LED circuit. -### Components Needed +You will need the following components: -- Arduino Nano RP2040 Connect -- 1x LED -- 1x 220Ω resistor +- Arduino Nano RP2040 Connect +- 1× LED +- 1× 220 Ω resistor - Breadboard and jumper wires -#### Circuit Diagram +## Connect the LED to the Arduino Nano RP2040 -- **Anode (long leg) of the LED** → Connect to **GPIO pin D2** via the 220Ω resistor -- **Cathode (short leg)** → Connect to **GND** +- **Anode (long leg) of the LED** → connect to **GPIO pin D2** through a 220 Ω resistor +- **Cathode (short leg)** → connect to **GND** -![example image alt-text#center](images/led_connection.png "Figure 14. Circuit Connection") +![Diagram showing the physical breadboard circuit connecting an LED to GPIO D2 and GND on the Arduino Nano RP2040 alt-text#center](images/led_connection.png) -![example image alt-text#center](images/led_connection_schematic.png "Figure 15. Circuit Schematic Connection") +![Schematic diagram showing the LED connected between GPIO D2 and GND with a 220 Ω resistor in series alt-text#center](images/led_connection_schematic.png) -### Step 2: Set Up the Arduino IDEs +## Set up the Arduino IDE To program and deploy your trained model to the Arduino Nano RP2040, you first need to configure your development environment. -Follow the detailed setup instructions provided in the [Arduino Nano RP2040 Install Guide](https://learn.arm.com/install-guides/arduino-pico/) +Follow the steps in the [Arduino Nano RP2040 install guide](/install-guides/arduino-pico/). -This guide will walk you through: +This guide walks you through: -- Installing the Arduino IDE +- Installing the Arduino IDE - Adding the board support package for the Nano RP2040 -{{% notice Note %}} -**Note:** Follow every instruction in the guide **except** `How do I set up the Raspberry Pi Pico W?`, as it is not needed for this project. -{{% /notice %}} +{{< notice Note >}} +Follow all steps in the guide except *How do I set up the Raspberry Pi Pico W?*, which is not required for this project. +{{< /notice >}} -### Step 3: Select Your Board and Port in the Arduino IDE +## Select your board and port in the Arduino IDE -First, open the **Arduino IDE**. +Open the Arduino IDE. To select your board: -To select your board: - -1. Go to **Tools** > **Board**. -2. From the list, choose **Arduino Nano RP2040 Connect**. +Go to **Tools** > **Board** then select **Arduino Nano RP2040 Connect** To select your port: -1. Connect your Arduino board to your computer using a USB cable. -2. Go to **Tools** > **Port**. -3. Select the port labeled with your board’s name, e.g., `COM4 (Arduino Nano RP2040 Connect)` or `/dev/cu.usbmodem...` on macOS. +Connect your board to your computer using a USB cable, then follow these steps: + +- Go to **Tools** > **Port** +- Select the port labeled with your board's name, such as `COM4 (Arduino Nano RP2040 Connect)` or `/dev/cu.usbmodem...` on macOS. -*Your Arduino IDE is now ready to upload code to the Arduino Nano RP2040.* +Your Arduino IDE is now ready to upload code to the Arduino Nano RP2040. diff --git a/content/learning-paths/embedded-and-microcontrollers/edge/overview.md b/content/learning-paths/embedded-and-microcontrollers/edge/overview.md index a637b275d6..eba2464dc5 100644 --- a/content/learning-paths/embedded-and-microcontrollers/edge/overview.md +++ b/content/learning-paths/embedded-and-microcontrollers/edge/overview.md @@ -1,71 +1,95 @@ --- -title: Overview +title: Overview weight: 2 -### FIXED, DO NOT MODIFY +# FIXED, DO NOT MODIFY layout: learningpathall --- -This section introduces the related topics that make out the basis for this learning path. Review it before proceeding to the step-by-step tutorial. +{{< notice Note >}} +This section introduces the key concepts that form the foundation of Edge AI. Review it before starting this Learning Path. +{{< /notice >}} # Edge AI -Edge AI refers to artificial intelligence models that run directly on edge devices, processing data locally rather than relying on cloud computing. These models are optimized for real-time decision-making on resource-constrained devices, such as microcontrollers, embedded systems, and IoT sensors. -**TinyML (Tiny Machine Learning)** is a subset of Edge AI that focuses specifically on deploying machine learning models on ultra-low-power microcontrollers and resource-constrained devices. These microcontrollers typically have limited computational resources — often less than 1 MB of flash memory and only a few hundred kilobytes of RAM — and are designed to run on minimal power, sometimes for years on a single coin-cell battery. Despite these constraints, TinyML enables such devices to perform on-device inference, allowing them to make intelligent decisions in real time without needing to send data to the cloud. This opens the door for smart functionality in low-cost, battery-powered devices used in applications such as environmental monitoring, wearables, smart homes, industrial sensors, and more. +Edge AI refers to artificial intelligence models that run directly on edge devices, processing data locally rather than relying on cloud computing. These models are optimized for real-time decision-making on resource-constrained devices such as microcontrollers, embedded systems, and IoT sensors. -## Key Characteristics of Edge AI and TinyML +TinyML (Tiny Machine Learning) is a subset of Edge AI that focuses on deploying machine learning models on ultra-low-power microcontrollers. These devices typically have less than 1 MB of flash memory and a few hundred kilobytes of RAM, and they are designed to run for extended periods on minimal power, which is often for years on a single coin-cell battery. -Key features of Edge AI and TinyML include; +Despite these constraints, TinyML enables on-device inference, allowing edge devices to make intelligent decisions in real time without sending data to the cloud. This makes smart functionality possible in low-cost, battery-powered devices used in applications such as environmental monitoring, wearables, smart homes, and industrial sensors. -- **Low Power Consumption**: Designed to run on batteries or harvested energy for months or years. +## Key characteristics of Edge AI and TinyML -- **Small Model Size**: Models are optimized (e.g., quantized or pruned) to fit into a few kilobytes or megabytes. +Key features of Edge AI and TinyML include: -- **Limited Compute & Memory** : Typically operates with <1MB RAM and very limited storage. +- **Low power consumption**: designed to run on batteries or harvested energy for months or years +- **Small model size**: models are optimized (for example, quantized or pruned) to fit into a few kilobytes or megabytes +- **Limited compute and memory**: typically operate with under 1 MB of RAM and very limited storage +- **Real-time inference**: immediate local decision-making (for example, wake-word detection) +- **Low latency**: no reliance on cloud; inference is performed on-device +- **Applications**: often used in audio classification, gesture detection, and anomaly detection -- **Real-Time Inference** : Enables immediate local decision-making (e.g., wake-word detection). +Example devices include Arduino Nano 33 BLE Sense, STM32 MCUs, Raspberry Pi Pico, and Arduino Nano RP2040 Connect. -- **Low Latency** : No reliance on cloud – inference is performed on-device. +## Run AI models on resource-constrained devices -- **Applications** : Often used in audio classification, gesture detection, anomaly detection, etc. +Running AI on edge devices presents challenges. These devices often lack high-performance CPUs or GPUs, making compute power and memory usage key concerns. Since many edge devices run on batteries, energy efficiency is also critical. -- **Example Devices** : Arduino Nano 33 BLE Sense, STM32 MCUs, Raspberry Pi Pico, Arduino Nano RP2040 Connect, and more. +To overcome these constraints, models are optimized using techniques such as *quantization*, *pruning*, and *knowledge distillation*. These methods reduce model size and resource requirements while maintaining acceptable accuracy. -## Running AI Models on Resource-Constrained Devices +## Edge AI workflow -Running AI on edge devices presents several challenges. These devices often lack high-performance CPUs or GPUs, making computational power a limiting factor. Limited RAM and storage require careful memory management, and since many edge devices run on batteries, energy efficiency is a critical concern. To overcome these constraints, models are optimized through techniques such as quantization, pruning, and knowledge distillation, which reduce model size while maintaining accuracy. +[1] Data collection +→ Sensors capture data (such as audio, motion, and vision) -## Edge AI Implementation Workflow +[2] Model training +→ Use cloud or local compute for training -The process of implementing Edge AI begins with data collection using sensors, such as cameras, microphones, or motion detectors. This data is then used to train machine learning models on high-performance machines, such as cloud servers or workstations. Once trained, the models undergo optimization to reduce size and computational requirements before being deployed on microcontrollers or Arm-based microprocessors. Finally, inference takes place, where the model processes real-time data directly on the device to make decisions. +[3] Model optimization +→ Apply quantization, pruning, or distillation + +[4] Deployment +→ Flash model onto Arm-based edge device + +[5] On-device inference +→ Device makes real-time predictions locally ## Applications of Edge AI -Edge AI is used in a wide range of applications. In smart homes, voice assistants like Amazon Alexa rely on on-device speech recognition to process wake words. Security systems use AI-driven cameras to detect motion and identify anomalies, while energy management systems optimize power usage by analyzing real-time data from HVAC units. +Edge AI is used in a wide range of real-world applications: + +- **Smart homes**: voice assistants process wake words locally; security systems detect motion and identify anomalies +- **Wearables**: smartwatches detect heart rate irregularities; fitness trackers analyze motion patterns +- **Industrial systems**: predictive maintenance uses vibration and temperature data; safety sensors shut down equipment automatically +- **Agriculture**: AI-powered sensors optimize irrigation and fertilizer use +- **Autonomous systems**: onboard AI enables real-time navigation and obstacle avoidance + -Wearable devices also benefit from Edge AI. Smartwatches monitor health by detecting heart rate irregularities, and fitness trackers use AI-powered motion analysis to improve exercise tracking. +## The BLERP framework -In industrial settings, predictive maintenance applications rely on IoT sensors to monitor vibrations and temperatures, helping prevent machinery failures. Smart agriculture systems use soil condition sensors to optimize irrigation and fertilization, while autonomous vehicles process sensor data for real-time navigation and obstacle detection. +To recall the benefits of Edge AI, the **BLERP** mnemonic highlights five critical aspects: -## The BLERP mnemonic +| Area | Description | +|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **B – Bandwidth** | Reduces the need to send large amounts of data to the cloud, which is especially useful for video or sensor streams. | +| **L – Latency** | Enables real-time decision-making by processing data locally, with no round trips to the cloud. Crucial for self-driving cars, health monitors, and offline use. | +| **E – Economics** | Running models on-device reduces long-term costs related to cloud compute, data transfer, and power usage. | +| **R – Reliability** | Devices remain functional even when disconnected from the internet which is important for mission-critical or remote deployments. | +| **P – Privacy** | Data stays on-device, reducing risk and helping meet regulatory requirements like GDPR or HIPAA. | -To help remember the benefits of **Edge AI**, **BLERP** highlights the critical aspects of deploying machine learning models on edge devices. First used by Situnayake in 2023, the abbreviation expands to **Bandwidth, Latency, Economics, Reliability, and Privacy**. These components are key to understanding the advantages of processing data on-device rather than relying on the cloud. The table below provides an overview of each component and its importance in Edge AI applications. +## Why learn Edge AI? -| Area | Description | -|------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| B – Bandwidth | Edge AI reduces the amount of data that needs to be sent to the cloud. This is critical when working with high-volume data like video or sensor streams. Processing locally helps avoid congestion and dependency on internet speed. | -| L – Latency | Edge devices can make real-time decisions faster because they don't rely on cloud round trips. One of the significant benefits of Edge AI is low latency - processing occurs on-device without needing to send data to the cloud. This is crucial for applications requiring real-time decision-making, such as self-driving cars or medical monitoring devices. Additionally, Edge AI allows devices to function in offline environments, making it ideal for remote locations with limited connectivity. | -| E – Economics | Running models locally on low-power edge devices is often cheaper in the long run. It reduces cloud compute costs, data transmission costs, and energy consumption. | -| R – Reliability | Edge AI systems can continue functioning even with limited or no internet connection. This makes them more robust in remote areas, mission-critical applications, or offline settings. | -| P – Privacy | Data can be processed locally without being transmitted to external servers, reducing the risk of data breaches and complying with privacy regulations like GDPR or HIPAA. | +Edge AI is revolutionizing industries by making smart, local decision-making possible at the device level: -## Why Learn Edge AI? +- **Healthcare**: enables remote diagnostics and patient monitoring +- **Agriculture**: improves yield through intelligent irrigation and pest control +- **Manufacturing**: reduces downtime through predictive maintenance and quality inspection -Edge AI is transforming multiple industries. In healthcare, AI-powered medical diagnostics assist in early disease detection, while remote patient monitoring improves access to care. In agriculture, AI-driven sensors optimize soil conditions and pest control, leading to higher yields and resource efficiency. The manufacturing sector benefits from predictive maintenance and quality inspection, reducing downtime and improving productivity. -## Next Steps +By bringing intelligence to the edge, developers can create responsive, efficient, and secure systems that operate independently of constant internet access. -To build effective TinyML and Edge AI projects, one needs more than just data—**both software and hardware** play a critical role in the development process. While data forms the foundation for training machine learning models, the **software** enables data processing, model development, and deployment, and the **hardware** provides the physical platform for running these models at the edge. +## Next steps -In this learning path, you will build a model that recognize specific voice commands, which will be used to **control LEDs on the Arduino Nano RP2040 Connect**. In the following steps, both software and hardware components will be discussed in detail. +To build effective TinyML and Edge AI solutions, you’ll need both high-quality data and the right combination of software and hardware. In this Learning Path, you’ll train a model to recognize specific voice commands and use those commands to control LEDs on the Arduino Nano RP2040 Connect. +In the next steps, you’ll walk through each part of the process in detail. diff --git a/content/learning-paths/embedded-and-microcontrollers/edge/program-and-deployment.md b/content/learning-paths/embedded-and-microcontrollers/edge/program-and-deployment.md index 57a43ed544..9c99f325c1 100644 --- a/content/learning-paths/embedded-and-microcontrollers/edge/program-and-deployment.md +++ b/content/learning-paths/embedded-and-microcontrollers/edge/program-and-deployment.md @@ -1,49 +1,79 @@ --- -title: Program your first tinyML device +title: Program your first TinyML device weight: 5 ### FIXED, DO NOT MODIFY layout: learningpathall --- -# Programming your first tinyML device +## Program and deploy your first TinyML device -This section helps you create a complete sketch that you can upload onto your Arduino Nano RP2040. +This section shows you how to create a complete sketch that you can upload onto your Arduino Nano RP2040. -## Step 1: Create a New Sketch +You’ll begin by setting up a new sketch in the Arduino IDE, then add the library generated by Edge Impulse. After reviewing a sample program, you’ll upload it to your board to run real-time audio inference and control an LED based on voice commands. -1. Open the **Arduino IDE**. -2. Go to **File** > **New**. -3. A new sketch (blank code window) will open, ready for you to start writing your code. -4. Save your sketch by going to **File** > **Save**. Choose a name and location for your file. +By the end of this section, your board will be able to respond to the words "on" and "off". It is a simple, but powerful, demonstration of TinyML in action. -## Step 2: Upload the Arduino Library from Edge Impulse +## Create a new sketch -1. After creating and saving your sketch, go to **Sketch** > **Include Library** > **Add .ZIP Library**. -2. In the file dialog that opens, navigate to the location of the **ZIP file** you exported from Edge Impulse in [Set up your environment](/learning-paths/embedded-and-microcontrollers/edge/software-edge-impulse/) -3. Select the **ZIP file** and click **Open**. +To create a new sketch in the Arduino IDE, follow these steps: -## Step 3: Include the Library in Your Sketch +* Open the **Arduino IDE**, and then go to **File** > **New**. +* A new sketch window will open, and you will see a blank sketch with the default structure, which includes two main functions: `setup()` and `loop()`. The `setup()` function runs once when the program starts, and the `loop()` function runs continuously after that. +* You will see a blank sketch with the default structure, which includes two main functions: `setup()` and `loop()`. The `setup()` function runs once when the program starts, and the `loop()` function runs continuously after that. +* Now save your sketch using **File** > **Save**, and give it a clear name. For example, you can name it **"TinyML_Voice_Control"** or something similar that describes your project. -Finally, to include the library and model in your sketch, go to **Sketch** > **Include Library** and select the newly installed library and model from the list. +{{< notice Note >}} +Make sure to save it in a location where you can easily find it later, such as your **Documents** folder or a dedicated **Arduino** folder. +{{< /notice >}} -{{% notice Note %}} -The libray should be of the form `Name_of_your_library_inferencing.h` -{{% /notice %}} -# Code walk-through +## Upload the Arduino library from Edge Impulse + +* After creating and saving your sketch, go to **Sketch** > **Include Library** > **Add .ZIP Library** +* In the file dialog that opens, navigate to the location of the ZIP file you exported from Edge Impulse. +* Select the ZIP file and click **Open** +* The Arduino IDE will automatically install the library and any dependencies it requires. + +## Include the library in your sketch + +To include the library and model in your sketch, go to **Sketch** > **Include Library** and select the newly installed Edge Impulse library from the list. + +{{< notice Note >}} +The header file should be named something like `Name_of_your_library_inferencing.h` +{{< /notice >}} -In the example repository, you will find a code snippet `Code_Sample.ino`, which is used as a sketch for the project. Before running the code, it’s important to understand what each part does. +This will add the necessary include statement at the top of your sketch, allowing you to use the functions and classes provided by the Edge Impulse library. + +## Code walk-through + +In the example repository, you will find a code snippet named `Code_Sample.ino`, which is used as a sketch for this project. Before running the code, it’s important to understand what each part does. Take a few minutes to read through the comments and logic in the sketch before uploading it to your board. The code is available in the example repository, and below is a walk-through of the steps. -## Include Necessary Libraries and Define Data Structure for Inference +## Include the library in your sketch + +To include the library and model in your sketch, go to **Sketch** > **Include Library** and select the newly installed Edge Impulse library from the list. + +{{< notice Note >}} +The header file should be named something like `Name_of_your_library_inferencing.h` +{{< /notice >}} + +This will add the necessary include statement at the top of your sketch, allowing you to use the functions and classes provided by the Edge Impulse library. -This block sets up the core dependencies for running Edge Impulse inference on audio input. It includes necessary libraries and defines a structure `inference_t` that holds the audio buffer and relevant state needed to manage sampling and inferencing. +## Code walk-through + +In the example repository, you will find a code snippet named `Code_Sample.ino`, which is used as a sketch for this project. Before running the code, it’s important to understand what each part does. + +Take a few minutes to read through the comments and logic in the sketch before uploading it to your board. The code is available in the example repository, and below is a walk-through of the steps. + +## Include libraries and define inference data structure + +This block sets up the core dependencies for running Edge Impulse inference on audio input. It includes the necessary libraries and defines a structure `inference_t` that holds the audio buffer and relevant state needed to manage sampling and inferencing. ```c #include // Include the Edge Impulse inference SDK for running the model -#include // Include the Pulse Density Modulation (PDM) library for audio input +#include // Include the Pulse Density Modulation (PDM) library for audio input // Define a structure to store inference-related audio data typedef struct { @@ -54,16 +84,16 @@ typedef struct { } inference_t; ``` -## Code to Define Global Variables for Inference and Sample Buffer +## Define global variables for inference and sample buffer -Declares and initializes global variables used for storing inference data, raw audio samples, debug mode flag, and a recording readiness flag. These are essential for coordinating the data collection and inference process. +This block declares and initializes global variables used to store inference data, raw audio samples, a debug flag, and a recording readiness flag. These variables coordinate the data collection and inference process. ```c static inference_t inference; // Global instance of the inference structure to manage audio data static signed short sampleBuffer[2048]; // Buffer to temporarily store raw audio samples (16-bit signed) static bool debug_nn = false; // Flag to enable/disable detailed neural network debug output static volatile bool record_ready = false; // Flag indicating when the system is ready to start recording (volatile due to use in ISR) -``` + ## Setup Function for Initializing the Serial and Microphone @@ -98,9 +128,10 @@ void setup() { The macro `#define LED_PIN 2` specifies the pin number to which the LED is connected. You can change this value to any available digital pin on your board. {{% /notice %}} -## Main Loop to Handle Inference and Control LED +## Main loop to handle inference and control LED + +The `loop()` function runs continuously after `setup()`. It performs the inference process, including recording audio, running the classifier, and controlling an LED based on the model's prediction. -Defines the `loop()` function, which runs continuously after `setup()`. It performs the inference process, including recording audio, running the classifier, and controlling an LED based on the inference result. ```c void loop() { @@ -152,9 +183,9 @@ In this example, the LED will turn on if the model predicts the "on" label with You are free to modify these threshold values to better suit your application or improve model response. {{% /notice %}} -## PDM Data Ready Callback for Buffer Management +## PDM data ready callback for buffer management -Defines the `pdm_data_ready_inference_callback()` function, which is triggered when the Pulse Density Modulation (PDM) buffer is full. It reads available audio data and stores it in the inference buffer for processing. +The `pdm_data_ready_inference_callback()` function is triggered when the Pulse Density Modulation (PDM) buffer is full. It reads the available audio data and stores it in the inference buffer for processing. ```c /* PDM buffer full callback */ @@ -177,7 +208,7 @@ static void pdm_data_ready_inference_callback(void) { } ``` -## Initialize Microphone for Inference +## Initialize microphone for inference Allocates memory for audio sampling, configures the PDM microphone, and prepares the system to begin audio inference. @@ -215,7 +246,7 @@ static bool microphone_inference_start(uint32_t n_samples) { ``` -## Microphone Inference Record Function +## Microphone inference record function Defines the `microphone_inference_record()` function, which waits for the microphone buffer to be filled with audio samples. Once the buffer is ready, it resets the buffer state and prepares for the next recording cycle. @@ -231,7 +262,7 @@ static bool microphone_inference_record(void) { } ``` -## Microphone Data Handling and Cleanup +## Microphone data handling and cleanup Defines two functions: `microphone_audio_signal_get_data()`, which converts raw audio data from the buffer into a float format, and `microphone_inference_end()`, which cleans up by stopping the PDM and freeing memory allocated for the audio buffer. @@ -249,7 +280,7 @@ static void microphone_inference_end(void) { } ``` -## Function to Print the Inference Results +## Function to print the inference results Defines the `print_inference_result()` function, which outputs the results of the inference, including the timing of different stages — **DSP**, classification, anomaly detection — and the prediction values for each class. @@ -280,53 +311,51 @@ void print_inference_result(ei_impulse_result_t result) { The `ei_printf` command is a custom logging function from the Edge Impulse SDK, used for printing debug or inference-related information to the serial monitor, optimized for embedded systems. It works similarly to `printf` but is tailored for the Edge Impulse environment. You can find the complete `Code_Sample.ino` in the example repository and try it out yourself. {{% /notice %}} -# Run Your Code +## Run your code -Now that you have a good understanding of the code, you should run it on your device. With your **Arduino Nano RP2040** plugged into your computer, and the correct [board and port](/learning-paths/embedded-and-microcontrollers/egde/connect-and-set-up-arduino/) selected in the Arduino IDE, follow these steps: +Now that you have a good understanding of the code, you should run it on your device. With your **Arduino Nano RP2040** plugged into your computer, and the correct board and port selected in the Arduino IDE, follow these steps: -#### If you're using the **Upload Button** +If you're using the **Upload Button**: -1. Click the **right-facing arrow** at the top-left of the Arduino IDE window. -2. The IDE will compile your code and upload it to your board. -3. Wait for the message **“Done uploading.”** to appear at the bottom of the IDE. +- Click the **right-facing arrow** at the top-left of the Arduino IDE window. +- The IDE will compile your code and upload it to your board. +- Wait for the message **“Done uploading.”** to appear at the bottom of the IDE. -#### If you're using the **Sketch Menu** +If you're using the **Sketch Menu**: -1. Go to **Sketch** > **Upload**. -2. The IDE will compile and upload your sketch to the board. -3. Once the upload is complete, you’ll see **“Done uploading.”** at the bottom. +- Go to **Sketch** > **Upload**. +- The IDE will compile and upload your sketch to the board. +- Once the upload is complete, you’ll see **“Done uploading.”** at the bottom. Your board should now start running the uploaded code automatically. -### Verify Your Code is Running +## Verify your code is running -To further confirm that your code is running properly: +To confirm that your code is running properly: -1. Go to **Tools** > **Serial Monitor** in the Arduino IDE. -2. Set the baud rate to **115200** (if it's not already). -3. Observe the output messages: - - Start and end of recording - - Inference process - - Predictions for each label +- Go to **Tools** > **Serial Monitor** in the Arduino IDE +- Set the baud rate to **115200** (if it's not already) +- Observe the output messages: + - Start and end of recording + - Inference process + - Predictions for each label These messages indicate that your model is working and processing voice input as expected. -### Recording Your Voice to Toggle the LED +## Record your voice to toggle the LED -1. Wait for the **"Recording..."** message to appear on the Serial Monitor. This indicates that the system is ready to record your voice input. +- Wait for the **Recording...** message to appear on the Serial Monitor. This means the system is ready to capture voice input. +- Say your command (for example, "on" or "off") clearly and promptly. The system records for only about one second. +- The model will make a prediction based on your voice input and toggle the LED accordingly. +- You can adjust the **threshold** values in the code to control how confident the prediction must be before the LED toggles. This helps fine-tune the system’s responsiveness. +- If the LED turns on when you say "on" and turns off when you say "off", your system is working correctly. -2. Speak your command (e.g., "on" or "off") quickly, as the system only records for a brief 1-second window. Once that window closes, inference will take place, and the system will process the voice command. -3. The system will make a prediction based on the input, toggling the LED accordingly. +## Serial Monitor output -4. You can adjust the **threshold** for prediction accuracy in the code to fine-tune when the LED should toggle, based on the prediction confidence level. This helps control how sensitive the system is to voice commands. +Your Serial Monitor should look like the image below: -### Serial Monitor Output +![example image alt-text#center](images/serial_monitor.png "Circuit Connection") -Your Serial Monitor should look like the image below. +Congratulations, you’ve successfully programmed your first TinyML microcontroller! You've also built a functional, smart system to control an LED with your voice. -![example image alt-text#center](images/serial_monitor.png "Figure 16. Circuit Connection") - -{{% notice Congratulations %}} -You’ve successfully programmed your first TinyML microcontroller! You've also built a functional, smart system to control an LED with your voice. -{{% /notice %}} diff --git a/content/learning-paths/embedded-and-microcontrollers/edge/software-edge-impulse.md b/content/learning-paths/embedded-and-microcontrollers/edge/software-edge-impulse.md index 6401b0022d..e091998f92 100644 --- a/content/learning-paths/embedded-and-microcontrollers/edge/software-edge-impulse.md +++ b/content/learning-paths/embedded-and-microcontrollers/edge/software-edge-impulse.md @@ -1,169 +1,200 @@ --- -title: Set up your environment +title: Train and deploy a TinyML audio classifier with Edge Impulse weight: 3 -### FIXED, DO NOT MODIFY +# FIXED, DO NOT MODIFY layout: learningpathall --- -# Using Edge Impulse to Train TinyML Models -Now that the foundational concepts of TinyML and Edge AI are clear, it's time to move from theory to practice. **Edge Impulse** is an easy to use platform for training TinyML models. It provides an end-to-end pipeline for collecting data, designing features, training models, and deploying them to edge devices. In this section, you will explore how Edge Impulse is used to train models specifically for ultra-low-power microcontrollers, bridging the gap between machine learning and real-world embedded applications. +## Using Edge Impulse + +Now that you've learned the foundational concepts of Edge AI and TinyML, it's time to move from theory to practice. + +{{< notice Note >}} +In this section, you'll use Edge Impulse to train a voice-command classification model and prepare it for deployment to the Arduino Nano RP2040 Connect. +{{< /notice >}} ## What is Edge Impulse? -**Edge Impulse** is a development platform designed to simplify the process of building, training, and deploying machine learning (ML) models on **embedded systems and edge devices**, such as microcontrollers, sensors, and single-board computers (e.g., Raspberry Pi, Arduino). +Edge Impulse is a development platform that simplifies building, training, and deploying machine learning (ML) models on embedded systems and edge devices such as microcontrollers, sensors, and single-board computers like the Raspberry Pi or Arduino. ## Key Features of Edge Impulse | Feature | Description | |-----------------------|-----------------------------------------------------------------------------------------------------| -| **Data Collection** | Collects data from sensors (e.g., accelerometers, microphones, cameras) in real time. | -| **Preprocessing** | Provides tools for signal processing and feature extraction tailored for embedded systems. | -| **Model Training** | Supports built-in ML algorithms and integrates with frameworks like TensorFlow. | -| **Edge Optimization** | Automatically optimizes models to run efficiently on low-power edge hardware. | -| **Deployment** | Enables seamless deployment to microcontrollers, RTOS-based boards, and Linux devices. | -| **Web-Based Platform**| Fully browser-based interface for managing projects and workflows without needing local setup. | +| **Data collection** | collects data from sensors, such as accelerometers, microphones, and cameras in real time | +| **Preprocessing** | provides tools for signal processing and feature extraction tailored for embedded systems | +| **Model training** | supports built-in ML algorithms and integrates with frameworks like TensorFlow | +| **Edge optimization** | automatically optimizes models to run efficiently on low-power edge hardware | +| **Deployment** | enables seamless deployment to microcontrollers, RTOS-based boards, and Linux devices | +| **Web-based platform**| fully browser-based interface for managing projects and workflows without needing local setup | --- -## Why It’s Important in Edge AI and TinyML +## Why it's important in Edge AI and TinyML + +Edge Impulse offers the following benefits: + +- Bridges the gap between machine learning and embedded development +- Accelerates prototyping and deployment of AI features directly on hardware +- Supports TinyML applications that run on devices with very limited memory and compute power +- Works with popular hardware platforms such as Arduino, Raspberry Pi, Nordic, and STMicroelectronics + +## Get started with Edge Impulse + +To begin working with TinyML models, visit [Edge Impulse](https://edgeimpulse.com). You'll need to create a free account to access the full platform. -- **Bridges the gap** between machine learning and embedded development. -- **Accelerates prototyping** and deployment of AI features directly on hardware. -- **Supports TinyML** applications that run on devices with very limited memory and compute power. -- Works with popular hardware platforms like **Arduino**, **Raspberry Pi**, **Nordic**, **STMicroelectronics**, and more. +In the following sections, you'll walk through each key page on the Edge Impulse platform using snapshots as a guide. These visuals will help you understand what actions to take and how each part of the interface contributes to building and deploying your machine learning model. -## Getting Started with Edge Impulse -To begin working with TinyML models, visit the **[Edge Impulse](https://edgeimpulse.com)**. You’ll need to create a free account to access the full platform. In the following sections, you will walk through each key page on the Edge Impulse platform using the attached snapshots as guide. These will help you understand what actions to take and how each part of the interface contributes to building and deploying your machine learning model. -![example image alt-text#center](images/1.png "Figure 1. Home Page of Edge Impulse") +![Screenshot of the Edge Impulse home page showing the main navigation and project dashboard alt-text#center](images/1.png "Home page of Edge Impulse website") -### Step 1: Create a New Project -Once you’ve created your account and logged in, the first step is to **create a new project**. Give your project a name that clearly reflects its purpose—this helps with easy identification, especially if you plan to build multiple models later on. For example, if you're building a keyword spotting model, you might name it "Wake Word Detection". You’ll also need to select the appropriate **project type** and **project setting**, as shown in the snapshot below. +## Create a new project -![example image alt-text#center](images/3.png "Figure 2. New Project Setup") +After you create your account and log in, the first step is to create a new project. Give your project a name that clearly reflects its purpose. This helps with easy identification, especially if you plan to build multiple models. -### Step 2: Configure the Target Device +For example, if you're building a keyword-spotting model, you might name it `Wake word detection`. -After creating your project, the next step is to **configure the target device**. Since you are using the **Arduino Nano RP2040 Connect**, click the highlighted button to begin device configuration, as shown in the snapshot below. This ensures that the data collection, model training, and deployment steps are optimized for your specific hardware. +You'll also need to select the appropriate **project type** and **project settings**, as shown in the screenshot below. -The specifications of the Arduino Nano RP2040 Connect board can be found on [Arduino’s official page](https://store.arduino.cc/products/arduino-nano-rp2040-connect). +![Screenshot showing the new project creation page in Edge Impulse, with fields for project name, type, and target device alt-text#center](images/3.png "New project setup") -Follow the exact settings in the attached snapshot to complete the configuration. +## Configure the target device -![example image alt-text#center](images/4.png "Figure 3. Configure Arduino Nano RP2040") +After creating your project, the next step is to configure the target device. As you're using the **Arduino Nano RP2040 Connect**, click the button highlighted in the screenshot to begin the configuration process. This ensures that data collection, model training, and deployment are optimized for your hardware. -### Step 3: Add the Dataset +You can find the full specifications for the Arduino Nano RP2040 Connect on [Arduino’s official product page](https://store.arduino.cc/products/arduino-nano-rp2040-connect). -With your device configured, the next step is to **add your dataset** to the project. Click on the **"Add existing data"** button and follow the configuration settings shown in the attached snapshot. This allows you to upload pre-recorded data instead of collecting it live, which can save time during the development phase. +Follow the settings shown in the screenshot to complete the configuration. -An **example repository** has been set up with some assets to be used throughout this Learning Path. You can clone it with the following command: +![Screenshot showing the Edge Impulse device configuration page with Arduino Nano RP2040 Connect selected alt-text#center](images/4.png "Configure Arduino Nano RP2040") + + +## Add the dataset + +With your device configured, the next step is to add your dataset to the project. Click **Add existing data** and follow the configuration settings shown in the screenshot. This lets you upload pre-recorded data instead of collecting it live, which can save time during development. + +An example repository has been prepared with assets used throughout this Learning Path. You can clone it with the following command: ```bash git clone https://github.com/e-dudzi/Learning-Path.git ``` -The repository contains a `Dataset.zip` file containing the dataset used in the project. Extract it on your local machine. For convenience, the dataset has already been split into **training** and **testing**. +The repository contains a `Dataset.zip` file with the dataset used in this project. Extract the contents to your local machine. For convenience, the dataset is already split into **training** and **testing** sets. + +![Screenshot showing the Edge Impulse interface with the Add existing data panel open, used to upload pre-recorded datasets alt-text#center](images/6.png "Adding existing data") -![example image alt-text#center](images/6.png "Figure 4. Add Existing Data") {{% notice Note %}} -Do **not** check the **Green** highlighted area during upload. The dataset already includes metadata. Enabling that option may result in **much slower upload times** and is unnecessary for this project. +Do not check the green highlighted area during upload. The dataset already includes metadata. Enabling that option may result in much slower upload times and is unnecessary for this project. {{% /notice %}} -![example image alt-text#center](images/7.png "Figure 5. Dataset Overview") +![Screenshot showing the Data acquisition tab in Edge Impulse with uploaded samples organized by label alt-text#center](images/7.png "Dataset overview") -### Dataset Uploaded Successfully +## Dataset uploaded successfully + +After the dataset is uploaded, you'll see the samples listed in the **Data acquisition** tab, organized by their labels. You can click each sample to inspect the raw signal, view metadata, and even listen to the audio recordings directly within the Edge Impulse interface. This helps confirm that the uploaded data is accurate and ready for training. -This is what you should see after the dataset has been successfully uploaded. The data samples will appear in the **Data acquisition** tab, categorized by their respective labels. You can click on each sample to inspect the raw signal, view metadata, and even **listen to the audio recordings** directly within the Edge Impulse interface. This helps verify that the uploaded data is accurate and usable for training. {{% notice Note %}} -This dataset is made up of **four labels**: `on`, `off`, `noise`, and `unknown`. +This dataset is consists of four labels: +- on +- off +- noise +- unknown {{% /notice %}} -![example image alt-text#center](images/8.png "Figure 6. Dataset Overview") +![Screenshot showing the Impulse design interface in Edge Impulse with input, processing, and learning blocks configured alt-text#center](images/8.png "Dataset overview") -### Step 4: Create the Impulse +## Create the impulse -Now that your data is ready, it's time to create the **impulse**, which defines the flow of data from input to output through processing blocks. Click on the **"Create Impulse"** button in the menu and configure it exactly as shown in the snapshot below. This typically includes setting the input data type (e.g., audio), adding a **processing block** (such as MFCC for audio), and a **learning block** (such as a neural network classifier). +Now that your data is ready, it's time to create the impulse. An impulse defines the flow of data from input to output using processing and learning blocks. -After configuring everything, **don’t forget to save your impulse**. +Click **Create impulse** in the menu and configure it as shown in the screenshot. Typically, this includes setting the input data type (such as audio), adding a **processing block** (like MFCC), and a **learning block** (such as a neural network classifier). -![example image alt-text#center](images/9.png "Figure 7. Create Impulse") +After configuring the impulse, make sure to **save your changes**. -### Step 5: Configure the MFCC Block -Next, you'll configure the **MFCC (Mel Frequency Cepstral Coefficients)** processing block, which transforms the raw audio data into features suitable for model training. Click on **"MFCC"** in the left-hand menu under the **"Impulse Design"** section. +![example image alt-text#center](images/9.png "Create Impulse") -Set the parameters exactly as shown in the snapshot below. These settings determine how the audio input is broken down and analyzed. Once you're done, be sure to **save the parameters**. These parameters are chosen for this path. Modifications can be made once you are familiar with Edge Impulse. +## Configure the MFCC block -![example image alt-text#center](images/10.png "Figure 8. MFCC Block Configuration") +Next, you'll configure the **MFCC (Mel Frequency Cepstral Coefficients)** processing block. This block transforms raw audio into features suitable for training your model. -{{% notice Note %}} -The **green highlighted section** on the MFCC configuration page gives an estimate of how the model will perform **on the target device**. This includes information like memory usage (RAM/Flash) and latency, helping you ensure the model fits within the constraints of your hardware. -{{% /notice %}} +Click **MFCC** in the left-hand menu under the **Impulse design** section. -### Step 6: Generate Features +Set the parameters exactly as shown in the screenshot. These settings determine how the audio input is segmented and analyzed. Once complete, make sure to **save the parameters**. -After saving the MFCC parameters, the next step is to generate features from your dataset. Click on the **"Generate features"** button highlighted. Edge Impulse will process all your data samples using the MFCC configuration and create a set of features suitable for training a machine learning model. +These defaults are chosen for this Learning Path, but you can experiment with different values once you're more familiar with Edge Impulse. -Once the feature generation is complete, you'll see a **2D visualization plot** that shows how the dataset is distributed across the four labels: `on`, `off`, `noise`, and `unknown`. This helps to visually confirm whether the different classes are well-separated and learnable by the model. +![Screenshot showing the MFCC configuration page in Edge Impulse with time and frequency parameters set for feature extraction alt-text#center](images/10.png "MFCC block configuration") -![example image alt-text#center](images/12.png "Figure 9. Feature Explorer") +{{< notice Note >}} +The green-highlighted section on the MFCC configuration page provides an estimate of how the model will perform on the target device. This includes memory usage (RAM and flash) and latency, helping ensure the model fits within hardware constraints. +{{< /notice >}} -### Step 7: Setting Up the Classifier +## Generate features -Now it's time to configure the **neural network classifier**, which will learn to recognize the different audio commands. Click on the **"Classifier"** button in the left-hand menu under **Impulse Design** and set the parameters exactly as shown in the snapshot below. +After saving the MFCC parameters, the next step is to generate features from your dataset. Click **Generate features** to begin. Edge Impulse will process your data using the MFCC configuration and create a feature set for training. -{{% notice Note %}} -For this learning path, a learning rate of `0.002` was chosen, although the snapshot shows a value of `0.005`. You are free to experiment with different values to improve model accuracy. However, using `0.002` is recommended as a good starting point. -{{% /notice %}} +When complete, you'll see a **2D feature plot** that shows how the data is distributed across the four labels: `on`, `off`, `noise`, and `unknown`. This helps visually confirm whether the classes are distinct and learnable. + +![Screenshot showing the feature explorer in Edge Impulse with a 2D visualization of four labeled audio classes alt-text#center](images/12.png "Feature explorer") + +## Set up the classifier + +Now it's time to configure the **neural network classifier**, which learns to recognize different audio commands. Click **Classifier** under **Impulse design** and set the parameters as shown in the screenshot. + +{{< notice Note >}} +For this Learning Path, use a learning rate of `0.002` even though the screenshot shows `0.005`. You can experiment later, but `0.002` is a good starting point. +{{< /notice >}} + +Once all parameters are set, click **Save and train** to begin training your model. -Once all the parameters are set, click on **"Save and train"** to start training your model. +![Screenshot showing the classifier configuration screen in Edge Impulse with neural network settings for audio classification alt-text#center](images/13.png) -![example image alt-text#center](images/13.png "Figure 10. Classifier Settings") +## Review model performance -### Step 8: Reviewing Model Performance +After training is complete, Edge Impulse displays key performance metrics, including **accuracy**, **loss**, and a **confusion matrix**. -After the training process is complete, Edge Impulse will display the **model's performance**, including its overall **accuracy**, **loss**, and a **confusion matrix**. +![Screenshot showing training results in Edge Impulse, including a confusion matrix and accuracy/loss graph alt-text#center](images/14.png) -![example image alt-text#center](images/14.png "Figure 11. Model Performance") +- **Accuracy**: how often the model correctly predicts labels +- **Loss**: how far off the predictions are - lower values indicate better performance +- **Confusion matrix**: shows how well the model classified each label and where it made errors -- **Accuracy** reflects how often the model predicts the correct label. -- **Loss** indicates how far the model’s predictions are from the actual labels during training — a lower loss generally means better performance. -- The **confusion matrix** shows how well the model predicted each of the four labels (`on`, `off`, `noise`, `unknown`), and can help identify patterns of misclassification. +Review these results to evaluate how well your model learned. If needed, adjust the configuration or retrain to improve accuracy. -Review these metrics to determine if the model is learning effectively. If needed, adjust the model parameters or revisit earlier steps to improve performance. +**On-device performance (EON Compiler – RAM optimized):** -**On-Device Performance (EON Compiler - RAM Optimized):** +| Metric | Value | +|------------------|---------| +| Inference time | 6 ms | +| Peak RAM usage | 12.5 KB | +| Flash usage | 49.7 KB | -| Metric | Value | -|--------------------|-----------| -| Inference Time | 6 ms | -| Peak RAM Usage | 12.5 KB | -| Flash Usage | 49.7 KB | +![Screenshot showing on-device model performance stats including RAM, flash, and latency alt-text#center](images/15.png) -![example image alt-text#center](images/15.png "Figure 12. Model Performance") +You can also use the pre-trained model included in `ei-edge-ai-tutorials-arduino-1.0.1.zip` from the example repository. -You can also use the pre-trained model in the `ei-edge-ai-tutorials-arduino-1.0.1.zip` archive, from the example repository. +## Deploy the model -### Final Step: Deploying the Model +To run the trained model on your Arduino Nano RP2040 Connect, export it as an Arduino library: -To use the trained model on your Arduino Nano RP2040, follow the steps below to export it as an Arduino library. +* Click **Deployment** in the left menu +* Search for "Arduino" and select **Arduino library** +* If the export doesn't start automatically, click **Build** -1. Click on the **Deployment** tab from the menu. -2. In the **search bar**, type **"Arduino"** to filter the export options. -3. Select **Arduino library** from the list. -4. If the export process does not start automatically, click **Build**. The model will be downloaded as a `.zip` file. +The model will be downloaded as a `.zip` file, which you can import into the Arduino IDE. -![example image alt-text#center](images/16.png "Figure 13. Model Deployment") +![Screenshot showing the Edge Impulse deployment page with Arduino library export selected alt-text#center](images/16.png) -## Next Steps +## Next steps -In the following steps, you will move from model training to real-world deployment. Specifically, you will: +In the next section, you'll move from model training to real-world deployment. Specifically, you will: -- Connect an **LED** to the **Arduino Nano RP2040** board. -- Set up the **Arduino IDE** for development. -- Program the board and **deploy the trained model** to recognize voice commands which will be used to turn `ON` and `OFF` the LED +- Connect an LED to the Arduino Nano RP2040 board +- Set up the Arduino IDE for development +- Deploy the trained model to recognize voice commands and control the LED diff --git a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/1-prerequisites.md b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/1-prerequisites.md index 16c4e3573e..c1ba894988 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/1-prerequisites.md +++ b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/1-prerequisites.md @@ -12,7 +12,7 @@ In this Learning Path, you'll learn how to convert the Stable Audio Open Small m Your first task is to prepare a development environment with the required software: -- Android NDK: version r25b or newer. +- Android NDK: version r27b or newer. - Python: version 3.10 or newer (tested with 3.10). - CMake: version 3.16.0 or newer (tested with 3.28.1). @@ -33,9 +33,7 @@ Download and install [Python version 3.10](https://www.python.org/downloads/rele {{< tabpane code=true >}} {{< tab header="Linux">}} -sudo add-apt-repository ppa:deadsnakes/ppa -sudo apt update -sudo apt install python3.10 python3.10-venv python3.10-pip +sudo apt install -y python3.10 python3.10-venv {{< /tab >}} {{< tab header="MacOS">}} brew install python@3.10 @@ -56,7 +54,7 @@ CMake is an open-source tool that automates the build process for software proje {{< tabpane code=true >}} {{< tab header="Linux">}} sudo apt update -sudo apt install cmake +sudo apt install cmake g++ git {{< /tab >}} {{< tab header="MacOS">}} brew install cmake @@ -98,20 +96,44 @@ You can verify the installation and check the version with: bazel --version ``` -### Install Android NDK +### Install Android SDK and NDK + +To build a native Android application you will need to install the Android SDK: + +{{< tabpane code=true >}} + {{< tab header="Linux">}} +cd $WORKSPACE +sudo apt install openjdk-21-jdk openjdk-21-jre unzip +wget https://dl.google.com/android/repository/commandlinetools-linux-13114758_latest.zip +unzip -o commandlinetools-linux-13114758_latest.zip +mkdir -p $WORKSPACE/Android/Sdk +export ANDROID_HOME=$WORKSPACE/Android +export ANDROID_SDK_HOME=$ANDROID_HOME/Sdk +$WORKSPACE/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_SDK_HOME --install "platform-tools" "platforms;android-35" "build-tools;35.0.0" + {{< /tab >}} + {{< tab header="MacOS">}} +cd $WORKSPACE +wget https://dl.google.com/android/repository/commandlinetools-mac-13114758_latest.zip +unzip -o commandlinetools-linux-13114758_latest.zip +mkdir -p $WORKSPACE/Android/Sdk +export ANDROID_HOME=$WORKSPACE/Android +export ANDROID_SDK_HOME=$ANDROID_HOME/Sdk +$WORKSPACE/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_SDK_HOME --install "platform-tools" "platforms;android-35" "build-tools;35.0.0" + {{< /tab >}} +{{< /tabpane >}} To run the model on Android, install Android Native Development Kit (Android NDK): {{< tabpane code=true >}} {{< tab header="Linux">}} cd $WORKSPACE -wget https://dl.google.com/android/repository/android-ndk-r25b-linux.zip -unzip android-ndk-r25b-linux.zip +wget https://dl.google.com/android/repository/android-ndk-r27b-linux.zip +unzip android-ndk-r27b-linux.zip {{< /tab >}} {{< tab header="MacOS">}} cd $WORKSPACE -wget https://dl.google.com/android/repository/android-ndk-r25b-darwin.zip -unzip android-ndk-r25b-darwin.zip +wget https://dl.google.com/android/repository/android-ndk-r27b-darwin.zip +unzip android-ndk-r27b-darwin.zip {{< /tab >}} {{< /tabpane >}} @@ -119,12 +141,12 @@ For easier access and execution of Android NDK tools, add these to the `PATH` an {{< tabpane code=true >}} {{< tab header="Linux">}} -export NDK_PATH=$WORKSPACE/android-ndk-r25b/ +export NDK_PATH=$WORKSPACE/android-ndk-r27b/ export ANDROID_NDK_HOME=$NDK_PATH export PATH=$NDK_PATH/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH {{< /tab >}} {{< tab header="MacOS">}} -export NDK_PATH=$WORKSPACE/android-ndk-r25b/ +export NDK_PATH=$WORKSPACE/android-ndk-r27b/ export ANDROID_NDK_HOME=$NDK_PATH export PATH=$NDK_PATH/toolchains/llvm/prebuilt/darwin-x86_64/bin/:$PATH {{< /tab >}} diff --git a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/2-testing-model.md b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/2-testing-model.md index beef52daea..f36934bf2e 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/2-testing-model.md +++ b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/2-testing-model.md @@ -23,7 +23,7 @@ You may need to fill out a form with your contact information to use the model: Download and copy the configuration file `model_config.json` and the model itself, `model.ckpt`, to your workspace directory, and verify they exist by running the command: ```bash -ls $WORKSPACE/model_config.json $WORKSPACE/model.ckpt +ls -lha $WORKSPACE/model_config.json $WORKSPACE/model.ckpt ``` You can learn more about this model [here](https://huggingface.co/stabilityai/stable-audio-open-small). diff --git a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/3-converting-model.md b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/3-converting-model.md index af2aa9f56d..e1af045a54 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/3-converting-model.md +++ b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/3-converting-model.md @@ -35,6 +35,8 @@ To avoid dependency issues, create a virtual environment. For example, you can u cd $WORKSPACE python3.10 -m venv .venv source .venv/bin/activate +export PYTHON_BIN_PATH=$WORKSPACE/.venv/bin/python3 +export PYTHON_LIB_PATH=$WORKSPACE/.venv/lib/python3.10/site-packages ``` ## Clone the examples repository diff --git a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/5-creating-simple-program-for-android.md b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/5-creating-simple-program-for-android.md index 38b2c4850d..2ff7a09244 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/5-creating-simple-program-for-android.md +++ b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/5-creating-simple-program-for-android.md @@ -81,12 +81,13 @@ From there, you can then run the audiogen application, which requires just three * **Model Path:** The directory containing your LiteRT models and spiece.model files * **Prompt:** A text description of the desired audio (e.g., warm arpeggios on house beats 120BPM with drums effect) * **CPU Threads:** The number of CPU threads to use (e.g., 4) +* **Seed:** A random number to seed the generation (e.g. 1234) Play around with the advice from [Download and test the model](../2-testing-model) section. ```bash cd /data/local/tmp/app -LD_LIBRARY_PATH=. ./audiogen . "warm arpeggios on house beats 120BPM with drums effect" 4 +LD_LIBRARY_PATH=. ./audiogen . "warm arpeggios on house beats 120BPM with drums effect" 4 1234 exit ``` diff --git a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/_index.md b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/_index.md index 87385c6769..d395b19c0e 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/_index.md +++ b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/_index.md @@ -13,9 +13,9 @@ learning_objectives: - Run the application on an Android smartphone and generate an audio snippet. prerequisites: - - A Linux-based x86 or macOS development machine with at least 8 GB of RAM (tested on Ubuntu 20.04.4 LTS with x86_64). + - A Linux-based x86 or macOS development machine with at least 8 GB of RAM and 50 GB of disk space (tested on Ubuntu 22.04 with x86_64). - A [HuggingFace](https://huggingface.co/) account. - - An Android phone and a cable to connect it to your development machine. + - An Android phone in [developer mode](https://developer.android.com/studio/debug/dev-options) and a cable to connect it to your development machine. author: - Nina Drozd diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/00_overview.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/00_overview.md new file mode 100644 index 0000000000..2e3ddadafe --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/00_overview.md @@ -0,0 +1,37 @@ +--- +title: Overview +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## The AFM-4.5B model + +AFM-4.5B is a 4.5-billion-parameter foundation model designed to balance accuracy, efficiency, and broad language coverage. Trained on nearly 7 trillion tokens of carefully filtered data, it performs well across a wide range of languages, including Arabic, English, French, German, Hindi, Italian, Korean, Mandarin, Portuguese, Russian, and Spanish. + +In this Learning Path, you'll deploy AFM-4.5B using [Llama.cpp](https://github.com/ggerganov/llama.cpp) on an Arm-based AWS Graviton4 instance. You’ll walk through the full workflow, from setting up your environment and compiling the runtime, to downloading, quantizing, and running inference on the model. You'll also evaluate model quality using perplexity, a common metric for measuring how well a language model predicts text. + +This hands-on guide helps developers build cost-efficient, high-performance LLM applications on modern Arm server infrastructure using open-source tools and real-world deployment practices. + +### LLM deployment workflow on Arm Graviton4 + +- **Provision compute**: launch an EC2 instance using a Graviton4-based instance type (for example, `c8g.4xlarge`) + +- **Set up your environment**: install the required build tools and dependencies (such as CMake, Python, and Git) + +- **Build the inference engine**: clone the [Llama.cpp](https://github.com/ggerganov/llama.cpp) repository and compile the project for your Arm-based environment + +- **Prepare the model**: download the **AFM-4.5B** model files from Hugging Face and use Llama.cpp's quantization tools to reduce model size and optimize performance + +- **Run inference**: load the quantized model and run sample prompts using Llama.cpp. + +- **Evaluate model quality**: calculate **perplexity** or use other metrics to assess model performance + +{{< notice Note>}} +You can reuse this deployment flow with other models supported by Llama.cpp by swapping out the model file and adjusting quantization settings. +{{< /notice >}} + + + + diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/01_launching_a_graviton4_instance.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/01_launching_a_graviton4_instance.md index 772e4d96c5..9769329b62 100644 --- a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/01_launching_a_graviton4_instance.md +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/01_launching_a_graviton4_instance.md @@ -1,6 +1,6 @@ --- -title: Launching a Graviton4 instance -weight: 2 +title: Provision your Graviton4 environment +weight: 3 ### FIXED, DO NOT MODIFY layout: learningpathall @@ -8,164 +8,111 @@ layout: learningpathall ## Requirements - - An AWS account +Before you begin, make sure you have the following: - - Access to launch an EC2 instance of type `c8g.4xlarge` (or larger) with at least 128 GB of storage +- An AWS account +- Permission to launch a Graviton4 EC2 instance of type `c8g.4xlarge` (or larger) +- At least 128 GB of available storage -For more information about creating an EC2 instance using AWS refer to [Getting Started with AWS](/learning-paths/servers-and-cloud-computing/csp/aws/). +If you're new to EC2, check out the Learning Path [Getting Started with AWS](/learning-paths/servers-and-cloud-computing/csp/aws/). -## AWS Console Steps +## Create an SSH key pair -Follow these steps to launch your EC2 instance using the AWS Management Console: +To deploy the Arcee AFM-4.5B model, you need an EC2 instance running on Arm-based Graviton4 hardware. -### Step 1: Create an SSH Key Pair +To do this, start by signing in to the [AWS Management Console](https://console.aws.amazon.com), change to your preferred region, and navigate to the **EC2** service. -1. **Navigate to EC2 Console** +From there, you can create an SSH key pair that allows you to connect to your instance securely. - - Go to the [AWS Management Console](https://console.aws.amazon.com) +## Set up secure access - - Search for "EC2" and click on "EC2" service +Open the **Key Pairs** section under **Network & Security** in the sidebar, and create a new key pair named `arcee-graviton4-key`. -2. **Create Key Pair** +Next, select **RSA** as the key type, and **.pem** as the file format. Once you create the key, your browser will download the `.pem` file automatically. - - In the left navigation pane, click "Key Pairs" under "Network & Security" +To ensure the key remains secure and accessible, move the `.pem` file to your SSH configuration directory, and update its permissions to restrict access. - - Click "Create key pair" +To do this, on macOS or Linux, run: - - Enter name: `arcee-graviton4-key` +```bash +mkdir -p ~/.ssh +mv arcee-graviton4-key.pem ~/.ssh/ +chmod 400 ~/.ssh/arcee-graviton4-key.pem +``` - - Select "RSA" as the key pair type +## Launch and configure the EC2 instance - - Select ".pem" as the private key file format +In the left sidebar of the EC2 dashboard, select **Instances**, and then **Launch instances**. - - Click "Create key pair" +Use the following settings to configure your instance: - - The private key file will automatically download to your computer +- **Name**: `Arcee-Graviton4-Instance` +- **Application and OS image**: + - Select the **Quick Start** tab + - Select **Ubuntu Server 24.04 LTS (HVM), SSD Volume Type** + - Ensure the architecture is set to **64-bit (ARM)** +- **Instance type**: select `c8g.4xlarge` or larger +- **Key pair name**: select `arcee-graviton4-key` from the list -3. **Secure the Key File** +## Configure network - - Move the downloaded `.pem` file to the SSH configuration directory +To enable internet access, choose a VPC with at least one public subnet. - ```bash - mkdir -p ~/.ssh - mv arcee-graviton4-key.pem ~/.ssh - ``` +Then select a public subnet from the list. - - Set proper permissions on macOS or Linux: +Under **Auto-assign public IP**, select **Enable**. - ```bash - chmod 400 ~/.ssh/arcee-graviton4-key.pem - ``` +## Configure firewall -### Step 2: Launch EC2 Instance +Select **Create security group**. Then select **Allow SSH traffic from** and select **My IP**. -1. **Start Instance Launch** +{{% notice Note %}} +You'll only be able to connect to the instance from your current host, which is the most secure setting. Avoid selecting **Anywhere** unless absolutely necessary, as this setting allows anyone on the internet to attempt a connection. - - In the left navigation pane, click "Instances" under "Instances" - - - Click "Launch instances" button - -2. **Configure Instance Details** - - - **Name and tags**: Enter `Arcee-Graviton4-Instance` as the instance name - - - **Application and OS Images**: - - Click "Quick Start" tab - - - Select "Ubuntu" - - - Choose "Ubuntu Server 24.04 LTS (HVM), SSD Volume Type" - - - **Important**: Ensure the architecture shows "64-bit (ARM)" for Graviton compatibility - - - **Instance type**: - - Click on "Select instance type" - - - Select `c8g.4xlarge` or larger - -3. **Configure Key Pair** - - In "Key pair name", select the SSH keypair you created earlier (`Arcee-Graviton4-Instance`) - -4. **Configure Network Settings** - - - **Network**: Select a VPC with a least one public subnet. - - - **Subnet**: Select a public subnet in the VPC - - - **Auto-assign Public IP**: Enable - - - **Firewall (security groups)** - - - Click on "Create security group" - - - Click on "Allow SSH traffic from" - - - In the dropdown list, select "My IP". - - -{{% notice Notes %}} -You will only be able to connect to the instance from your current host, which is the safest setting. Selecting "Anywhere" allows anyone on the Internet to attempt to connect; use at your own risk. - -Although this demonstration only requires SSH access, it is possible to use one of your existing security groups as long as it allows SSH traffic. +You only need SSH access for this Learning Path. If you already have a security group that allows inbound SSH traffic, you can reuse it. {{% /notice %}} -5. **Configure Storage** - - - **Root volume**: - - Size: `128` GB - - - Volume type: `gp3` - -7. **Review and Launch** - - - Review all settings in the "Summary" section +## Configure storage - - Click "Launch instance" +Set the **root volume size** to `128` GB, then select **gp3** as the volume type. -### Step 3: Monitor Instance Launch +## Review and launch the instance -1. **View Launch Status** +Review all your configuration settings, and when you're ready, select **Launch instance** to create your EC2 instance. - After a few seconds, you should see a message similar to this one: +## Monitor the instance launch - `Successfully initiated launch of instance (i-)` +After a few seconds, you should see a confirmation message like this: - If instance launch fails, please review your settings and try again. +``` +Successfully initiated launch of instance (i-xxxxxxxxxxxxxxxxx) +``` -2. **Get Connection Information** +If the launch fails, double-check the instance type, permissions, and network settings. - - Click on the instance id, or look for the instance in the Instances list in the EC2 console. +To retrieve the connection details, go to the **Instances** list in the EC2 dashboard. - - In the "Details" tab of the instance, note the "Public DNS" host name +Then select your instance by selecting **Instance ID**. - - This is the host name you'll use to connect via SSH, aka `PUBLIC_DNS_HOSTNAME` +In the **Details** tab, copy the **Public DNS** value - you’ll use this to connect through SSH. -### Step 4: Connect to Your Instance +## Connect to your instance -1. **Open Terminal/Command Prompt** +Open a terminal and connect to the instance using the SSH key you downloaded earlier: -2. **Connect via SSH** - ```bash - ssh -i ~/.ssh/arcee-graviton4-key.pem ubuntu@ - ``` +```bash +ssh -i ~/.ssh/arcee-graviton4-key.pem ubuntu@ +``` -3. **Accept Security Warning** +When prompted, type `yes` to confirm the connection. - - When prompted about authenticity of host, type `yes` - - - You should now be connected to your Ubuntu instance - -### Important Notes - -- **Region Selection**: Ensure you're in your preferred AWS region before launching - -- **AMI Selection**: The Ubuntu 24.04 LTS AMI must be ARM64 compatible for Graviton processors - -- **Security**: Think twice about allowing SSH from anywhere (0.0.0.0/0). It is strongly recommended to restrict access to your IP address. - -- **Storage**: The 128GB EBS volume is sufficient for the Arcee model and dependencies - -- **Backup**: Consider creating AMIs or snapshots for backup purposes +You should now be connected to your Ubuntu instance running on Graviton4. +{{% notice Note %}} +**Region**: make sure you're launching in your preferred AWS region. +**AMI**: confirm that the selected AMI supports the Arm64 architecture. +**Security**: for best practice, restrict SSH access to your own IP. +**Storage**: 128 GB is sufficient for the AFM-4.5B model and dependencies. +**Backup**: consider creating an AMI or snapshot after setup is complete. +{{% /notice %}} diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/02_setting_up_the_instance.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/02_setting_up_the_instance.md index c85c8f0bc4..8b8c53c779 100644 --- a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/02_setting_up_the_instance.md +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/02_setting_up_the_instance.md @@ -1,51 +1,58 @@ --- -title: Setting up the instance -weight: 3 +title: Configure your Graviton4 environment +weight: 4 ### FIXED, DO NOT MODIFY layout: learningpathall --- -In this step, you'll set up the Graviton4 instance with all the necessary tools and dependencies required to build and run the Arcee Foundation Model. This includes installing the build tools and Python environment. +In this step, you'll set up the Graviton4 instance with the tools and dependencies required to build and run the Arcee Foundation Model. This includes installing system packages and a Python environment. -## Step 1: Update Package List +## Update the package list + +Run the following command to update your local APT package index: ```bash sudo apt-get update ``` -This command updates the local package index from the repositories: +This step ensures you have the most recent metadata about available packages, including versions and dependencies. It helps prevent conflicts when installing new packages. -- Downloads the latest package lists from all configured APT repositories -- Ensures you have the most recent information about available packages and their versions -- This is a best practice before installing new packages to avoid potential conflicts -- The package index contains metadata about available packages, their dependencies, and version information +## Install system dependencies -## Step 2: Install System Dependencies +Install the build tools and Python environment: ```bash sudo apt-get install cmake gcc g++ git python3 python3-pip python3-virtualenv libcurl4-openssl-dev unzip -y ``` -This command installs all the essential development tools and dependencies: +This command installs the following tools and dependencies: + +- **CMake**: cross-platform build system generator used to compile and build Llama.cpp + +- **GCC and G++**: GNU C and C++ compilers for compiling native code + +- **Git**: version control system for cloning repositories + +- **Python 3**: Python interpreter for running Python-based tools and scripts + +- **Pip**: Python package manager + +- **Virtualenv**: tool for creating isolated Python environments + +- **libcurl4-openssl-dev**: development files for the curl HTTP library -- **cmake**: Cross-platform build system generator used to compile Llama.cpp -- **gcc & g++**: GNU C and C++ compilers for building native code -- **git**: Version control system for cloning repositories -- **python3**: Python interpreter for running Python-based tools and scripts -- **python3-pip**: Python package installer for managing Python dependencies -- **python3-virtualenv**: Tool for creating isolated Python environments -- **libcurl4-openssl-dev**: client-side URL transfer library +- **Unzip**: tool to extract `.zip` files (used in some model downloads) -The `-y` flag automatically answers "yes" to prompts, making the installation non-interactive. +The `-y` flag automatically approves the installation of all packages without prompting. -## What's Ready Now? +## Ready for build and deployment -After completing these steps, your Graviton4 instance has: +After completing the setup, your instance includes the following tools and environments: - A complete C/C++ development environment for building Llama.cpp -- Python 3 with pip for managing Python packages +- Python 3, pip, and virtualenv for managing Python tools and environments - Git for cloning repositories -- All necessary build tools for compiling optimized ARM64 binaries +- All required dependencies for compiling optimized Arm64 binaries -The system is now prepared for the next steps: building Llama.cpp and downloading the Arcee Foundation Model. +You're now ready to build Llama.cpp and download the Arcee Foundation Model. diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/03_building_llama_cpp.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/03_building_llama_cpp.md index b4cdcf0f7a..95fa1e416f 100644 --- a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/03_building_llama_cpp.md +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/03_building_llama_cpp.md @@ -1,16 +1,17 @@ --- -title: Building Llama.cpp -weight: 4 +title: Build Llama.cpp +weight: 5 ### FIXED, DO NOT MODIFY layout: learningpathall --- +## Build the Llama.cpp inference engine -In this step, you'll build Llama.cpp from source. Llama.cpp is a high-performance C++ implementation of the LLaMA model that's optimized for inference on various hardware platforms, including Arm-based processors like Graviton4. +In this step, you'll build Llama.cpp from source. Llama.cpp is a high-performance C++ implementation of the LLaMA model, optimized for inference on a range of hardware platforms,including Arm-based processors like AWS Graviton4. -Even though AFM-4.5B has a custom model architecture, we're able to use the vanilla version of Llama.cpp as the Arcee AI team has contributed the appropriate modeling code. +Even though AFM-4.5B uses a custom model architecture, you can still use the standard Llama.cpp repository - Arcee AI has contributed the necessary modeling code upstream. -## Step 1: Clone the Repository +## Clone the repository ```bash git clone https://github.com/ggerganov/llama.cpp @@ -18,28 +19,28 @@ git clone https://github.com/ggerganov/llama.cpp This command clones the Llama.cpp repository from GitHub to your local machine. The repository contains the source code, build scripts, and documentation needed to compile the inference engine. -## Step 2: Navigate to the Project Directory +## Navigate to the project directory ```bash cd llama.cpp ``` -Change into the llama.cpp directory to run the build process. This directory contains the `CMakeLists.txt` file and source code structure. +Change into the llama.cpp directory to run the build process. This directory contains the `CMakeLists.txt` file and all source code. -## Step 3: Configure the Build with CMake +## Configure the build with CMake ```bash cmake -B . ``` -This command uses CMake to configure the build system: +This command configures the build system using CMake: -- `-B .` specifies that the build files should be generated in the current directory -- CMake will detect your system's compiler, libraries, and hardware capabilities -- It will generate the appropriate build files (Makefiles on Linux) based on your system configuration +- `-B .` tells CMake to generate build files in the current directory +- CMake detects your system's compiler, libraries, and hardware capabilities +- It produces Makefiles (on Linux) or platform-specific build scripts for compiling the project -The CMake output should include the information below, indicating that the build process will leverage the Neoverse V2 architecture's specialized instruction sets designed for AI/ML workloads. These optimizations are crucial for achieving optimal performance on Graviton4: +If you're running on Graviton4, the CMake output should include hardware-specific optimizations targeting the Neoverse V2 architecture. These optimizations are crucial for achieving high performance on Graviton4: ```output -- ARM feature DOTPROD enabled @@ -50,33 +51,40 @@ The CMake output should include the information below, indicating that the build -- Adding CPU backend variant ggml-cpu: -mcpu=neoverse-v2+crc+sve2-aes+sve2-sha3+dotprod+i8mm+sve ``` -- **DOTPROD: Dot Product** - Hardware-accelerated dot product operations for neural network computations -- **SVE: Scalable Vector Extension** - Advanced vector processing capabilities that can handle variable-length vectors up to 2048 bits, providing significant performance improvements for matrix operations -- **MATMUL_INT8: Matrix multiplication units** - Dedicated hardware for efficient matrix operations common in transformer models, accelerating the core computations of large language models -- **FMA: Fused Multiply-Add - Optimized floating-point operations that combine multiplication and addition in a single instruction -- **FP16 Vector Arithmetic - Hardware support for 16-bit floating-point vector operations, reducing memory usage while maintaining good numerical precision +These features enable advanced CPU instructions that accelerate inference performance on Arm64: -## Step 4: Compile the Project +- **DOTPROD: Dot Product**: hardware-accelerated dot product operations for neural network workloads + +- **SVE (Scalable Vector Extension)**: advanced vector processing capabilities that can handle variable-length vectors up to 2048 bits, providing significant performance improvements for matrix operations + +- **MATMUL_INT8**: integer matrix multiplication units optimized for transformers + +- **FMA**: fused multiply-add operations to speed up floating-point math + +- **FP16 vector arithmetic**: 16-bit floating-point vector operations to reduce memory use without compromising precision + +## Compile the project ```bash cmake --build . --config Release -j16 ``` -This command compiles the Llama.cpp project: -- `--build .` tells CMake to build the project using the files in the current directory -- `--config Release` specifies a Release build configuration, which enables optimizations and removes debug symbols +This command compiles the Llama.cpp source code: + +- `--build .` tells CMake to build the project in the current directory +- `--config Release` enables optimizations and strips debug symbols - `-j16` runs the build with 16 parallel jobs, which speeds up compilation on multi-core systems like Graviton4 -The build process will compile the C++ source code into executable binaries optimized for your ARM64 architecture. This should only take a minute. +The build process compiles the C++ source code into executable binaries optimized for the Arm64 architecture. Compilation typically takes under a minute. -## What is built? +## Key binaries after compilation -After successful compilation, you'll have several key command-line executables in the `bin` directory: -- `llama-cli` - The main inference executable for running LLaMA models -- `llama-server` - A web server for serving model inference over HTTP -- `llama-quantize` - a tool for model quantization to reduce memory usage -- Various utility programs for model conversion and optimization +After compilation, you'll find several key command-line tools in the `bin` directory: +- `llama-cli`: the main inference executable for running LLaMA models +- `llama-server`: a web server for serving model inference over HTTP +- `llama-quantize`: a tool for model quantization to reduce memory usage +- Additional utilities for model conversion and optimization -You can find more information in the llama.cpp [GitHub repository](https://github.com/ggml-org/llama.cpp/tree/master/tools). +You can find more tools and usage details in the llama.cpp [GitHub repository](https://github.com/ggml-org/llama.cpp/tree/master/tools). -These binaries are specifically optimized for ARM64 architecture and will provide excellent performance on your Graviton4 instance. +These binaries are specifically optimized for the Arm architecture and will provide excellent performance on your Graviton4 instance. diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/04_install_python_dependencies_for_llama_cpp.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/04_install_python_dependencies_for_llama_cpp.md index f21d281408..b680dcf7eb 100644 --- a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/04_install_python_dependencies_for_llama_cpp.md +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/04_install_python_dependencies_for_llama_cpp.md @@ -1,66 +1,80 @@ --- -title: Installing Python dependencies for llama.cpp -weight: 5 +title: Install Python dependencies +weight: 6 ### FIXED, DO NOT MODIFY layout: learningpathall --- +## Overview In this step, you'll set up a Python virtual environment and install the required dependencies for working with Llama.cpp. This ensures you have a clean, isolated Python environment with all the necessary packages for model optimization. -## Step 1: Create a Python Virtual Environment +## Create a Python virtual environment ```bash virtualenv env-llama-cpp ``` -This command creates a new Python virtual environment named `env-llama-cpp`: -- Virtual environments provide isolated Python environments that prevent conflicts between different projects -- The `env-llama-cpp` directory will contain its own Python interpreter and package installation space -- This isolation ensures that the Llama.cpp dependencies won't interfere with other Python projects on your system -- Virtual environments are essential for reproducible development environments +This command creates a new Python virtual environment named `env-llama-cpp`, which has the following benefits: +- Provides an isolated Python environment to prevent package conflicts between projects +- Creates a local directory containing its own Python interpreter and installation space +- Ensures Llama.cpp dependencies don’t interfere with your global Python setup +- Supports reproducible and portable development environments -## Step 2: Activate the Virtual Environment +## Activate the virtual environment + +Run the following command to activate the virtual environment: ```bash source env-llama-cpp/bin/activate ``` +This command does the following: + +- Runs the activation script, which modifies your shell environment +- Updates your shell prompt to show `env-llama-cpp`, indicating the environment is active +- Updates `PATH` to use so the environment’s Python interpreter +- Ensures all `pip` commands install packages into the isolated environment -This command activates the virtual environment: -- The `source` command executes the activation script, which modifies your current shell environment -- Depending on you sheel, your command prompt may change to show `(env-llama-cpp)` at the beginning, indicating the active environment. This will be reflected in the following commands. -- All subsequent `pip` commands will install packages into this isolated environment -- The `PATH` environment variable is updated to prioritize the virtual environment's Python interpreter +## Upgrade pip to the latest version -## Step 3: Upgrade pip to the Latest Version +Before installing dependencies, it’s a good idea to upgrade pip: ```bash pip install --upgrade pip ``` +This command: -This command ensures you have the latest version of pip: -- Upgrading pip helps avoid compatibility issues with newer packages -- The `--upgrade` flag tells pip to install the newest available version -- This is a best practice before installing project dependencies -- Newer pip versions often include security fixes and improved package resolution +- Ensures you have the latest version of pip +- Helps avoid compatibility issues with modern packages +- Applies the `--upgrade` flag to fetch and install the newest release +- Brings in security patches and better dependency resolution logic -## Step 4: Install Project Dependencies +## Install project dependencies + +Use the following command to install all required Python packages: ```bash pip install -r requirements.txt ``` -This command installs all the Python packages specified in the requirements.txt file: -- The `-r` flag tells pip to read the package list from the specified file -- `requirements.txt` contains a list of Python packages and their version specifications -- This ensures everyone working on the project uses the same package versions -- The installation will include packages needed for model loading, inference, and any Python bindings for Llama.cpp +This command does the following: + +- Uses the `-r` flag to read the list of dependencies from `requirements.txt` +- Installs the exact package versions required for the project +- Ensures consistency across development environments and contributors +- Includes packages for model loading, inference, and Python bindings for `llama.cpp` + +This step sets up everything you need to run AFM-4.5B in your Python environment. + +## What the environment includes + +After the installation completes, your virtual environment includes: +- **NumPy**: for numerical computations and array operations +- **Requests**: for HTTP operations and API calls +- **Other dependencies**: additional packages required by llama.cpp's Python bindings and utilities +Your environment is now ready to run Python scripts that integrate with the compiled Llama.cpp binaries -## What is installed? +{{< notice Tip >}} +Before running any Python commands, make sure your virtual environment is activated. {{< /notice >}} -After successful installation, your virtual environment will contain: -- **NumPy**: For numerical computations and array operations -- **Requests**: For HTTP operations and API calls -- **Other dependencies**: Specific packages needed for Llama.cpp Python integration -The virtual environment is now ready for running Python scripts that interact with the compiled Llama.cpp binaries. Remember to always activate the virtual environment (`source env-llama-cpp/bin/activate`) before running any Python code related to this project. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/05_downloading_and_optimizing_afm45b.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/05_downloading_and_optimizing_afm45b.md index e293e74ff7..3feb85647e 100644 --- a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/05_downloading_and_optimizing_afm45b.md +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/05_downloading_and_optimizing_afm45b.md @@ -1,91 +1,97 @@ --- -title: Downloading and optimizing AFM-4.5B -weight: 6 +title: Download and optimize the AFM-4.5B model +weight: 7 ### FIXED, DO NOT MODIFY layout: learningpathall --- -In this step, you'll download the AFM-4.5B model from Hugging Face, convert it to the GGUF format for use with Llama.cpp, and create quantized versions to optimize memory usage and inference speed. +In this step, you’ll download the AFM-4.5B model from Hugging Face, convert it to the GGUF format for compatibility with `llama.cpp`, and generate quantized versions to optimize memory usage and improve inference speed. -The first release of the [Arcee Foundation Model](https://www.arcee.ai/blog/announcing-the-arcee-foundation-model-family) family, [AFM-4.5B](https://www.arcee.ai/blog/deep-dive-afm-4-5b-the-first-arcee-foundational-model) is a 4.5-billion-parameter frontier model that delivers excellent accuracy, strict compliance, and very high cost-efficiency. It was trained on almost 7 trillion tokens of clean, rigorously filtered data, and has been tested across a wide range of languages, including Arabic, English, French, German, Hindi, Italian, Korean, Mandarin, Portuguese, Russian, and Spanish +Make sure to activate your virtual environment before running any commands. The instructions below walk you through downloading and preparing the model for efficient use on AWS Graviton4. -Here are the steps to download and optimize the model for AWS Graviton4. Make sure to run them in the virtual environment you created at the previous step. - -## Step 1: Install the Hugging Face libraries +## Install the Hugging Face libraries ```bash pip install huggingface_hub hf_xet ``` -This command installs the Hugging Face Hub Python library, which provides tools for downloading models and datasets from the Hugging Face platform. The library includes the `huggingface-cli` command-line interface that you can use to download the AFM-4.5B model. +This command installs: + +- `huggingface_hub`: Python client for downloading models and datasets +- `hf_xet`: Git extension for fetching large model files stored on Hugging Face -## Step 2: Download the AFM-4.5B Model +These tools include the `huggingface-cli` command-line interface you'll use next. + +## Download the AFM-4.5B model ```bash huggingface-cli download arcee-ai/afm-4.5B --local-dir models/afm-4-5b ``` -This command downloads the AFM-4.5B model from the Hugging Face Hub: -- `arcee-ai/afm-4.5B` is the model identifier on Hugging Face Hub -- `--local-dir model/afm-4-5b` specifies the local directory where the model files will be stored -- The download includes the model weights, configuration files, and tokenizer data -- This is a 4.5 billion parameter model, so the download may take several minutes depending on your internet connection +This command downloads the model to the `models/afm-4-5b` directory: +- `arcee-ai/afm-4.5B` is the Hugging Face model identifier. +- The download includes the model weights, configuration files, and tokenizer data. +- This is a 4.5 billion parameter model, so the download can take several minutes depending on your internet connection. -## Step 3: Convert to GGUF Format +## Convert to GGUF format ```bash python3 convert_hf_to_gguf.py models/afm-4-5b deactivate ``` -The first command converts the downloaded Hugging Face model to the GGUF (GGML Universal Format) format: -- `convert_hf_to_gguf.py` is a conversion script that comes with Llama.cpp -- `models/afm-4-5b` is the input directory containing the Hugging Face model files -- The script reads the model architecture, weights, and configuration from the Hugging Face format -- It outputs a single `afm-4-5B-F16.gguf` ~15GB file in the `models/afm-4-5b/` directory -- GGUF is the native format used by Llama.cpp and provides efficient loading and inference +This command converts the downloaded Hugging Face model to GGUF (GGML Universal Format): +- `convert_hf_to_gguf.py` is a conversion script that comes with Llama.cpp. +- `models/afm-4-5b` is the input directory containing the Hugging Face model files. +- The script reads the model architecture, weights, and configuration from the Hugging Face format. +- It outputs a single `afm-4-5B-F16.gguf` ~15GB file in the same `models/afm-4-5b/` directory. +- GGUF is the native format for Llama.cpp, optimized for efficient loading and inference. Next, deactivate the Python virtual environment as future commands won't require it. -## Step 4: Create Q4_0 Quantized Version +## Create Q4_0 Quantized Version ```bash bin/llama-quantize models/afm-4-5b/afm-4-5B-F16.gguf models/afm-4-5b/afm-4-5B-Q4_0.gguf Q4_0 ``` This command creates a 4-bit quantized version of the model: -- `llama-quantize` is the quantization tool from Llama.cpp -- `afm-4-5B-F16.gguf` is the input GGUF model file in 16-bit precision -- `Q4_0` specifies 4-bit quantization with zero-point quantization -- This reduces the model size by approximately 45% (from ~15GB to ~8GB) -- The quantized model will use less memory and run faster, though with a small reduction in accuracy -- The output file will be named `afm-4-5B-Q4_0.gguf` +- `llama-quantize` is the quantization tool from Llama.cpp. +- `afm-4-5B-F16.gguf` is the input GGUF model file in 16-bit precision. +- `Q4_0` applies zero-point 4-bit quantization. +- This reduces the model size by approximately 45% (from ~15GB to ~8GB). +- The quantized model will use less memory and run faster, though with a small reduction in accuracy. +- The output file will be `afm-4-5B-Q4_0.gguf`. + +## Arm optimization -**ARM Optimization**: ARM has contributed highly optimized kernels for Q4_0 quantization that leverage the Neoverse v2 instruction sets. These low-level math routines accelerate typical deep learning operations, providing significant performance improvements on ARM-based processors like Graviton4. +Arm has contributed optimized kernels for Q4_0 that use Neoverse V2 instruction sets. These low-level routines accelerate math operations, delivering strong performance on Graviton4. -These instruction sets enable Llama.cpp to perform quantized operations much faster than generic implementations, making ARM processors highly competitive for inference workloads. +These instruction sets allow Llama.cpp to run quantized operations significantly faster than generic implementations, making Arm processors a competitive choice for inference workloads. -## Step 5: Create Q8_0 Quantized Version +## Create a Q8_0 quantized version ```bash bin/llama-quantize models/afm-4-5b/afm-4-5B-F16.gguf models/afm-4-5b/afm-4-5B-Q8_0.gguf Q8_0 ``` This command creates an 8-bit quantized version of the model: -- `Q8_0` specifies 8-bit quantization with zero-point quantization -- This reduces the model size by approximately 70% (from ~15GB to ~4.4GB) -- The 8-bit version provides a better balance between memory usage and accuracy compared to 4-bit -- The output file will be named `afm-4-5B-Q8_0.gguf` -- This version is often preferred for production use when memory constraints allow +- `Q8_0` specifies 8-bit quantization with zero-point compression. +- This reduces the model size by approximately 70% (from ~15GB to ~4.4GB). +- The 8-bit version provides a better balance between memory usage and accuracy than 4-bit quantization. +- The output file is named `afm-4-5B-Q8_0.gguf`. +- Commonly used in production scenarios where memory resources are available. + +## Arm optimization -**ARM Optimization**: Similar to Q4_0, ARM has contributed optimized kernels for Q8_0 quantization that take advantage of Neoverse v2 instruction sets. These optimizations provide excellent performance for 8-bit operations while maintaining higher accuracy compared to 4-bit quantization. +Similar to Q4_0, Arm has contributed optimized kernels for Q8_0 quantization that take advantage of Neoverse V2 instruction sets. These optimizations provide excellent performance for 8-bit operations while maintaining higher accuracy compared to 4-bit quantization. -## What is available now? +## Model files ready for inference After completing these steps, you'll have three versions of the AFM-4.5B model: - `afm-4-5B-F16.gguf` - The original full-precision model (~15GB) -- `afm-4-5B-Q4_0.gguf` - 4-bit quantized version (~8GB) for memory-constrained environments -- `afm-4-5B-Q8_0.gguf` - 8-bit quantized version (~4.4GB) for balanced performance and memory usage +- `afm-4-5B-Q4_0.gguf` - 4-bit quantized version (~4.4GB) for memory-constrained environments +- `afm-4-5B-Q8_0.gguf` - 8-bit quantized version (~8GB) for balanced performance and memory usage -These models are now ready to be used with the Llama.cpp inference engine for text generation and other language model tasks. \ No newline at end of file +These models are now ready to be used with the `llama.cpp` inference engine for text generation and other language model tasks. diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/06_running_inference.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/06_running_inference.md index b1c9aeb471..0e84e97f56 100644 --- a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/06_running_inference.md +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/06_running_inference.md @@ -1,37 +1,39 @@ --- -title: Running inference with AFM-4.5B -weight: 7 +title: Run inference with AFM-4.5B +weight: 8 ### FIXED, DO NOT MODIFY layout: learningpathall --- -Now that you have the AFM-4.5B models in GGUF format, you can run inference using various Llama.cpp tools. In this step, you'll explore different ways to interact with the model for text generation, benchmarking, and evaluation. +Now that you have the AFM-4.5B models in GGUF format, you can run inference using various Llama.cpp tools. In this step, you'll explore how to generate text, benchmark performance, and interact with the model through both command-line and HTTP APIs. -## Using llama-cli for Interactive Text Generation -The `llama-cli` tool provides an interactive command-line interface for text generation. This is perfect for testing the model's capabilities and having conversations with it. +## Use llama-cli for interactive text generation -### Basic Usage +The `llama-cli` tool provides an interactive command-line interface for text generation. This is ideal for quick testing and hands-on exploration of the model's behavior. + +## Basic usage ```bash bin/llama-cli -m models/afm-4-5b/afm-4-5B-Q8_0.gguf -n 256 --color ``` -This command starts an interactive session with the model: +This command starts an interactive session: -- `-m models/afm-4-5b/afm-4-5B-Q8_0.gguf` specifies the model file to load -- `-n 512` sets the maximum number of tokens to generate per response +- `-m` (model file path) specifies the model file to load +- `-n 256` sets the maximum number of tokens to generate per response +- `--color` enables colored terminal output - The tool will prompt you to enter text, and the model will generate a response -In this example, `llama-cli` uses 16 vCPUs. You can try different values with `-t `. +In this example, `llama-cli` uses 16 vCPUs. You can try different values with `-t `. -### Example Interactive Session +### Example interactive session Once you start the interactive session, you can have conversations like this: ```console -> Give me a brief explanation of the attention mechnanism in transformer models. +> Give me a brief explanation of the attention mechanism in transformer models. In transformer models, the attention mechanism allows the model to focus on specific parts of the input sequence when computing the output. Here's a simplified explanation: 1. **Key-Query-Value (K-Q-V) computation**: For each input element, the model computes three vectors: @@ -48,9 +50,9 @@ In transformer models, the attention mechanism allows the model to focus on spec The attention mechanism allows transformer models to selectively focus on specific parts of the input sequence, enabling them to better understand context and relationships between input elements. This is particularly useful for tasks like machine translation, where the model needs to capture long-range dependencies between input words. ``` -To exit the interactive session, type `Ctrl+C` or `/bye`. +To exit the session, type `Ctrl+C` or `/bye`. -This will display performance statistics: +You'll then see performance metrics like this: ```bash llama_perf_sampler_print: sampling time = 26.66 ms / 356 runs ( 0.07 ms per token, 13352.84 tokens per second) @@ -60,28 +62,28 @@ llama_perf_context_print: eval time = 13173.66 ms / 331 runs ( 39 llama_perf_context_print: total time = 129945.08 ms / 355 tokens ``` -In this example, our 8-bit model running on 16 threads generated 355 tokens, at over 25 tokens per second (`eval time`). +In this example, the 8-bit model running on 16 threads generated 355 tokens, at ~25 tokens per second (`eval time`). -### Example Non-Interactive Session +## Run a non-interactive prompt -Now, try the 4-bit model in non-interactive mode: +You can also use `llama-cli` in one-shot mode with a prompt: ```bash -bin/llama-cli -m models/afm-4-5b/afm-4-5B-Q4_0.gguf -n 256 --color -no-cnv -p "Give me a brief explanation of the attention mechnanism in transformer models." +bin/llama-cli -m models/afm-4-5b/afm-4-5B-Q4_0.gguf -n 256 --color -no-cnv -p "Give me a brief explanation of the attention mechanism in transformer models." ``` -This command starts an non-interactive session with the model: -- `-m models/afm-4-5b/afm-4-5B-Q4_0.gguf` specifies the model file to load -- `-no-cnv` disable the conversation mode -- `-p` sets the prompt sent to the model -- The tool will prompt you to enter text, and the model will generate a response +This command: +- Loads the 4-bit model +- Disables conversation mode using `-no-cnv` +- Sends a one-time prompt using `-p` +- Prints the generated response and exits -Here, you should see the model generating at about 40 tokens per second. This shows how a more aggressive quantization recipe helps deliver faster performance. +The 4-bit model delivers faster generation—expect around 40 tokens per second on Graviton4. This shows how a more aggressive quantization recipe helps deliver faster performance. -## Using llama-server for API Access +## Use llama-server for API access -The `llama-server` tool runs the model as a web server, allowing you to make HTTP requests for text generation. This is useful for integrating the model into applications or for batch processing. +The `llama-server` tool runs the model as a web server compatible with the OpenAI API format, allowing you to make HTTP requests for text generation. This is useful for integrating the model into applications or for batch processing. -### Starting the Server +## Start the server ```bash bin/llama-server -m models/afm-4-5b/afm-4-5B-Q4_0.gguf \ @@ -90,17 +92,17 @@ bin/llama-server -m models/afm-4-5b/afm-4-5B-Q4_0.gguf \ --ctx-size 4096 ``` -This starts a server that: +This starts a local server that: - Loads the specified model - Listens on all network interfaces (`0.0.0.0`) - Accepts connections on port 8080 -- Uses a 4096-token context window +- Supports a 4096-token context window -### Making API Requests +### Make an API request -Once the server is running, you can make requests using curl or any HTTP client. As `llama-server` is compatible with the popular OpenAI API, we'll use in the following examples. +Once the server is running, you can make requests using curl, or any HTTP client. -Open a new terminal on the AWS instance and run: +Open a new terminal on the AWS instance, and run: ```bash curl -X POST http://localhost:8080/v1/chat/completions \ @@ -118,7 +120,7 @@ curl -X POST http://localhost:8080/v1/chat/completions \ }' ``` -You get an answer similar to this one: +The response includes the model’s reply and performance metrics: ```json { @@ -155,4 +157,12 @@ You get an answer similar to this one: } ``` +## What's next? + +You’ve now successfully: + +- Run AFM-4.5B in interactive and non-interactive modes +- Tested performance with different quantized models +- Served the model as an OpenAI-compatible API endpoint + You can also interact with the server using Python with the [OpenAI client library](https://github.com/openai/openai-python), enabling streaming responses, and other features. diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/07_evaluating_the_quantized_models.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/07_evaluating_the_quantized_models.md index bf390d985e..f23b06cc3a 100644 --- a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/07_evaluating_the_quantized_models.md +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/07_evaluating_the_quantized_models.md @@ -1,21 +1,21 @@ --- -title: Evaluating the quantized models -weight: 8 +title: Benchmark and evaluate the quantized models +weight: 9 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Using llama-bench for Performance Benchmarking +## Benchmark performance using llama-bench -The [`llama-bench`](https://github.com/ggml-org/llama.cpp/tree/master/tools/llama-bench) tool allows you to measure the performance characteristics of your model, including inference speed and memory usage. +Use the [`llama-bench`](https://github.com/ggml-org/llama.cpp/tree/master/tools/llama-bench) tool to measure model performance, including inference speed and memory usage. -### Basic Benchmarking +## Run basic benchmarks -You can benchmark multiple model versions to compare their performance: +Benchmark multiple model versions to compare performance: ```bash -# Benchmark the full precision model +# Benchmark the full-precision model bin/llama-bench -m models/afm-4-5b/afm-4-5B-F16.gguf # Benchmark the 8-bit quantized model @@ -25,14 +25,16 @@ bin/llama-bench -m models/afm-4-5b/afm-4-5B-Q8_0.gguf bin/llama-bench -m models/afm-4-5b/afm-4-5B-Q4_0.gguf ``` -Running each model on 16 vCPUs, you should see results like: +Typical results on a 16 vCPU instance: - **F16 model**: ~15-16 tokens/second, ~15GB memory usage - **Q8_0 model**: ~25 tokens/second, ~8GB memory usage - **Q4_0 model**: ~40 tokens/second, ~4.4GB memory usage -The exact performance will depend on your specific instance configuration and load. +Your actual results might vary depending on your specific instance configuration and system load. -### Advanced Benchmarking +## Run advanced benchmarks + +Use this command to benchmark performance across prompt sizes and thread counts: ```bash bin/llama-bench -m models/afm-4-5b/afm-4-5B-Q4_0.gguf \ @@ -41,13 +43,13 @@ bin/llama-bench -m models/afm-4-5b/afm-4-5B-Q4_0.gguf \ -t 8,16,24 ``` -This command: -- Loads the model and runs inference benchmarks -- `-p`: Evaluates a random prompt of 128, and 512 tokens -- `-n`: Generates 128 tokens -- `-t`: Run the model on 4, 8, and 16 threads +This command does the following: +- Loads the 4-bit model and runs inference benchmarks +- `-p`: evaluates prompt lengths of 128, 256, and 512 tokens +- `-n`: generates 128 tokens +- `-t`: runs inference using 4, 8, and 24 threads -The results should look like this: +Here’s an example of how performance scales across threads and prompt sizes: | model | size | params | backend | threads | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: | @@ -61,28 +63,32 @@ The results should look like this: | llama 8B Q4_0 | 4.33 GiB | 8.03 B | CPU | 16 | pp512 | 190.18 ± 0.03 | | llama 8B Q4_0 | 4.33 GiB | 8.03 B | CPU | 16 | tg128 | 40.99 ± 0.36 | -It's pretty amazing to see that with only 4 threads, the 4-bit model can still generate at the very comfortable speed of 15 tokens per second. We could definitely run several copies of the model on the same instance to serve concurrent users or applications. +Even with just four threads, the Q4_0 model achieves comfortable generation speeds. On larger instances, you can run multiple concurrent model processes to support parallel workloads. + +To benchmark batch inference, use [`llama-batched-bench`](https://github.com/ggml-org/llama.cpp/tree/master/tools/batched-bench). -You can also try [`llama-batched-bench`](https://github.com/ggml-org/llama.cpp/tree/master/tools/batched-bench) to benchmark performance on batch sizes larger than 1. +## Evaluate model quality using llama-perplexity -## Using llama-perplexity for Model Evaluation +Use the llama-perplexity tool to measure how well each model predicts the next token in a sequence. Perplexity is a measure of how well a language model predicts text. It gives you insight into the model’s confidence and predictive ability, representing the average number of possible next tokens the model considers when predicting each word: -Perplexity is a measure of how well a language model predicts text. It represents the average number of possible next tokens the model considers when predicting each word. A lower perplexity score indicates the model is more confident in its predictions and generally performs better on the given text. For example, a perplexity of 2.0 means the model typically considers 2 possible tokens when making each prediction, while a perplexity of 10.0 means it considers 10 possible tokens on average. +- A lower perplexity score indicates the model is more confident in its predictions and generally performs better on the given text. +- For example, a perplexity of 2.0 means the model typically considers ~2 tokens per step when making each prediction, while a perplexity of 10.0 means it considers 10 possible tokens on average, indicating more uncertainty. The `llama-perplexity` tool evaluates the model's quality on text datasets by calculating perplexity scores. Lower perplexity indicates better quality. -### Downloading a Test Dataset +## Download a test dataset -First, download the Wikitest-2 test dataset. +Use the following script to download and extract the Wikitext-2 dataset: ```bash sh scripts/get-wikitext-2.sh ``` +This script downloads and extracts the dataset to a local folder named `wikitext-2-raw`. -### Running Perplexity Evaluation +## Run a perplexity evaluation -Next, measure perplexity on the test dataset. +Run the llama-perplexity tool to evaluate how well each model predicts the Wikitext-2 test set: ```bash bin/llama-perplexity -m models/afm-4-5b/afm-4-5B-F16.gguf -f wikitext-2-raw/wiki.test.raw @@ -90,9 +96,15 @@ bin/llama-perplexity -m models/afm-4-5b/afm-4-5B-Q8_0.gguf -f wikitext-2-raw/wik bin/llama-perplexity -m models/afm-4-5b/afm-4-5B-Q4_0.gguf -f wikitext-2-raw/wiki.test.raw ``` -If you want to speed things up, you can add the `--chunks` option to use a fraction of 564 chunks contained in the test dataset. +{{< notice Tip >}} +To reduce runtime, add the `--chunks` flag to evaluate a subset of the data. For example: `--chunks 50` runs the evaluation on the first 50 text blocks. +{{< /notice >}} + +## Run the evaluation as a background script + +Running a full perplexity evaluation on all three models takes about 5 hours. To avoid SSH timeouts and keep the process running after logout, wrap the commands in a shell script and run it in the background. -On the full dataset, these three commands will take about 5 hours. You should run them in a shell script to avoid SSH timeouts. +Create a script named ppl.sh: For example: ```bash @@ -109,7 +121,7 @@ bin/llama-perplexity -m models/afm-4-5b/afm-4-5B-Q4_0.gguf -f wikitext-2-raw/wik Here are the full results. -| Model | Generation Speed (tokens/s, 16 vCPUs) | Memory Usage | Perplexity (Wikitext-2) | +| Model | Generation speed (tokens/s, 16 vCPUs) | Memory Usage | Perplexity (Wikitext-2) | |:-------:|:----------------------:|:------------:|:----------:| | F16 | ~15–16 | ~15 GB | TODO | | Q8_0 | ~25 | ~8 GB | TODO | diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/08_conclusion.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/08_conclusion.md index a7effd0311..2faa4f371d 100644 --- a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/08_conclusion.md +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/08_conclusion.md @@ -1,66 +1,66 @@ --- -title: Conclusion -weight: 9 +title: Review what you built +weight: 10 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Conclusion +## Wrap up your AFM-4.5B deployment -Congratulations! You have successfully completed the journey of deploying the Arcee AFM-4.5B foundation model on AWS Graviton4. +Congratulations! You have completed the process of deploying the Arcee AFM-4.5B foundation model on AWS Graviton4. -Here is a summary of what you learned. - -### What you built +Here’s a summary of what you built and how you can take your knowledge forward. Using this Learning Path, you have: -1. **Launched a Graviton4-powered EC2 instance** - Set up a c8g.4xlarge instance running Ubuntu 24.04 LTS, leveraging AWS's latest Arm-based processors for optimal performance and cost efficiency. +- **Launched a Graviton4-powered EC2 instance** – you set up a `c8g.4xlarge` instance running Ubuntu 24.04 LTS, leveraging Arm-based compute for optimal price–performance. -2. **Configured the development environment** - Installed essential tools and dependencies, including Git, build tools, and Python packages needed for machine learning workloads. +- **Configured the development environment** – you installed tools and dependencies, including Git, build tools, and Python packages for machine learning workloads. -3. **Built Llama.cpp from source** - Compiled the optimized inference engine specifically for Arm64 architecture, ensuring maximum performance on Graviton4 processors. +- **Built Llama.cpp from source** – you compiled the inference engine specifically for the Arm64 architecture to maximize performance on Graviton4. -4. **Downloaded and optimized AFM-4.5B** - Retrieved the 4.5-billion parameter Arcee Foundation Model and converted it to the efficient GGUF format, then created quantized versions (8-bit and 4-bit) to balance performance and memory usage. +- **Downloaded and optimized AFM-4.5B** – you retrieved the 4.5-billion-parameter Arcee Foundation Model, converted it to the GGUF format, and created quantized versions (8-bit and 4-bit) to reduce memory usage and improve speed. -5. **Ran inference and evaluation** - Tested the model's capabilities through interactive conversations, API endpoints, and comprehensive benchmarking to measure speed, memory usage, and model quality. +- **Ran inference and evaluation** – you tested the model using interactive sessions and API endpoints, and benchmarked speed, memory usage, and model quality. -### Key Performance Insights +## Key performance insights The benchmarking results demonstrate the power of quantization and Arm-based computing: -- **Memory efficiency**: The 4-bit quantized model uses only ~4.4GB of RAM compared to ~15GB for the full precision model -- **Speed improvements**: Quantization delivers 2-3x faster inference speeds (40+ tokens/second vs 15-16 tokens/second) -- **Cost optimization**: Lower memory requirements enable running on smaller, more cost-effective instances -- **Quality preservation**: The quantized models maintain excellent perplexity scores, showing minimal quality degradation +- **Memory efficiency** – the 4-bit model uses only ~4.4 GB of RAM compared to ~15 GB for the full-precision version +- **Speed improvements** – inference with Q4_0 is 2–3x faster (40+ tokens/sec vs. 15–16 tokens/sec) +- **Cost optimization** – lower memory needs enable smaller, more affordable instances +- **Quality preservation** – the quantized models maintain strong perplexity scores, showing minimal quality loss + +## The AWS Graviton4 advantage -### The Graviton4 Advantage +AWS Graviton4 processors, built on the Arm Neoverse V2 architecture, provide: -AWS Graviton4 processors, built on Arm Neoverse-V2 architecture, provide: - Superior performance per watt compared to x86 alternatives -- Cost savings of 20-40% for compute-intensive workloads +- Cost savings of 20–40% for compute-intensive workloads - Optimized memory bandwidth and cache hierarchy for AI/ML workloads - Native Arm64 support for modern machine learning frameworks -### Next Steps and Call to Action +## Next steps for deploying AFM-4.5B on Arm -Now that you have a fully functional AFM-4.5B deployment, here are some exciting ways to extend your learning: +Now that you have a fully functional AFM-4.5B deployment, here are some ways to extend your learning: -**Production Deployment** +**Production deployment**: - Set up auto-scaling groups for high availability - Implement load balancing for multiple model instances - Add monitoring and logging with CloudWatch - Secure your API endpoints with proper authentication -**Application Development** -- Build a web application using the llama-server API +**Application development**: +- Build a web application using the `llama-server` API - Create a chatbot or virtual assistant - Develop content generation tools - Integrate with existing applications via REST APIs -The combination of Arcee AI's efficient foundation models, Llama.cpp's optimized inference engine, and AWS Graviton4's powerful Arm processors creates a compelling platform for deploying production-ready AI applications. Whether you're building chatbots, content generators, or research tools, this stack provides the performance, cost efficiency, and flexibility needed for modern AI workloads. +Together, Arcee AI’s foundation models, Llama.cpp’s efficient runtime, and Graviton4’s compute capabilities give you everything you need to build scalable, production-grade AI applications. -For more information on Arcee AI and how we can help you build high-quality, secure, and cost-efficient AI, solution, please visit [www.arcee.ai](https://www.arcee.ai). +From chatbots and content generation to research tools, this stack strikes a balance between performance, cost, and developer control. +For more information on Arcee AI, and how you can build high-quality, secure, and cost-efficient AI solutions, please visit www.arcee.ai. diff --git a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/_index.md b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/_index.md index 4623d917d2..7a7b23f235 100644 --- a/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/_index.md @@ -1,5 +1,5 @@ --- -title: Deploy Arcee AFM-4.5B on AWS Graviton4 +title: Deploy Arcee AFM-4.5B on Arm-based AWS Graviton4 with Llama.cpp draft: true cascade: @@ -7,23 +7,22 @@ cascade: minutes_to_complete: 30 -who_is_this_for: This is an introductory topic for developers and engineers who want to deploy the Arcee AFM-4.5B small language model on an AWS Arm-based instance. AFM-4.5B is a 4.5-billion-parameter frontier model that delivers excellent accuracy, strict compliance, and very high cost-efficiency. It was trained on almost 7 trillion tokens of clean, rigorously filtered data, and has been tested across a wide range of languages, including Arabic, English, French, German, Hindi, Italian, Korean, Mandarin, Portuguese, Russian, and Spanish. +who_is_this_for: This Learning Path is for developers and ML engineers who want to deploy Arcee's AFM-4.5B small language model on AWS Graviton4 instances using Llama.cpp. learning_objectives: - - Launch and set up an Arm-based Graviton4 virtual machine on Amazon Web Services. - - Build Llama.cpp from source. - - Download AFM-4.5B from Hugging Face. - - Quantize AFM-4.5B with Llama.cpp. - - Deploy the model and run inference with Llama.cpp. - - Evaluate the quality of quantized models by measuring perplexity. + - Launch an Arm-based EC2 instance on AWS Graviton4 + - Build and install Llama.cpp from source + - Download and quantize the AFM-4.5B model from Hugging Face + - Run inference on the quantized model using Llama.cpp + - Evaluate model quality by measuring perplexity prerequisites: - - An [AWS account](https://aws.amazon.com/) with permission to launch c8g (Graviton4) instances. - - Basic familiarity with SSH. + - An [AWS account](https://aws.amazon.com/) with permission to launch Graviton4 (`c8g.4xlarge` or larger) instances + - Basic familiarity with Linux and SSH author: Julien Simon -### Tags +# Tags # Tagging metadata, see the Learning Path guide for the allowed values skilllevels: Introductory subjects: ML @@ -31,7 +30,7 @@ arm_ips: - Neoverse tools_software_languages: - Amazon Web Services - - Linux + - Hugging Face - Python - Llama.cpp operatingsystems: @@ -39,28 +38,28 @@ operatingsystems: further_reading: - - resource: - title: Arcee AI - link: https://www.arcee.ai - type: Website - - resource: - title: Announcing Arcee Foundation Models - link: https://www.arcee.ai/blog/announcing-the-arcee-foundation-model-family - type: Blog - - resource: - title: AFM-4.5B, the First Arcee Foundation Model - link: https://www.arcee.ai/blog/deep-dive-afm-4-5b-the-first-arcee-foundational-model - type: Blog - - resource: - title: Amazon EC2 Graviton Instances - link: https://aws.amazon.com/ec2/graviton/ - type: Documentation - - resource: - title: Amazon EC2 Documentation - link: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ - type: Documentation + - resource: + title: Arcee AI + link: https://www.arcee.ai + type: website + - resource: + title: Announcing the Arcee Foundation Model family + link: https://www.arcee.ai/blog/announcing-the-arcee-foundation-model-family + type: blog + - resource: + title: Deep Dive - AFM-4.5B, the first Arcee Foundation Model + link: https://www.arcee.ai/blog/deep-dive-afm-4-5b-the-first-arcee-foundational-model + type: blog + - resource: + title: Amazon EC2 Graviton instances + link: https://aws.amazon.com/ec2/graviton/ + type: documentation + - resource: + title: Amazon EC2 User Guide + link: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ + type: documentation -### FIXED, DO NOT MODIFY +# FIXED, DO NOT MODIFY # ================================================================================ weight: 1 # _index.md always has weight of 1 to order correctly layout: "learningpathall" # All files under learning paths have this same wrapper diff --git a/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-token.md b/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-token.md index 49c7573f1c..342c49d5a0 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-token.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-token.md @@ -13,10 +13,10 @@ In this section, you will obtain an example CCA attestation token. To set up the First, start by removing any existing Go installation. Next, obtain the install files and run the following command: ```bash -rm -rf /usr/local/go +sudo rm -rf /usr/local/go wget https://go.dev/dl/go1.23.3.linux-$(dpkg --print-architecture).tar.gz -tar -C /usr/local -xzf go1.23.3.linux-$(dpkg --print-architecture).tar.gz +sudo tar -C /usr/local -xzf go1.23.3.linux-$(dpkg --print-architecture).tar.gz ``` Export the installation path and add it to your `$PATH environment` variable: diff --git a/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-verification.md b/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-verification.md index ff56474c4e..98e2ef27c5 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-verification.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-verification.md @@ -16,7 +16,7 @@ The URL for reaching this experimental verifier service is https://veraison.test To check that you can reach the Linaro attestation verifier service, run the following command: ```bash -curl https://veraison.test.linaro.org:8443/.well-known/veraison/verification +curl https://veraison.test.linaro.org:8443/.well-known/veraison/verification | jq ``` This is a simple call to query the well-known characteristics of the verification service. If it succeeds, it will return a JSON response that looks something like this: @@ -63,7 +63,7 @@ If you can reach the verification service, you are now ready to use it to evalua One of the properties that was returned in the previous step was the public key of the verification service. This key will be needed later to check the signature on the attestation results. All that is needed in this step is to copy the contents of the `ear-verification-key` field from the previous step and save it to a separate JSON file. -The easiest way to do this is to use the `jq` utility. +The easiest way to do this is to use the `jq` utility. You can save the public key by repeating the curl command from the previous step and use `jq` to filter the response down to just the public key part. Save it into a file called `pkey.json`: ```bash diff --git a/content/learning-paths/servers-and-cloud-computing/cca-veraison/evaluate-result.md b/content/learning-paths/servers-and-cloud-computing/cca-veraison/evaluate-result.md index f86747886c..44e92ee955 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-veraison/evaluate-result.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-veraison/evaluate-result.md @@ -34,7 +34,7 @@ The attestation result is a JWT file, which stands for JSON Web Token. This mean The following command will use the `arc` tool, which you built in the previous step, to verify the cryptographic signature on the attestation result, and display the result in a human-readable format: ```bash -./arc verify --pkey $HOME/pkey.json --color $HOME/attestation_result.jwt +./arc verify --pkey $HOME/pkey.json $HOME/attestation_result.jwt ``` This command produces quite a lot of output. However, you will notice that a large amount of the output simply reflects the contents of the CCA attestation token that you inspected earlier with the evcli tool. The most interesting part of the output is towards the bottom, and should look like this: diff --git a/content/migration/_index.md b/content/migration/_index.md index da2e6ca810..85da72ff9b 100644 --- a/content/migration/_index.md +++ b/content/migration/_index.md @@ -65,7 +65,13 @@ Oracle Cloud Infrastructure (OCI) Ampere [Arm-based compute](https://www.oracle. | A2 | AmpereOne | VM.Standard.A2, VM.Optimized3.A2 | Tailored for high-performance and memory-intensive workloads. | {{< /tab >}} + {{< tab header="Alibaba Cloud">}} +Alibaba Cloud offers [3 instance types](https://www.alibabacloud.com/help/en/ecs/user-guide/general-purpose-instance-families?spm=a2c63.p38356.help-menu-25365.d_4_1_2_2.6f87303fTup4Vv#g8y) with Yitian710 processors, c8y/g8y/r8y. The largest instance has 128 vCPUs and 1 TB of memory. A wide variety of instance sizes are available, including bare-metal. +| Generation | Arm CPU | Instance types | Comments | +| --------------|--------------|--------------------|-----------| +| Yitian710 | Neoverse-N2 | c8y, g8y, r8y | Arm based instance| + {{< /tab >}} {{< /tabpane-normal >}} Read [Get started with Servers and Cloud Computing](https://learn.arm.com/learning-paths/servers-and-cloud-computing/intro) to learn more and find additional cloud service providers. @@ -98,6 +104,7 @@ The below resources are curated to address migration challenges on specific clou - [Porting architecture specific intrinsics](https://learn.arm.com/learning-paths/cross-platform/intrinsics/) - perfect for porting intrinsics from another architecture. - [Arm software install guides](https://learn.arm.com/install-guides) - good for quickly installing common tools and software. - [simd.info](https://simd.info/) - a searchable reference tool for C intrinsics for SIMD engines. +- [migrate-ease](https://github.com/migrate-ease/migrate-ease) - scan your source code for suggestions to migrate to Arm Neoverse, get started [here](https://learn.arm.com/learning-paths/servers-and-cloud-computing/migrate-ease/). - [Arm Infrastructure Solutions blog](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/) - an Arm-specific technical blog. - [Arm Learning Paths for Servers and Cloud](https://learn.arm.com/learning-paths/servers-and-cloud-computing/) - general tutorials for Arm servers. You can search for specific cloud service providers, including [AWS](https://learn.arm.com/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=aws/#), [Google Cloud](https://learn.arm.com/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=google-cloud/#), [Microsoft Azure](https://learn.arm.com/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=microsoft-azure/#), and [Oracle](https://learn.arm.com/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=oracle/#). diff --git a/data/stats_current_test_info.yml b/data/stats_current_test_info.yml index 98c60af541..48c5619c87 100644 --- a/data/stats_current_test_info.yml +++ b/data/stats_current_test_info.yml @@ -1,5 +1,5 @@ summary: - content_total: 382 + content_total: 387 content_with_all_tests_passing: 0 content_with_tests_enabled: 61 sw_categories: @@ -54,8 +54,7 @@ sw_categories: tests_and_status: [] armpl: readable_title: Arm Performance Libraries - tests_and_status: - - ubuntu:latest: passed + tests_and_status: [] aws-cli: readable_title: AWS CLI tests_and_status: [] diff --git a/data/stats_weekly_data.yml b/data/stats_weekly_data.yml index 682ee0df3a..ad533ae6fa 100644 --- a/data/stats_weekly_data.yml +++ b/data/stats_weekly_data.yml @@ -6668,3 +6668,230 @@ avg_close_time_hrs: 0 num_issues: 19 percent_closed_vs_total: 0.0 +- a_date: '2025-07-14' + content: + automotive: 2 + cross-platform: 33 + embedded-and-microcontrollers: 42 + install-guides: 103 + iot: 6 + laptops-and-desktops: 38 + mobile-graphics-and-gaming: 34 + servers-and-cloud-computing: 128 + total: 386 + contributions: + external: 97 + internal: 511 + github_engagement: + num_forks: 30 + num_prs: 7 + individual_authors: + adnan-alsinan: 2 + alaaeddine-chakroun: 2 + albin-bernhardsson: 1 + alex-su: 1 + alexandros-lamprineas: 1 + andrew-choi: 2 + andrew-kilroy: 1 + annie-tallund: 4 + arm: 3 + arnaud-de-grandmaison: 5 + aude-vuilliomenet: 1 + avin-zarlez: 1 + barbara-corriero: 1 + basma-el-gaabouri: 1 + ben-clark: 1 + bolt-liu: 2 + brenda-strech: 1 + chaodong-gong: 1 + chen-zhang: 1 + christophe-favergeon: 1 + christopher-seidl: 7 + cyril-rohr: 1 + daniel-gubay: 1 + daniel-nguyen: 2 + david-spickett: 2 + dawid-borycki: 33 + diego-russo: 2 + dominica-abena-o.-amanfo: 1 + elham-harirpoush: 2 + florent-lebeau: 5 + "fr\xE9d\xE9ric--lefred--descamps": 2 + gabriel-peterson: 5 + gayathri-narayana-yegna-narayanan: 2 + georgios-mermigkis: 1 + geremy-cohen: 3 + gian-marco-iodice: 1 + graham-woodward: 1 + han-yin: 1 + iago-calvo-lista: 1 + james-whitaker: 1 + jason-andrews: 103 + jeff-young: 1 + joana-cruz: 1 + joe-stech: 6 + johanna-skinnider: 2 + jonathan-davies: 2 + jose-emilio-munoz-lopez: 1 + julie-gaskin: 5 + julio-suarez: 6 + jun-he: 1 + kasper-mecklenburg: 1 + kieran-hejmadi: 12 + koki-mitsunami: 2 + konstantinos-margaritis: 8 + kristof-beyls: 1 + leandro-nunes: 1 + liliya-wu: 1 + mark-thurman: 1 + masoud-koleini: 1 + mathias-brossard: 1 + michael-hall: 5 + na-li: 1 + nader-zouaoui: 2 + nikhil-gupta: 1 + nina-drozd: 1 + nobel-chowdary-mandepudi: 6 + odin-shen: 7 + owen-wu: 2 + pareena-verma: 46 + paul-howard: 3 + peter-harris: 1 + pranay-bakre: 5 + preema-merlin-dsouza: 1 + przemyslaw-wirkus: 2 + qixiang-xu: 1 + rani-chowdary-mandepudi: 1 + rin-dobrescu: 1 + roberto-lopez-mendez: 2 + ronan-synnott: 45 + shuheng-deng: 1 + thirdai: 1 + tianyu-li: 2 + tom-pilar: 1 + uma-ramalingam: 1 + varun-chari: 2 + visualsilicon: 1 + willen-yang: 1 + ying-yu: 2 + yiyang-fan: 1 + zach-lasiuk: 2 + zhengjun-xing: 2 + issues: + avg_close_time_hrs: 0 + num_issues: 16 + percent_closed_vs_total: 0.0 +- a_date: '2025-07-21' + content: + automotive: 2 + cross-platform: 33 + embedded-and-microcontrollers: 43 + install-guides: 103 + iot: 6 + laptops-and-desktops: 38 + mobile-graphics-and-gaming: 34 + servers-and-cloud-computing: 128 + total: 387 + contributions: + external: 97 + internal: 511 + github_engagement: + num_forks: 30 + num_prs: 12 + individual_authors: + adnan-alsinan: 2 + alaaeddine-chakroun: 2 + albin-bernhardsson: 1 + alex-su: 1 + alexandros-lamprineas: 1 + andrew-choi: 2 + andrew-kilroy: 1 + annie-tallund: 4 + arm: 3 + arnaud-de-grandmaison: 5 + aude-vuilliomenet: 1 + avin-zarlez: 1 + barbara-corriero: 1 + basma-el-gaabouri: 1 + ben-clark: 1 + bolt-liu: 2 + brenda-strech: 1 + bright-edudzi-gershon-kordorwu: 1 + chaodong-gong: 1 + chen-zhang: 1 + christophe-favergeon: 1 + christopher-seidl: 7 + cyril-rohr: 1 + daniel-gubay: 1 + daniel-nguyen: 2 + david-spickett: 2 + dawid-borycki: 33 + diego-russo: 2 + dominica-abena-o.-amanfo: 1 + elham-harirpoush: 2 + florent-lebeau: 5 + "fr\xE9d\xE9ric--lefred--descamps": 2 + gabriel-peterson: 5 + gayathri-narayana-yegna-narayanan: 2 + georgios-mermigkis: 1 + geremy-cohen: 3 + gian-marco-iodice: 1 + graham-woodward: 1 + han-yin: 1 + iago-calvo-lista: 1 + james-whitaker: 1 + jason-andrews: 103 + jeff-young: 1 + joana-cruz: 1 + joe-stech: 6 + johanna-skinnider: 2 + jonathan-davies: 2 + jose-emilio-munoz-lopez: 1 + julie-gaskin: 5 + julio-suarez: 6 + jun-he: 1 + kasper-mecklenburg: 1 + kieran-hejmadi: 12 + koki-mitsunami: 2 + konstantinos-margaritis: 8 + kristof-beyls: 1 + leandro-nunes: 1 + liliya-wu: 1 + mark-thurman: 1 + masoud-koleini: 1 + mathias-brossard: 1 + michael-hall: 5 + na-li: 1 + nader-zouaoui: 2 + nikhil-gupta: 1 + nina-drozd: 1 + nobel-chowdary-mandepudi: 6 + odin-shen: 7 + owen-wu: 2 + pareena-verma: 46 + paul-howard: 3 + peter-harris: 1 + pranay-bakre: 5 + preema-merlin-dsouza: 1 + przemyslaw-wirkus: 2 + qixiang-xu: 1 + rani-chowdary-mandepudi: 1 + rin-dobrescu: 1 + roberto-lopez-mendez: 2 + ronan-synnott: 45 + shuheng-deng: 1 + thirdai: 1 + tianyu-li: 2 + tom-pilar: 1 + uma-ramalingam: 1 + varun-chari: 2 + visualsilicon: 1 + willen-yang: 1 + ying-yu: 2 + yiyang-fan: 1 + zach-lasiuk: 2 + zhengjun-xing: 2 + issues: + avg_close_time_hrs: 0 + num_issues: 21 + percent_closed_vs_total: 0.0 diff --git a/themes/arm-design-system-hugo-theme/layouts/partials/learning-paths/introduction.html b/themes/arm-design-system-hugo-theme/layouts/partials/learning-paths/introduction.html index 6b9393fee3..ca1d7fa7ae 100644 --- a/themes/arm-design-system-hugo-theme/layouts/partials/learning-paths/introduction.html +++ b/themes/arm-design-system-hugo-theme/layouts/partials/learning-paths/introduction.html @@ -21,7 +21,7 @@

Who is this for?

{{ .Params.who_is_this_for }}

What will you learn?

-

Upon completion of this learning path, you will be able to:

+

Upon completion of this Learning Path, you will be able to:

    {{ range .Params.learning_objectives }}
  • {{.}}