diff --git a/.github/workflows/test-lp.yml b/.github/workflows/test-lp.yml index 47b3f60270..fd187b6ef5 100644 --- a/.github/workflows/test-lp.yml +++ b/.github/workflows/test-lp.yml @@ -1,5 +1,8 @@ name: Test Learning Path on: pull_request +env: + HUGO_VERSION: 0.130.0 + jobs: Test-Pull-Request: runs-on: ubuntu-24.04-arm @@ -8,9 +11,13 @@ jobs: uses: actions/checkout@v4 with: ref: ${{ steps.vars.outputs.branch-name }} + - name: Setup Hugo + uses: peaceiris/actions-hugo@v3 + with: + hugo-version: ${{ env.HUGO_VERSION }} + extended: true - name: Run hugo command to test site builds run: | - sudo apt-get install -y hugo hugo - name: Get all changed markdown files id: changed-markdown-files diff --git a/.wordlist.txt b/.wordlist.txt index eb990b7b46..796f136e22 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -4459,4 +4459,19 @@ Arcee's commandlinetools ppl rollout -sdkmanager \ No newline at end of file +sdkmanager +Ntegral +OEMs +TKN +VHD +inet +tekton +tektoncd +tkn +verifiably +vhd +AssetLib +PerformanceStudio +VkThread +precompiled +rollouts \ No newline at end of file diff --git a/content/install-guides/azure-cli.md b/content/install-guides/azure-cli.md index d7dc3b14e0..d1625082c8 100644 --- a/content/install-guides/azure-cli.md +++ b/content/install-guides/azure-cli.md @@ -56,7 +56,7 @@ Execute the following command to verify the Azure CLI is installed correctly az version ``` -You should see an output similar to below +You should see an output similar to: ```output { @@ -94,7 +94,7 @@ The pip install updates $HOME/.profile with the path the `az` executable. Check source $HOME/.profile az version ``` -You should see an output similar to below +You should see an output similar to: ```output { diff --git a/content/install-guides/oc.md b/content/install-guides/oc.md new file mode 100644 index 0000000000..3088e105d6 --- /dev/null +++ b/content/install-guides/oc.md @@ -0,0 +1,232 @@ +--- +title: OpenShift CLI (oc) + +author: Jason Andrews + +official_docs: https://docs.redhat.com/en/documentation/openshift_container_platform/4.18/html/cli_tools/openshift-cli-oc#cli-getting-started +minutes_to_complete: 10 + +additional_search_terms: +- OpenShift +- Kubernetes + +layout: installtoolsall +multi_install: false +multitool_install_part: false +test_images: +- ubuntu:latest +test_maintenance: false +tool_install: true +weight: 1 +--- +The OpenShift command-line interface (CLI), `oc`, allows you to work with OpenShift Container Platform projects from a terminal. You can use `oc` to create applications, manage OpenShift Container Platform projects, and perform administrative tasks. + +The OpenShift CLI is a superset of the Kubernetes `kubectl` command. When you install `oc`, you get both the OpenShift-specific functionality and all standard Kubernetes `kubectl` commands in a single tool. This means you can use `oc` to manage both OpenShift and standard Kubernetes resources. + +The OpenShift CLI is available for macOS and Linux and supports the Arm architecture. + +## What should I consider before installing the OpenShift CLI? + +This article provides a quick solution to install the latest version of the OpenShift CLI for Ubuntu on Arm and macOS with Apple Silicon. + +Confirm that you are using an Arm-based computer by running: + +```bash { target="ubuntu:latest" } +uname -m +``` + +If you are on Arm Linux the output should be: + +```output +aarch64 +``` + +If you are on macOS with Apple Silicon the output should be: + +```output +arm64 +``` + +## How do I download and install the OpenShift CLI? + +There are multiple ways to install the OpenShift CLI. The methods below download the latest stable version directly from the OpenShift mirror. + +### Install on Arm Linux + +To install the OpenShift CLI on Arm Linux: + +```bash { target="ubuntu:latest" } +curl -LO https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable/openshift-client-linux-arm64.tar.gz +tar -xzf openshift-client-linux-arm64.tar.gz +sudo mv oc kubectl /usr/local/bin/ +rm openshift-client-linux-arm64.tar.gz README.md +``` + +### Install on macOS + +To install the OpenShift CLI on macOS with Apple Silicon: + +```console +curl -LO https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable/openshift-client-mac-arm64.tar.gz +tar -xzf openshift-client-mac-arm64.tar.gz +sudo mv oc kubectl /usr/local/bin/ +rm openshift-client-mac-arm64.tar.gz README.md +``` + +Both installations include both `oc` and `kubectl` commands. The `oc` command provides the full OpenShift functionality, while the `kubectl` command gives you compatibility with standard Kubernetes clusters. Since `oc` is a superset of `kubectl`, you can use `oc` for all Kubernetes operations, but having both commands available gives you flexibility in your workflow. + +{{% notice Tip %}} +You can use either `oc` or `kubectl` to manage standard Kubernetes resources. However, `oc` includes OpenShift-specific features and is preferred for working with OpenShift clusters. +{{% /notice %}} + + +## Understanding oc and kubectl + +The OpenShift CLI (`oc`) is built as a superset of the Kubernetes CLI (`kubectl`). + +This means: + +- You can use `oc` anywhere you would use `kubectl` +- OpenShift-specific features such as `oc login`, `oc new-project`, `oc new-app`, are available to manage OpenShift resources + +## How do I verify the OpenShift CLI installation? + +Verify the OpenShift CLI is installed by checking the version: + +```bash { target="ubuntu:latest" } +oc version --client +``` + +The output shows the client version information: + +```output +Client Version: 4.19.3 +Kustomize Version: v5.5.0 +``` + +You can also verify that `kubectl` is available and shows the same version (since it's the same binary): + +```bash { target="ubuntu:latest" } +kubectl version --client +``` + +The output shows the kubectl client version: + +```output +Client Version: v1.32.1 +Kustomize Version: v5.5.0 +``` + +Both commands are now available for managing Kubernetes and OpenShift resources. + +## How do I get started with the OpenShift CLI? + +To get help with available commands, run: + +```console +oc help +``` + +This displays the main command groups and options: + +```output +OpenShift Client + +This client helps you develop, build, deploy, and run your applications on any +OpenShift or Kubernetes cluster. It also includes the administrative +commands for managing a cluster under the 'adm' subcommand. + +Usage: + oc [flags] + +Basic Commands: + login Log in to a server + new-project Request a new project + new-app Create a new application + status Show an overview of the current project + project Switch to another project + projects Display existing projects + explain Get documentation for a resource + +Build and Deploy Commands: + rollout Manage the rollout of a resource + rollback Revert part of an application back to a previous deployment + new-build Create a new build configuration + start-build Start a new build + cancel-build Cancel running, pending, or new builds + import-image Import images from a Docker registry + tag Tag existing images into image streams + +Application Management Commands: + create Create a resource from a file or from stdin + apply Apply a configuration to a resource by file name or stdin + get Display one or many resources + describe Show details of a specific resource or group of resources + edit Edit a resource on the server + set Commands that help set specific features on objects + label Update the labels on a resource + annotate Update the annotations on a resource + expose Expose a replicated application as a service or route + delete Delete resources by file names, stdin, resources and names, or by resources and label selector + scale Set a new size for a deployment, replica set, or replication controller + autoscale Autoscale a deployment or replica set + secrets Manage secrets + serviceaccounts Manage service accounts in your project + +Troubleshooting and Debugging Commands: + logs Print the logs for a resource + rsh Start a shell session in a pod + rsync Copy files between a local file system and a pod + port-forward Forward one or more local ports to a pod + debug Launch a new instance of a pod for debugging + exec Execute a command in a container + proxy Run a proxy to the Kubernetes API server + attach Attach to a running container + run Run a particular image on the cluster + cp Copy files and directories to and from containers + wait Experimental: Wait for a specific condition on one or many resources + +Advanced Commands: + adm Tools for managing a cluster + create Create a resource from a file or from stdin + replace Replace a resource by file name or stdin + patch Update fields of a resource + process Process a template into list of resources + export Export resources so they can be used elsewhere + extract Extract secrets or config maps to disk + observe Observe changes to resources and react to them (experimental) + policy Manage authorization policy + auth Inspect authorization + image Useful commands for managing images + registry Commands for working with the registry + idle Idle scalable resources + api-versions Print the supported API versions on the server, in the form of "group/version" + api-resources Print the supported API resources on the server + cluster-info Display cluster information + diff Diff the live version against a would-be applied version + kustomize Build a kustomization target from a directory or URL + +Settings Commands: + logout End the current server session + config Modify kubeconfig files + whoami Return information about the current session + completion Output shell completion code for the specified shell (bash, zsh, fish, or powershell) + +Other Commands: + help Help about any command + plugin Provides utilities for interacting with plugins + version Print the client and server version information + +Use "oc --help" for more information about a given command. +Use "oc options" for a list of global command-line options (applies to all commands). +``` + +To connect to an OpenShift cluster, you need to log in using: + +```console +oc login +``` + +Replace `` with your OpenShift cluster's URL. You will be prompted for your username and password. + +You are now ready to use the OpenShift CLI to manage your OpenShift Container Platform projects and applications. diff --git a/content/install-guides/tkn.md b/content/install-guides/tkn.md new file mode 100644 index 0000000000..7329a6bd99 --- /dev/null +++ b/content/install-guides/tkn.md @@ -0,0 +1,197 @@ +--- +title: Tekton CLI (tkn) + +author: Jason Andrews +official_docs: https://tekton.dev/docs/cli/ + +minutes_to_complete: 10 +additional_search_terms: +- tekton +- pipelines +- ci/cd +- kubernetes +- openshift +- devops + +layout: installtoolsall +multi_install: false +multitool_install_part: false +test_images: +- ubuntu:latest +test_maintenance: false +tool_install: true +weight: 1 +--- + +The Tekton CLI, `tkn`, is a command-line interface for Tekton Pipelines. It allows you to create, manage, and interact with Tekton resources such as tasks, pipelines, and pipeline runs from your terminal. + +Tekton CLI is available for macOS and Linux and supports the Arm architecture. + +## What should I consider before installing the Tekton CLI? + +This article provides a quick way to install the latest version of the Tekton CLI for Ubuntu on Arm and macOS with Apple Silicon. + +Confirm you are using an Arm computer by running: + +```bash { target="ubuntu:latest" } +uname -m +``` + +If you are on Arm Linux the output should be: + +```output +aarch64 +``` + +If you are on macOS with Apple Silicon the output should be: + +```output +arm64 +``` + +## How do I download and install the Tekton CLI? + +You can install the Tekton CLI in multiple ways. The following methods download the latest stable release directly from GitHub. + +### Install on Arm Linux + +To install the Tekton CLI on Arm Linux: + +```bash { target="ubuntu:latest" } +TKN_VERSION=$(curl -s https://api.github.com/repos/tektoncd/cli/releases/latest | grep tag_name | cut -d '"' -f 4) +curl -LO https://github.com/tektoncd/cli/releases/download/${TKN_VERSION}/tektoncd-cli-${TKN_VERSION#v}_Linux-ARM64.deb +sudo apt install ./tektoncd-cli-${TKN_VERSION#v}_Linux-ARM64.deb +``` + +{{% notice Tip %}} +If the version query fails, you can manually set `TKN_VERSION` to a known stable version like `v0.41.0`. +{{% /notice %}} + +### Install on macOS + +To install the Tekton CLI on macOS with Apple Silicon: + +```console +TKN_VERSION=$(curl -s https://api.github.com/repos/tektoncd/cli/releases/latest | grep tag_name | cut -d '"' -f 4) +curl -LO https://github.com/tektoncd/cli/releases/download/${TKN_VERSION}/tkn_${TKN_VERSION#v}_Darwin_all.tar.gz +tar -xzf tkn_${TKN_VERSION#v}_Darwin_all.tar.gz +sudo mv tkn /usr/local/bin/ +rm tkn_${TKN_VERSION#v}_Darwin_all.tar.gz README.md LICENSE +``` + +Alternatively, you can install using Homebrew on macOS: + +```console +brew install tektoncd-cli +``` + +## How do I verify the Tekton CLI installation? + +Verify the Tekton CLI is installed by checking the version: + +```bash { target="ubuntu:latest" } +tkn version +``` + +The output shows the client version information: + +```output +Client version: 0.41.0 +``` + +You can also check that the command is working by displaying the help: + +```bash { target="ubuntu:latest" } +tkn help +``` + +This displays the main command groups and options: + +```output +CLI for tekton pipelines + +Usage: + tkn [flags] + tkn [command] + +Available Commands: + bundle Manage Tekton Bundles + chain Manage Chains + clustertask Manage ClusterTasks + clustertriggerbinding Manage ClusterTriggerBindings + completion Prints shell completion scripts + eventlistener Manage EventListeners + help Help about any command + hub Interact with tekton hub + pipeline Manage pipelines + pipelinerun Manage pipeline runs + resource Manage pipeline resources + task Manage Tasks + taskrun Manage TaskRuns + triggerbinding Manage TriggerBindings + triggertemplate Manage TriggerTemplates + version Prints version information + +Flags: + -h, --help help for tkn + +Use "tkn [command] --help" for more information about a command. +``` + +## How do I get started with the Tekton CLI? + +To use the Tekton CLI effectively, you need access to a Kubernetes cluster with Tekton Pipelines installed. You can check if Tekton is available in your cluster: + +```console +tkn pipeline list +``` + +If Tekton Pipelines is not installed, you might see an error message. In that case, you need to install Tekton Pipelines on your cluster first. + +### Common Tekton CLI commands + +Below are some common commands to get you started. + +List pipelines: + +```console +tkn pipeline list +``` + +List pipeline runs: + +```console +tkn pipelinerun list +``` + +List tasks: + +```console +tkn task list +``` + +List task runs: + +```console +tkn taskrun list +``` + +Start a pipeline: + +```console +tkn pipeline start +``` + +Show pipeline run logs: + +```console +tkn pipelinerun logs +``` + +Describe a pipeline: + +```console +tkn pipeline describe +``` + +You are now ready to use the Tekton CLI to manage your CI/CD pipelines with Tekton Pipelines on Kubernetes or OpenShift. diff --git a/content/learning-paths/automotive/_index.md b/content/learning-paths/automotive/_index.md index 8a002e10db..d58b01c2eb 100644 --- a/content/learning-paths/automotive/_index.md +++ b/content/learning-paths/automotive/_index.md @@ -20,9 +20,12 @@ operatingsystems_filter: tools_software_languages_filter: - Automotive: 1 - C: 1 +- DDS: 1 - Docker: 2 - Python: 2 - Raspberry Pi: 1 - ROS 2: 1 - ROS2: 2 +- Rust: 1 +- Zenoh: 1 --- diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/1_functional_safety.md b/content/learning-paths/automotive/openadkit2_safetyisolation/1_functional_safety.md deleted file mode 100644 index d08f796580..0000000000 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/1_functional_safety.md +++ /dev/null @@ -1,134 +0,0 @@ ---- -title: Functional Safety for automotive software development -weight: 2 - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- - -## Why Functional Safety Matters in Automotive Software - -Functional Safety refers to a system's ability to detect potential faults and respond appropriately to ensure that the system remains in a safe state, preventing harm to individuals or damage to equipment. - -This is particularly important in automotive, autonomous driving, medical devices, industrial control, robotics and aerospace applications, where system failures can lead to severe consequences. - -In software development, Functional Safety focuses on minimizing risks through software design, testing, and validation to ensure that critical systems operate in a predictable, reliable, and verifiable manner. This means developers must consider: -- Error detection mechanisms -- Exception handling -- Redundancy design -- Development processes compliant with safety standards - -### Definition and Importance of Functional Safety - -The core of Functional Safety lies in risk management, which aims to reduce the impact of system failures. - -In autonomous vehicles, Functional Safety ensures that if sensor data is incorrect, the system can enter a safe state, preventing incorrect driving decisions. - -The three core objectives of Functional Safety are: -1. Prevention: Reducing the likelihood of errors through rigorous software development processes and testing. In the electric vehicle, the battery systems monitor temperature to prevent overheating. -2. Detection: Quickly identifying errors using built-in diagnostic mechanisms, such as built-in self-test. -3. Mitigation: Controlling the impact of failures to ensure the overall safety of the system. - -This approach is critical in applications such as autonomous driving, flight control, and medical implants, where failures can result in severe consequences. - -### ISO 26262: Automotive Functional Safety Standard - -ISO 26262 is a functional safety standard specifically for automotive electronics and software systems. It defines a comprehensive V-model aligned safety lifecycle, covering all phases from requirement analysis, design, development, testing, to maintenance. - -Key Concepts of ISO 26262: -- ASIL (Automotive Safety Integrity Level) - - Evaluates the risk level of different system components (A, B, C, D, where D represents the highest safety requirement). - - For example: ASIL A can be Dashboard light failure (low risk) and ASIL D is Brake system failure (high risk). -- HARA (Hazard Analysis and Risk Assessment) - - Analyzes hazards and assesses risks to determine necessary safety measures. -- Safety Mechanisms - - Includes real-time error detection, system-level fault tolerance, and defined fail-safe or fail-operational fallback states. - -Typical Application Scenarios: -- Autonomous Driving Systems: - - Ensures that even if sensors (e.g., LiDAR, radar, cameras) provide faulty data, the vehicle will not make dangerous decisions. -- Powertrain Control: - - Prevents braking system failures that could lead to loss of control. -- Battery Management System (BMS): - - Prevents battery overheating or excessive discharge in electric vehicles. - -### Common Use Cases of Functional Safety in Automotive - -- Autonomous Driving: - - Ensures the vehicle can operate safely or enter a fail-safe state when sensors like LiDAR, radar, or cameras malfunction. - - Functional Safety enables real-time fault detection and fallback logic to prevent unsafe driving decisions. - -- Powertrain Control: - - Monitors throttle and brake signals to prevent unintended acceleration or braking loss. - - Includes redundancy, plausibility checks, and emergency overrides to maintain control under failure conditions. - -- Battery Management Systems (BMS): - - Protects EV batteries from overheating, overcharging, or deep discharge. - - Safety functions include temperature monitoring, voltage balancing, and relay cut-off mechanisms to prevent thermal runaway. - -These use cases highlight the need for a dedicated architectural layer that can enforce Functional Safety principles with real-time guarantees. -A widely adopted approach in modern automotive platforms is the Safety Island—an isolated compute domain designed to execute critical control logic independently of the main system. - -### Safety Island: Enabling Functional Safety in Autonomous Systems - -In automotive systems, a General ECU (Electronic Control Unit) typically runs non-critical tasks such as infotainment or navigation, whereas a Safety Island is dedicated to executing safety-critical control logic (e.g., braking, steering) with strong isolation, redundancy, and determinism. - -The table below compares the characteristics of a General ECU and a Safety Island in terms of their role in supporting Functional Safety. - -| Feature | General ECU | Safety Island | -|------------------------|----------------------------|--------------------------------------| -| Purpose | Comfort / non-safety logic | Safety-critical decision making | -| OS/Runtime | Linux, Android | RTOS, Hypervisor, or bare-metal | -| Isolation | Soft partitioning | Hard isolation (hardware-enforced) | -| Functional Safety Req | None to moderate | ISO 26262 ASIL-B to ASIL-D compliant | -| Fault Handling | Best-effort recovery | Deterministic safe-state response | - -This contrast highlights why safety-focused software needs a dedicated hardware domain with certified execution behavior. - -Safety Island is an independent safety subsystem separate from the main processor. It is responsible for monitoring and managing system safety. If the main processor fails or becomes inoperable, Safety Island can take over critical safety functions such as deceleration, stopping, and fault handling to prevent catastrophic system failures. - -Key Capabilities of Safety Island -- System Health Monitoring - - Continuously monitors the operational status of the main processor (e.g., ADAS control unit, ECU) and detects potential errors or anomalies. -- Fault Detection and Isolation - - Independently evaluates and initiates emergency handling if the main processing unit encounters errors, overheating, computational failures, or unresponsiveness. -- Providing Essential Safety Functions - - Even if the main system crashes, Safety Island can still execute minimal safety operations, such as: - - Autonomous Vehicles → Safe stopping (Fail-Safe Mode) - - Industrial Equipment → Emergency power cutoff or speed reduction - -### Why Safety Island Matters for Functional Safety - -Safety Island plays a critical role in Functional Safety by ensuring that the system can handle high-risk scenarios and minimize catastrophic failures. - -How Safety Island Enhances Functional Safety -1. Acts as an Independent Redundant Safety Layer - - Even if the main system fails, it can still operate independently. -2. Supports ASIL-D Safety Level - - Monitors ECU health status and executes emergency safety strategies, such as emergency braking. -3. Provides Independent Fault Detection and Recovery Mechanisms - - Fail-Safe: Activates a safe mode, such as limiting vehicle speed or switching to manual control. - - Fail-Operational: Ensures that high-safety applications, such as aerospace systems, can continue operating under certain conditions. - -### Functional Safety in the Software Development Lifecycle - -Functional Safety impacts both hardware and software development, particularly in areas such as requirement changes, version management, and testing validation. -For example, in ASIL-D level applications, every code modification requires a complete impact analysis and regression testing to ensure that new changes do not introduce additional risks. - -### Functional Safety Requirements in Software Development - -These practices ensure the software development process meets industry safety standards and can withstand system-level failures: -- Requirement Specification - - Clearly defining safety-critical requirements and conducting risk assessments. -- Safety-Oriented Programming - - Following MISRA C, CERT C/C++ standards and using static analysis tools to detect errors. -- Fault Handling Mechanisms - - Implementing redundancy design and health monitoring to handle anomalies. -- Testing and Verification - - Using Hardware-in-the-Loop (HIL) testing to ensure software safety in real hardware environments. -- Version Management and Change Control - - Using Git, JIRA, Polarion to track changes for safety audits. - -By establishing an ASIL Partitioning software development environment and leveraging SOAFEE technologies, you can enhance software consistency and maintainability in Functional Safety applications. - -This Learning Path follows [Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](/learning-paths/automotive/openadkit1_container/) and introduces Functional Safety design practices from the earliest development stages. \ No newline at end of file diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/1a_functional_safety.md b/content/learning-paths/automotive/openadkit2_safetyisolation/1a_functional_safety.md new file mode 100644 index 0000000000..9848838044 --- /dev/null +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/1a_functional_safety.md @@ -0,0 +1,25 @@ +--- +title: Why functional safety matters in software systems +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## What functional safety means for developers + +Functional safety helps systems detect faults and respond in ways that keep people and equipment safe. It ensures that even when errors occur, the system transitions into a known, safe state to prevent harm. + +This concept is foundational in domains like automotive, autonomous driving, medical devices, industrial control, robotics, and aerospace. In these systems, failures can have severe real-world consequences. + +In software development, functional safety focuses on minimizing risks through careful design, rigorous testing, and thorough validation. The goal is to make sure that critical systems behave predictably, reliably, and verifiably, even under fault conditions. + +To design for functional safety, developers must consider: + +- Error detection mechanisms +- Exception handling strategies +- Redundant system design +- Development processes that align with safety standards + + +In the following sections, you'll learn how to apply these principles throughout the software lifecycle, from early risk assessment and architectural design to runtime isolation and ISO 26262 compliance. diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/1b_purpose.md b/content/learning-paths/automotive/openadkit2_safetyisolation/1b_purpose.md new file mode 100644 index 0000000000..7363e87f80 --- /dev/null +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/1b_purpose.md @@ -0,0 +1,39 @@ +--- +title: Understand functional safety risks +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Manage risk with functional safety principles + +At its core, functional safety is about managing risk and reducing the impact of system failures. + +In autonomous vehicles, for example, functional safety ensures that if sensors produce unreliable or conflicting input, the vehicle can fall back to a known-safe state and maintain control. + +The three core objectives of functional safety are: + +- **Prevention** reduces the likelihood of errors through rigorous software development processes and testing. For example, electric vehicles monitor battery temperature to prevent overheating. +- **Detection** quickly identifies errors using built-in diagnostic mechanisms, such as self-test routines. +- **Mitigation** controls the impact of failures to ensure the system stays safe, even when something goes wrong. + +In practice, these principles might be implemented through: + +- Redundant sensor fusion code paths +- Timeout mechanisms for control loops +- Watchdog timers that reset components on fault detection +- Safe-state logic embedded in actuator control routines + +Together, prevention, detection, and mitigation form the foundation for building safer, more reliable software systems. + +In the next step, you’ll explore how functional safety principles are formalized through safety standards like ISO 26262 and applied to real-world systems. + + + + + + + + + diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/1c_iso26262.md b/content/learning-paths/automotive/openadkit2_safetyisolation/1c_iso26262.md new file mode 100644 index 0000000000..511c708777 --- /dev/null +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/1c_iso26262.md @@ -0,0 +1,39 @@ +--- +title: Apply ISO 26262 and ASIL levels +weight: 4 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- +## What ISO 26262 and ASIL levels mean for developers + +ISO 26262 is a functional safety standard for automotive electronics and software systems. It defines a structured safety lifecycle aligned with the V-model, spanning everything from initial requirements to final validation and maintenance. + +The V-model is a development framework where each design phase is paired with a corresponding test phase. This structure makes it easier to trace safety requirements from early specifications through to system verification. + +## Key concepts in ISO 26262 + +- **Automotive Safety Integrity Level (ASIL)** ranks the safety criticality of components on a scale from A (lowest) to D (highest). For example, ASIL A might apply to a dashboard indicator failure, while ASIL D applies to a brake system malfunction. + +- **Hazard Analysis and Risk Assessment (HARA)** identifies potential hazards and evaluates their risks to define the required safety goals and ASIL levels. + +- **Safety mechanisms** include techniques such as real-time fault detection, redundancy, and fallback modes like fail-safe and fail-operational behavior. + +{{% notice Note %}}In practice, many OEMs default to ASIL D for systems with any potential for passenger harm, even if the statistical likelihood of failure is low.{{% /notice %}} + +## Apply ISO 26262 to real-world systems + +ISO 26262 applies to many safety-critical vehicle systems: + +- **Autonomous driving systems** must respond safely to sensor errors (such as LiDAR, radar, or camera faults). Functional safety ensures the vehicle can enter a safe state and avoid unsafe decisions. + +- **Powertrain control** systems monitor throttle and braking inputs. Safety mechanisms such as redundancy, plausibility checks, and overrides prevent unintended acceleration or loss of braking function. + +- **Battery management systems (BMS)** protect electric vehicle batteries from overheating, overcharging, or deep discharge. Built-in safety functions monitor temperature, balance voltage, and isolate faulty circuits to prevent thermal runaway. + +These systems require dedicated hardware and software architectures that enforce functional safety guarantees. One common solution is the *safety island*, which is an isolated compute domain used to run safety-critical control logic independently from the main system. + + + + + diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/1d_safety_island_arch.md b/content/learning-paths/automotive/openadkit2_safetyisolation/1d_safety_island_arch.md new file mode 100644 index 0000000000..0b3f7c8588 --- /dev/null +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/1d_safety_island_arch.md @@ -0,0 +1,51 @@ +--- +title: Implement safety-critical isolation using safety island architecture +weight: 5 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- +## How safety islands support functional safety + +In automotive systems, a non-safety ECU (Electronic Control Unit) typically runs non-critical tasks such as infotainment or navigation. A safety island, by contrast, is dedicated to executing safety-critical control logic (for example, braking and steering) with strong isolation, redundancy, and determinism. + +The table below compares the characteristics of an ECU and a safety island in terms of their role in supporting functional safety. + +| Feature | ECU | Safety island | +|------------------------|----------------------------|--------------------------------------| +| Purpose | Comfort/non-safety logic | Safety-critical decision making | +| OS/runtime | Linux, Android | RTOS, hypervisor, or bare-metal | +| Isolation | Soft partitioning | Hardware-enforced isolation | +| Functional safety requirement | None to moderate | ISO 26262 ASIL-B to ASIL-D compliant | +| Fault handling | Best-effort recovery | Deterministic safe-state response | + +This comparison shows why safety-critical software depends on dedicated hardware domains to meet functional safety goals. + +If the main processor fails or becomes inoperable, a safety island can take over critical safety functions such as deceleration, stopping, and fault handling to prevent catastrophic system failures. + +{{% notice Tip %}} +Safety islands are often implemented as lockstep cores or separate MCUs that run on real-time operating systems (RTOS), offering guaranteed performance under fault conditions. +{{% /notice %}} + +## Key capabilities of a safety island +- **System health monitoring** continuously monitors the operational status of the main processor (for example, the ADAS control unit) and detects potential errors or anomalies +- **Fault detection and isolation** independently detects failures and initiates emergency handling for overheating, execution faults, or unresponsiveness +- **Essential safety functions continue to operate**, even if the main system crashes. A safety island can execute fallback operations, such as: + - Autonomous Vehicles → safe stopping (fail-safe mode) + - Industrial Equipment → emergency power cutoff or speed reduction + +## Why a safety island matters for functional safety + +A safety island provides a dedicated environment for executing critical safety functions. Its key characteristics include: + +- **Acting as an independent redundant safety layer** + - Operates safety logic independently of the main processor + +- **Supporting the ASIL-D safety level** + - Enables the system to meet the highest ISO 26262 requirements for critical operations + +- **Providing independent fault detection and recovery mechanisms:** + - *Fail-safe*: activating a minimal-risk mode, such as limiting vehicle speed or switching to manual control + - *Fail-operational*: allowing high-integrity systems, such as those in aerospace or autonomous driving, to continue functioning under fault conditions + +Safety islands play a key role in enabling ISO 26262 compliance by isolating safety-critical logic from general-purpose processing. They're a proven solution for improving system determinism, fault tolerance, and fallback behavior. \ No newline at end of file diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/1e_implement_functional_safety.md b/content/learning-paths/automotive/openadkit2_safetyisolation/1e_implement_functional_safety.md new file mode 100644 index 0000000000..a83767f462 --- /dev/null +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/1e_implement_functional_safety.md @@ -0,0 +1,35 @@ +--- +title: Functional safety for automotive software development +weight: 6 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## The software development lifecycle + +Functional safety affects both hardware and software development, particularly in areas such as requirement changes, version control, and test validation. For example, in ASIL-D level applications, every code change must go through a full impact analysis and regression testing to ensure it doesn't introduce new risks. + +## Software development practices for functional safety + +These practices ensure that software meets industry standards and can withstand system-level failures: +- **Defining requirements clearly** + - Specifying safety-critical requirements and conduct formal risk assessments. + +- **Following safety-oriented programming standards** + - Using MISRA C or CERT C/C++ and static analysis tools to detect unsafe behavior. + +- **Implementing fault-handling mechanisms** + - Using redundancy, health monitoring, and fail-safe logic to manage faults gracefully. + +- **Testing and verifying rigorously** + - Using Hardware-in-the-Loop (HIL) testing to validate behavior under realistic conditions. + +- **Tracking changes with version control and audits** + - Using tools like Git, JIRA, or Polarion to manage revisions and maintain traceability for audits. + +- **Building an ASIL-partitioned development environment and adopting SOAFEE technologies** to help improve software maintainability and ensure consistent compliance with functional safety standards. + +{{% notice Note %}} +This Learning Path builds on [Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](/learning-paths/automotive/openadkit1_container/). It introduces functional safety practices from the earliest stages of software development. +{{% /notice %}} diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/2_data_distribution_service.md b/content/learning-paths/automotive/openadkit2_safetyisolation/2_data_distribution_service.md index 19a36ba606..eabd2f6d9b 100644 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/2_data_distribution_service.md +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/2_data_distribution_service.md @@ -1,22 +1,27 @@ --- -title: How to Use Data Distribution Service (DDS) -weight: 3 +title: How to use Data Distribution Service (DDS) +weight: 7 ### FIXED, DO NOT MODIFY layout: learningpathall --- -### Introduction to DDS +## Introduction to DDS Data Distribution Service (DDS) is a real-time, high-performance middleware designed for distributed systems. It is particularly valuable in automotive software development, including applications such as autonomous driving (AD) and advanced driver assistance systems (ADAS). -DDS offers a decentralized architecture that enables scalable, low-latency, and reliable data exchange, making it ideal for managing high-frequency sensor streams. +DDS uses a decentralized architecture that supports scalable, low-latency, and reliable data exchange, which is ideal for managing high-frequency sensor streams. In modern vehicles, multiple sensors such as LiDAR, radar, and cameras must continuously communicate with compute modules. DDS ensures these components share data seamlessly and in real time, both within the vehicle and across infrastructure such as V2X systems, including traffic lights and road sensors. -### Why Automotive Software Needs DDS +{{% notice Tip %}} +To get started with open-source DDS on Arm platforms, see the [Installation Guide for CycloneDDS](https://learn.arm.com/install-guides/cyclonedds). +{{% /notice %}} + + +## Why automotive software needs DDS Next-generation automotive software architectures, such as SOAFEE, depend on deterministic, distributed communication. Traditional client-server models introduce latency and create single points of failure. In contrast, DDS uses a publish-subscribe model that enables direct, peer-to-peer communication across system components. @@ -26,7 +31,7 @@ Additionally, DDS provides a flexible Quality of Service (QoS) configuration, al These capabilities make DDS an essential backbone for autonomous vehicle stacks, where real-time sensor fusion and control coordination are critical for safety and performance. -### DDS Architecture and Operation +## DDS architecture and operation DDS uses a data-centric publish-subscribe (DCPS) model, allowing producers and consumers of data to communicate without direct dependencies. This modular approach enhances system flexibility and maintainability, making it well suited for complex automotive environments. @@ -41,7 +46,7 @@ Each domain contains multiple topics, representing specific data types such as v For example, in an autonomous vehicle, LiDAR, radar, and cameras continuously generate large amounts of sensor data. The perception module subscribes to these sensor topics, processes the data, and then publishes detected objects and road conditions to other components like path planning and motion control. Since DDS automatically handles participant discovery and message distribution, engineers do not need to manually configure communication paths, reducing development complexity. -### Real-World Use in Autonomous Driving +## Real-world use in autonomous driving DDS is widely used in autonomous driving systems, where real-time data exchange is crucial. A typical use case involves high-frequency sensor data transmission and decision-making coordination between vehicle subsystems. @@ -53,7 +58,7 @@ For example, Autoware, an open-source autonomous driving software stack, uses DD The Perception stack publishes detected objects from LiDAR and camera sensors to a shared topic, which is then consumed by the Planning module in real-time. Using DDS allows each subsystem to scale independently while preserving low-latency and deterministic communication. -### Publish-Subscribe Model and Data Transmission +## Publish-subscribe model and data transmission Let’s explore how DDS’s publish-subscribe model fundamentally differs from traditional communication methods in terms of scalability, latency, and reliability. @@ -68,11 +73,11 @@ DDS supports multiple transport mechanisms to optimize communication efficiency: * UDP or TCP/IP is used for inter-device communication, such as V2X applications where vehicles exchange safety-critical messages. * Automatic participant discovery eliminates the need for manual configuration, allowing DDS nodes to detect and establish connections dynamically. -#### Comparison of DDS and Traditional Communication Methods +## Compare DDS with traditional communication models The following table highlights how DDS improves upon traditional client-server communication patterns in the context of real-time automotive applications: -| Feature | Traditional Client-Server Architecture | DDS Publish-Subscribe Model | +| Feature | Traditional client-server architecture | DDS publish-subscribe model | |----------------------|--------------------------------------------|--------------------------- | | Data Transmission | Relies on a central server | Direct peer-to-peer communication | | Latency | Higher latency | Low latency | @@ -82,9 +87,9 @@ The following table highlights how DDS improves upon traditional client-server c These features make DDS a highly adaptable solution for automotive software engineers seeking to develop scalable, real-time communication frameworks. -In this section, you learned how DDS enables low-latency, scalable, and fault-tolerant communication for autonomous vehicle systems. +DDS is a critical building block in distributed automotive systems. By enabling scalable, low-latency communication and fault tolerance, it powers real-time coordination in modern architectures like ROS 2, SOAFEE, and Autoware. + +Its data-centric publish-subscribe architecture removes the limitations of traditional client-server models and forms the backbone of modern automotive software frameworks like ROS 2 and SOAFEE. -Its data-centric publish-subscribe architecture eliminates the limitations of traditional client-server models and forms the backbone of modern automotive software frameworks such as ROS 2 and SOAFEE. -To get started with open-source DDS on Arm platforms, refer to this [installation guide for Cyclonedds](https://learn.arm.com/install-guides/cyclonedds) on how to install open-source DDS on an Arm platform. diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md b/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md index 94c5d71714..b4db0659d8 100644 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md @@ -1,45 +1,41 @@ --- -title: Split into multiple cloud container instances -weight: 4 +title: Deploy OpenAD Kit across multiple cloud instances +weight: 8 ### FIXED, DO NOT MODIFY layout: learningpathall --- -### System Architecture and Component Design +## Refactor OpenAD Kit for distributed deployment -Now that you’ve explored the concept of a Safety Island, a dedicated subsystem responsible for executing safety-critical control logic, and learned how DDS (Data Distribution Service) enables real-time, distributed communication, you’ll refactor the original OpenAD Kit architecture into a multi-instance deployment. +Now that you’ve explored the concept of a safety island, a dedicated subsystem responsible for executing safety-critical control logic, and learned how DDS (Data Distribution Service) enables real-time, distributed communication, you’ll refactor the original OpenAD Kit architecture into a multi-instance deployment. -In [Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](http://learn.arm.com/learning-paths/automotive/openadkit1_container/), you deployed three container components on a single Arm-based instance, handling: -- Simulation environment +The predecessor Learning Path, [Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](http://learn.arm.com/learning-paths/automotive/openadkit1_container/), showed how to deploying three container components on a single Arm-based instance, to handle: +- The simulation environment - Visualization -- Planning-Control +- Planning and control -In this session, you will split the simulation and visualization stack from the planning-control logic and deploy them across two independent Arm-based instances. +In this Learning Path, you will split the simulation and visualization stack from the planning-control logic and deploy them across two independent Arm-based instances. These nodes communicate using ROS 2 with DDS as the middleware layer, ensuring low-latency and fault-tolerant data exchange between components. -### Architectural Benefits +## Architectural Benefits This architecture brings several practical benefits: -- Enhanced System Stability: -Decoupling components prevents resource contention and ensures that safety-critical functions remain deterministic and responsive. +- **Enhanced System Stability**: decoupling components prevents resource contention and ensures that safety-critical functions remain deterministic and responsive. -- Real-Time, Scalable Communication: -DDS enables built-in peer discovery and configurable QoS, removing the need for a central broker or manual network setup. +- **Real-Time, Scalable Communication**: DDS enables built-in peer discovery and configurable QoS, removing the need for a central broker or manual network setup. -- Improved Scalability and Performance Tuning: -Each instance can be tuned based on its workload—for example, simulation tasks can use GPU-heavy hardware, while planning logic may benefit from CPU-optimized setups. +- **Improved Scalability and Performance Tuning**: each instance can be tuned based on its workload,for example, simulation tasks can use GPU-heavy hardware, while planning logic might benefit from CPU-optimized setups. -- Support for Modular CI/CD Workflows: -With containerized separation, you can build, test, and deploy each module independently—enabling agile development and faster iteration cycles. +- **Support for Modular CI/CD Workflows**: with containerized separation, you can build, test, and deploy each module independently, which enables agile development and faster iteration cycles. -![img1 alt-text#center](aws_example.jpg "Figure 1: Split instance example in AWS") +![img1 alt-text#center](aws_example.jpg "Split instance example in AWS") -### Networking Setting +## Configure networking for DDS communication -To begin, launch two Arm-based VM instances. AWS EC2 is used, but you can use any Arm instances. +To begin, launch two Arm-based VM instances. AWS EC2 is used, but you can use any Arm-based instances. These instances will independently host your simulation and control workloads. @@ -55,22 +51,22 @@ To enable ROS 2 and DDS communication between the two nodes, configure network a If you are using AWS EC2, both instances should be assigned to the same Security Group. Within the EC2 Security Group settings: -- Add an inbound rule that allows all traffic from the same Security Group by setting he source to the security group itself. +- Add an inbound rule that allows all traffic from the same Security Group by setting the source to the security group itself. - Outbound traffic is typically allowed by default and usually does not require changes. -![img2 alt-text#center](security_group.jpg "Figure 2: AWS Security Group Setting") +![img2 alt-text#center](security_group.jpg "AWS Security Group Setting") This configuration allows automatic discovery and peer-to-peer communication between DDS participants across the two instances. Once both systems are operational, record the private IP addresses of each instance. You will need them when configuring CycloneDDS peer discovery in the next step. -### New Docker YAML Configuration Setting +## Update Docker and DDS configuration -Before you begin, ensure that Docker is installed on both of your development instances. Review the [Docker install guide](/install-guides/docker/docker-engine/) if needed. +Before you begin, ensure that Docker is installed on both of your development instances. Review the [Docker Install Guide](/install-guides/docker/docker-engine/) if needed. First, clone the demo repo and create xml file called `cycloneDDS.xml` -#### Step 1: Clone the repository and prepare configuration files +## Clone the repository and prepare configuration files ```bash git clone https://github.com/odincodeshen/openadkit_demo.autoware.git @@ -95,20 +91,20 @@ export COMMON_FILE=/home/ubuntu/openadkit_demo.autoware/docker/etc/simulation/co docker compose -f docker-compose.yml pull ``` -{{% notice info %}} -Each image is around 4–6 GB, so pulling them may vary depending on your network speed. +{{% notice Note %}} +Each image is around 4–6 GB, so the time to download them can vary depending on your network speed. {{% /notice %}} -This command will download all images defined in the docker-compose-2ins.yml file, including: +This command downloads all images defined in the docker-compose-2ins.yml file, including: - odinlmshen/autoware-simulator:v1.0 - odinlmshen/autoware-planning-control:v1.0 - odinlmshen/autoware-visualizer:v1.0 -#### Step 2: Configure CycloneDDS for Peer-to-Peer Communication +## Configure CycloneDDS for peer-to-peer communication The cycloneDDS.xml file is used to customize how CycloneDDS (the middleware used by ROS 2) discovers and communicates between distributed nodes. -Please copy the following configuration into docker/cycloneDDS.xml on both machines, and replace the IP addresses with the private IPs of each EC2 instance. +Copy the following configuration into docker/cycloneDDS.xml on both machines, and replace the IP addresses with the private IPs of each EC2 instance: ```xml @@ -128,7 +124,8 @@ Please copy the following configuration into docker/cycloneDDS.xml on both machi 1000 auto - + + @@ -141,19 +138,19 @@ Please copy the following configuration into docker/cycloneDDS.xml on both machi ``` {{% notice Note %}} -1. Make sure the network interface name (ens5) matches the one on your EC2 instances. You can verify this using `ip -br a`. -2. This configuration disables multicast and enables static peer discovery between the two machines using unicast. -3. You can find the more detail about CycloneDDS setting [Configuration](https://cyclonedds.io/docs/cyclonedds/latest/config/config_file_reference.html#cyclonedds-domain-internal-socketreceivebuffersize) +- Make sure the network interface name (for example, `ens5`) matches the one used by your EC2 instances. You can verify the instance name by running `ip -br a`. +- This configuration disables multicast and enables static peer discovery between the two machines using unicast. +- For more information on CycloneDDS settings, see the [Cyclone DDS Configuration Guide](https://cyclonedds.io/docs/cyclonedds/latest/config/config_file_reference.html#cyclonedds-domain-internal-socketreceivebuffersize). {{% /notice %}} -#### Step 3: Update the Docker Compose Configuration for Multi-Host Deployment +## Update the Docker Compose Configuration for Multi-Host Deployment To support running containers across two separate hosts, you’ll need to modify the docker/docker-compose-2ins.yml file. This includes removing inter-container dependencies and updating the network and environment configuration. -##### Remove Cross-Container Dependency +## Remove cross-container dependency -Since the planning-control and simulator containers will now run on different machines, you must remove any depends_on references between them to prevent Docker from attempting to start them on the same host. +Since the planning-control and simulator containers will run on separate machines, remove any `depends_on` references between them. This prevents Docker from treating them as interdependent services on a single host. ```YAML planning-control: @@ -162,9 +159,10 @@ Since the planning-control and simulator containers will now run on different ma # - simulator ``` -##### Enable Host Networking +## Enable host networking + +All three containers (visualizer, simulator, and planning-control) need access to the host’s network interfaces for DDS-based peer discovery. -All three containers (visualizer, simulator, planning-control) need access to the host’s network interfaces for DDS-based peer discovery. Replace Docker's default bridge network with host networking: ```YAML @@ -172,7 +170,7 @@ Replace Docker's default bridge network with host networking: network_mode: host ``` -##### Use CycloneDDS Configuration via Environment Variable +## Apply the CycloneDDS configuration using an environment variable To ensure that each container uses your custom DDS configuration, mount the current working directory and set the CYCLONEDDS_URI environment variable: @@ -185,7 +183,7 @@ To ensure that each container uses your custom DDS configuration, mount the curr Add this to every container definition to ensure consistent behavior across the deployment. -Here is the complete XML file: +Here is the complete YAML file: ```YAML services: @@ -258,7 +256,7 @@ services: Before moving to the next step, make sure that `docker-compose-2ins.yml` and `cycloneDDS.xml` are already present on both instances. -#### Step 4: Optimize Network Settings for DDS Communication +## Optimize Network Settings for DDS Communication In a distributed DDS setup, `high-frequency UDP traffic` between nodes may lead to `IP packet fragmentation` or `buffer overflows`, especially under load. These issues can degrade performance or cause unexpected system behavior. @@ -270,10 +268,10 @@ sudo sysctl net.ipv4.ipfrag_high_thresh=134217728 sudo sysctl -w net.core.rmem_max=2147483647 ``` -Explanation of Parameters -- `net.ipv4.ipfrag_time=3`: Reduces the timeout for holding incomplete IP fragments, helping free up memory more quickly. -- `net.ipv4.ipfrag_high_thresh=134217728`: Increases the memory threshold for IP fragment buffers to 128 MB, preventing early drops under high load. -- `net.core.rmem_max=2147483647`: Expands the maximum socket receive buffer size to support high-throughput DDS traffic. +## Explanation of parameters +- `net.ipv4.ipfrag_time=3`: reduces the timeout for holding incomplete IP fragments, helping free up memory more quickly. +- `net.ipv4.ipfrag_high_thresh=134217728`: increases the memory threshold for IP fragment buffers to 128 MB, preventing early drops under high load. +- `net.core.rmem_max=2147483647`: expands the maximum socket receive buffer size to support high-throughput DDS traffic. To ensure these settings persist after reboot, create a configuration file under /etc/sysctl.d/: @@ -295,18 +293,19 @@ Links to documentation: - [ROS2 documentation](https://docs.ros.org/en/humble/How-To-Guides/DDS-tuning.html#cyclone-dds-tuning) -#### Step 5: Verifying Cross-Instance DDS Communication with ROS 2 +## Verify DDS communication between instances using ROS 2 -To confirm that ROS 2 nodes can exchange messages across two separate EC2 instances using DDS, this test will walk you through a minimal publisher–subscriber setup using a custom topic. +To confirm that ROS 2 nodes can exchange messages across two separate EC2 instances using DDS, this test walks you through a minimal publisher-subscriber setup using a custom topic. -##### On Planning-Control Node (Publisher) +## On the planning-control node (publisher) On the first EC2 instance, you will publish a custom message to the /hello topic using ROS 2. This will simulate outbound DDS traffic from the planning-control container. -Set the required environment variables and launch the planning-control container. +Set the required environment variables and launch the planning-control container: ```bash +# Replace this path with the location where you cloned the repository, if different export SCRIPT_DIR=/home/ubuntu/openadkit_demo.autoware/docker export CONF_FILE=$SCRIPT_DIR/etc/simulation/config/fail_static_obstacle_avoidance.param.yaml export COMMON_FILE=$SCRIPT_DIR/etc/simulation/config/common.param.yaml @@ -317,7 +316,7 @@ export TIMEOUT=300 docker compose -f docker-compose-2ins.yml run --rm planning-control bash ``` -Once inside the container shell, activate the ROS 2 environment and start publishing to the /hello topic: +Once inside the container shell, activate the ROS 2 environment and start listening to the /hello topic: ```bash # Inside the container: @@ -326,7 +325,7 @@ ros2 topic list ros2 topic pub /hello std_msgs/String "data: Hello From Planning" --rate 1 ``` -##### On Simulator Node (Subscriber) side +## On the simulator node (subscriber) On the second EC2 instance, you will listen for the /hello topic using ros2 topic echo. This confirms that DDS communication from the planning node is received on the simulation node. @@ -353,7 +352,7 @@ export TIMEOUT=300 docker compose -f docker-compose-2ins.yml run --rm simulator bash ``` -Once inside the container shell, activate the ROS 2 environment and start publishing to the /hello topic: +Once inside the container shell, activate the ROS 2 environment and start listening to the /hello topic: ```bash # Inside the container: @@ -363,6 +362,11 @@ ros2 topic echo /hello ``` In the simulator container, you should see repeated outputs like: + +{{% notice tip %}} +If the subscriber shows a continuous stream of `/hello` messages, DDS discovery and ROS 2 communication between the nodes is working as expected. +{{% /notice %}} + ``` root@ip-172-31-19-5:/autoware# ros2 topic echo /hello data: Hello From Planning diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md b/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md index 12d69659c5..142aa3e88f 100644 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md @@ -1,34 +1,33 @@ --- -title: Executing OpenAD Kit in a Distributed ROS 2 Instances +title: Run OpenAD Kit across distributed ROS 2 instances -weight: 5 +weight: 9 ### FIXED, DO NOT MODIFY layout: learningpathall --- -### Demonstrating the Distributed OpenAD Kit in Action +## What you'll learn in this section -In this section, you’ll bring all the previous setup together and execute the full OpenAD Kit demo across two Arm-based instances. +The OpenAD Kit is an open-source reference design for autonomous driving workloads on Arm. It demonstrates how Autoware modules can run on scalable infrastructure, whether on a single machine or distributed across multiple compute nodes using ROS 2 and DDS. -OpenAD Kit is an open-source reference design for autonomous driving workloads on Arm. -It demonstrates how Autoware modules can be deployed on scalable infrastructure, whether on a single machine or split across multiple compute nodes. +In this section, you'll run the full OpenAD Kit demo across two Arm-based cloud instances using the setup from previous steps. -#### Preparing the Execution Scripts +## Set up launch scripts on both instances -This setup separates the simulation/visualization environment from the planning-control logic, allowing you to explore how ROS 2 nodes communicate over a distributed system using DDS (Data Distribution Service). +This setup separates the simulation and visualization environment from the planning and control logic, allowing you to explore how ROS 2 nodes communicate over a distributed system using DDS (Data Distribution Service). -To start the system, you need to configure and run separate launch commands on each machine. +To start the system, run separate launch scripts on each machine: On each instance, copy the appropriate launch script into the `openadkit_demo.autoware/docker` directory. {{< tabpane code=true >}} {{< tab header="Planning-Control" language="bash">}} - !/bin/bash + #!/bin/bash # Configure the environment variables export SCRIPT_DIR=/home/ubuntu/openadkit_demo.autoware/docker - CONF_FILE_PASS=$SCRIPT_DIR/etc/simulation/config/pass_static_obstacle_avoidance.param.yaml - CONF_FILE_FAIL=$SCRIPT_DIR/etc/simulation/config/fail_static_obstacle_avoidance.param.yaml + export CONF_FILE_PASS=$SCRIPT_DIR/etc/simulation/config/pass_static_obstacle_avoidance.param.yaml + export CONF_FILE_FAIL=$SCRIPT_DIR/etc/simulation/config/fail_static_obstacle_avoidance.param.yaml export CONF_FILE=$CONF_FILE_FAIL export COMMON_FILE=$SCRIPT_DIR/etc/simulation/config/common.param.yaml @@ -40,9 +39,9 @@ On each instance, copy the appropriate launch script into the `openadkit_demo.au TIMEOUT=120 CONF_FILE=$CONF_FILE_PASS docker compose -f "$SCRIPT_DIR/docker-compose-2ins.yml" up planning-control -d {{< /tab >}} - {{< tab header="Visualizer & Simulator" language="bash">}} + {{< tab header="Visualizer and simulator" language="bash">}} #!/bin/bash - SCRIPT_DIR=/home/ubuntu/openadkit_demo.autoware/docker + export SCRIPT_DIR=/home/ubuntu/openadkit_demo.autoware/docker export CONF_FILE_FAIL=$SCRIPT_DIR/etc/simulation/config/fail_static_obstacle_avoidance.param.yaml export CONF_FILE=$CONF_FILE_FAIL @@ -67,11 +66,11 @@ On each instance, copy the appropriate launch script into the `openadkit_demo.au {{< /tab >}} {{< /tabpane >}} -You can also find the prepared launch scripts `opad_planning.sh` and `opad_sim_vis.sh` inside the `openadkit_demo.autoware/docker` directory on both instances. +You can also find these scripts `opad_planning.sh` and `opad_sim_vis.sh` inside the `openadkit_demo.autoware/docker` directory on both instances. These scripts encapsulate the required environment variables and container commands for each role. -#### Running the Distributed OpenAD Kit Demo +## Run the distributed OpenAD Kit demo On the Planning-Control node, execute: @@ -85,18 +84,16 @@ On the Simulation and Visualization node, execute: ./opad_sim_vis.sh ``` +Once both machines are running their launch scripts, the Visualizer container exposes a web-accessible interface at: http://6080/vnc.html. -Once both machines are running their respective launch scripts, the Visualizer will generate a web-accessible interface on: - -http://[Visualizer public IP address]:6080/vnc.html +Open this link in your browser to observe the simulation in real time. The demo closely resembles the output in the [previous Learning Path, Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](http://learn.arm.com/learning-paths/automotive/openadkit1_container/4_run_openadkit/). -You can open this link in a browser to observe the demo behavior, which will closely resemble the output from the [previous learning path](http://learn.arm.com/learning-paths/automotive/openadkit1_container/4_run_openadkit/). +![Distributed OpenAD Kit simulation running on two Arm-based instances with visualizer and simulator coordination over DDS alt-text#center](split_aws_run.gif "Visualizer output from a distributed OpenAD Kit simulation showing ROS 2 modules running across two cloud instances using DDS communication.") -![img3 alt-text#center](split_aws_run.gif "Figure 4: Simulation") +You’ve now run the OpenAD Kit across two nodes with separated control and visualization roles. DDS enabled real-time, peer-to-peer communication between the ROS 2 nodes, supporting synchronized behavior across the planning and simulation components deployed on two separate instances. -The containers are now distributed across two separate instances, enabling real-time, cross-node communication. -Behind the scenes, this architecture demonstrates how DDS manages low-latency, peer-to-peer data exchange in a distributed ROS 2 environment. +The containers are now distributed across two separate instances, enabling real-time, cross-node communication. Behind the scenes, this architecture demonstrates how DDS manages low-latency, peer-to-peer data exchange in a distributed ROS 2 environment. -To support demonstration and validation, the simulator is configured to run three times sequentially, giving you multiple opportunities to observe how data flows between nodes and verify that communication remains stable across each cycle. +The simulator runs three times by default, giving you multiple chances to observe data flow and verify stable communication between nodes. -Now that you’ve seen the distributed system in action, consider exploring different QoS settings, network conditions, or even adding a third node to expand the architecture further. +Now that you’ve seen the distributed system in action, try modifying QoS settings, simulating network conditions, or scaling to a third node to explore more complex configurations. diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/_index.md b/content/learning-paths/automotive/openadkit2_safetyisolation/_index.md index 4319e4824a..55245d22ff 100644 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/_index.md +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/_index.md @@ -1,23 +1,19 @@ --- -title: Prototyping Safety-Critical Isolation for Autonomous Application on Neoverse - -draft: true -cascade: - draft: true +title: Prototype safety-critical isolation for autonomous driving systems on Neoverse minutes_to_complete: 60 -who_is_this_for: This Learning Path targets advanced automotive software engineers developing safety-critical systems. It demonstrates how to use Arm Neoverse cloud infrastructure to accelerate ISO-26262 compliant software prototyping and testing workflows. +who_is_this_for: This Learning Path is for automotive engineers developing safety-critical systems. You'll learn how to accelerate ISO 26262-compliant development workflows using Arm-based cloud compute, containerized simulation, and DDS-based communication. learning_objectives: - - Learn the Functional Safety principles—including risk prevention, fault detection, and ASIL compliance—to design robust and certifiable automotive software systems. - - Understand how DDS enables low-latency, scalable, and fault-tolerant data communication for autonomous driving systems using a publish-subscribe architecture. - - Distributed Development for Functional Safety. Learn how to split the simulation platform into two independent units and leverage distributed development architecture to ensure functional safety. + - Apply functional safety principles, including risk prevention, fault detection, and ASIL compliance, to build robust, certifiable automotive systems + - Use DDS and a publish-subscribe architecture for low-latency, scalable, and fault-tolerant communication in autonomous driving systems + - Implement distributed development by separating the simulation platform into independent, safety-isolated components prerequisites: - - Two Arm-based Neoverse cloud instances or a local Arm Neoverse Linux computer with at least 16 CPUs and 32GB of RAM. - - To have completed [Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](/learning-paths/automotive/openadkit1_container/). - - Basic knowledge of using Docker. + - Access to two Arm-based Neoverse cloud instances, or a local Arm Neoverse Linux system with at least 16 CPUs and 32 GB of RAM + - Completion of the [Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](/learning-paths/automotive/openadkit1_container/) Learning Path + - Basic familiarity with Docker author: - Odin Shen @@ -32,6 +28,7 @@ tools_software_languages: - Python - Docker - ROS2 + - DDS operatingsystems: - Linux diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/1_intro-zenoh.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/1_intro-zenoh.md index 5e1595325a..3b46ee7423 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/1_intro-zenoh.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/1_intro-zenoh.md @@ -1,12 +1,12 @@ --- -title: Introduction to Zenoh +title: Build scalable communication systems with Eclipse Zenoh weight: 2 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## The Need for Scalable Communication in Robotics and Edge Computing +## The need for scalable communication in robotics and edge computing Modern robotics and industrial IoT systems are evolving rapidly, from indoor collaborative arms on assembly lines to fleets of outdoor autonomous delivery robots. These applications must operate in real-time, often across multiple compute nodes, over networks that may span factory LANs, 5G cellular links, or even cloud data centers. @@ -15,36 +15,34 @@ Such systems require fast, reliable, and scalable data communication between nod This includes not just broadcasting sensor updates or actuator commands (pub/sub), but also performing state queries, storing values for later use, and even distributed computation across nodes. A modern protocol must be: -* Low-latency: Immediate delivery of time-critical messages. -* High-throughput: Efficient data flow across many devices. -* Decentralized: No reliance on central brokers or fixed infrastructure. -* Flexible: Able to run on lightweight edge devices, across WANs, or inside cloud-native environments. +* Low-latency: immediate delivery of time-critical messages +* High-throughput: efficient data flow across many devices +* Decentralized: no reliance on central brokers or fixed infrastructure +* Flexible: able to run on lightweight edge devices, across WANs, or inside cloud-native environments -Traditional communication stacks such as Data Distribution Service (DDS) serve as the backbone for middleware like ROS 2. However, DDS struggles in multi-network or wireless environments where multicast is unavailable, or NAT traversal is needed. -These constraints can severely impact deployment and performance at the edge. +Traditional communication stacks such as Data Distribution Service (DDS) serve as the backbone for middleware like ROS 2. However, DDS struggles in multi-network or wireless environments where multicast is unavailable, or NAT traversal is needed. Zenoh was developed to address these challenges with a unified approach to data movement, storage, and interaction across heterogeneous, distributed systems. - -## Zenoh: an open-source pub/sub protocol for the Industrial Edge +## What is Zenoh? A scalable pub/sub protocol for the industrial edge [Eclipse Zenoh](https://zenoh.io/) is a modern, open-source, data-centric communication protocol that goes beyond traditional pub/sub. Designed specifically for edge computing, industrial IoT, robotics, and autonomous systems, Zenoh unifies: -- Data in motion through a powerful and efficient pub/sub model. -- Data at rest via geo-distributed storage plugins. -- Data in use with direct query support. -- On-demand computation via queryable nodes that can generate data dynamically. +- Data in motion through a powerful and efficient pub/sub model +- Data at rest via geo-distributed storage plugins +- Data in use with direct query support +- On-demand computation via queryable nodes that can generate data dynamically Unlike most traditional stacks, Zenoh is fully decentralized and designed to operate across cloud-to-edge-to-thing topologies, making it ideal for industrial robotics, autonomous systems, and smart environments. It supports heterogeneous platforms, is implemented in Rust for performance and safety, and also offers bindings for Python, enabling rapid prototyping. -Zenoh is particularly effective in wireless, 5G, or cross-network deployments where multicast and DDS fall short. -Its routing engine avoids excessive discovery traffic, conserves bandwidth, and supports seamless bridging between legacy ROS 2/DDS apps and modern, optimized Zenoh networks using Zenoh-Bridge-DDS, a bridge between DDS systems and Zenoh networks. +Zenoh excels in wireless, 5G, and cross-network deployments, which are environments where multicast is unavailable and DDS often struggles. Its routing engine avoids excessive discovery traffic, conserves bandwidth, and supports seamless bridging between legacy ROS 2/DDS apps and modern, optimized Zenoh networks using Zenoh-Bridge-DDS, a bridge between DDS systems and Zenoh networks. In this Learning Path, you’ll use Zenoh to build and validate a multi-node distributed communication system across multiple Arm-based platforms, gaining hands-on experience with data exchange and coordination between edge devices. -The examples use Raspberry Pi boards, but you’re free to substitute any Cortex-A or Neoverse devices that support network connectivity running Linux. +You can substitute Raspberry Pi with any Linux-based Arm device that supports networking, such as a Cortex-A or Neoverse board. + +By the end of this Learning Path, you'll be able to: +- Explain the core architecture and data flow principles behind Eclipse Zenoh protocol, including its support for pub/sub, querying, and queryable edge functions +- Build and run distributed Zenoh examples across multiple Arm-based nodes using Raspberry Pi or other Arm Linux devices +- Rebuild and extend a Zenoh queryable node to simulate edge-side logic -Upon completion, you will be able to: -- Understand the core architecture and data flow principles behind Eclipse Zenoh, including its support for pub/sub, querying, and queryable edge functions. -- Build and run distributed Zenoh examples across multiple Arm-based nodes using Raspberry Pi or other Arm Linux devices. -- Rebuild and extend the Zenoh queryable example to simulate edge-side logic. diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/2_zenoh-install.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/2_zenoh-install.md index da1a28cbe4..2c40074624 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/2_zenoh-install.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/2_zenoh-install.md @@ -1,20 +1,21 @@ --- -title: Setting Up Zenoh on Arm Devices +title: Get started with Zenoh on Raspberry Pi and Arm Linux + weight: 3 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Setting Up Zenoh on Arm Devices +## Set up Zenoh on Arm devices -The following instructions have been verified on both Raspberry Pi 4 and 5 devices, but you can implement them on any Arm Linux device. +This section shows you how to install and build the open-source Eclipse Zenoh protocol on Arm-based devices like Raspberry Pi. -Before building Zenoh, make sure your system has the necessary development tools and runtime libraries. +The following instructions have been verified on Raspberry Pi 4 and 5, but you can use any Arm Linux device. These steps apply to Raspberry Pi and other Arm-based Linux platforms. Before building Zenoh, make sure your system has the necessary development tools and runtime libraries. -### Install the Rust development environment +## Install the Rust development environment -First, install the [Rust](https://www.rust-lang.org/) environment, since the core of Zenoh is developed using Rust to keep it safe and efficient. +First, install the [Rust](https://www.rust-lang.org/) environment. The core of Zenoh is developed in Rust for performance and safety. ```bash sudo apt update @@ -22,27 +23,26 @@ sudo apt install -y curl gcc curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y ``` -Near the end of the installation you will see the success message: +Near the end of the installation, you should see the message: ```output Rust is installed now. Great! ``` - -Make sure to source the environment to add Rust to your shell environment: +Source your shell environment to activate Rust: ```bash source "$HOME/.cargo/env" ``` -You can learn more using the [Rust install guide](/install-guides/rust/) for Arm Linux. +For more information, see the [Rust Install Guide](/install-guides/rust/) for Arm Linux. -### Install ROS 2 +## Install ROS 2 [Robot Operating System](https://www.ros.org/) is a set of software libraries and tools that help you build robot applications. ROS provides everything from drivers to state-of-the-art algorithms, as well as developer tools. It is completely open-source. -Follow the [ROS2 installation guide](/install-guides/ros2/) to install ROS 2 on your Arm platforms. +If you plan to use Zenoh alongside ROS 2, for example, to bridge DDS-based nodes, you should install ROS 2 before proceeding. See the [ROS2 Installation Guide](/install-guides/ros2/) to install ROS 2 on your Arm platforms. -### Download and build the Zenoh source +## Download and build the Zenoh source Clone the Zenoh repository: @@ -50,15 +50,13 @@ Clone the Zenoh repository: cd $HOME git clone https://github.com/eclipse-zenoh/zenoh.git ``` - -After cloning, use cargo to build the source: +Build the source using Cargo: ```bash cd zenoh cargo build --release --all-targets -j $(nproc) ``` - -This process will take several minutes depending on your device. Once the installation is complete, you should see: +This process will take several minutes depending on your device. When complete, you should see output like: ```output Updating crates.io index @@ -101,13 +99,16 @@ This may become a hard error in the future; see plugins/zenoh-plugin-storage-manager/tests/operations.rs:91:55 @@ -120,7 +121,7 @@ Edit the file `./plugins/zenoh-plugin-storage-manager/tests/operations.rs` and c use crate::path::to::Plugin; ``` -Run the build again: +Then rebuild: ```bash cargo clean && cargo build diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/3_zenoh-multinode.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/3_zenoh-multinode.md index 06e17d6012..bc7f7ccaf7 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/3_zenoh-multinode.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/3_zenoh-multinode.md @@ -1,39 +1,35 @@ --- -title: Setting Up a Multi-Node Environment +title: Containerize and deploy Zenoh across multiple Raspberry Pi devices + weight: 4 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Deploying Zenoh on Multiple Raspberry Pi Devices +## Deploy Zenoh on multiple Raspberry Pi devices After building Zenoh and its core examples, the next step is to deploy them across multiple Arm-based devices. -If you’ve already installed Zenoh on an Arm Cortex-A or Neoverse platform as shown in the previous section, you can simply copy the compiled binaries from `~/zenoh/target/release/` to each of your Raspberry Pi devices. - -However, to streamline deployment across multiple devices and ensure repeatability, this section demonstrates how to package Zenoh into a Docker image for batch rollout and scalable testing. - -This containerized approach not only simplifies deployment on Raspberry Pi, but also integrates seamlessly with Arm cloud platforms such as AWS Graviton Arm Cortex-A Linux or Arm Virtual Hardware—enabling a consistent cloud-to-edge development and validation workflow. +If you’ve already installed Zenoh on an Arm Cortex-A or Neoverse platform as shown in the previous section, you can copy the compiled binaries from `~/zenoh/target/release/` to each of your Raspberry Pi devices. -In this session, you’ll use Raspberry Pi boards to simulate a scalable distributed environment. The same workflow applies to any Arm Linux system, including cloud instances and virtual hardware. +To simplify and scale deployment across multiple devices, this section shows how to containerize Zenoh with Docker for streamlined distribution and consistent multi-node testing. This containerized approach enables repeatable rollouts and makes it easier to test distributed communication across Raspberry Pi, Arm cloud instances (like AWS Graviton), and Arm Virtual Hardware. -This setup allows you to simulate real-world, cross-node communication scenarios, making it ideal for evaluating Zenoh’s performance in robotics and industrial IoT applications. +In this session, you’ll use Raspberry Pi boards to simulate a scalable distributed environment. The same workflow applies to any Arm Linux system, including cloud instances and virtual hardware. This setup allows you to simulate real-world, cross-node communication scenarios, making it ideal for evaluating Zenoh’s performance in robotics and industrial IoT applications. This setup simulates real-world, cross-node communication scenarios, making it ideal for robotics and industrial IoT applications that require fast, decentralized messaging using pub/sub, storage, and query models. -### Install Docker on Raspberry Pi -To simplify this process and ensure consistency, you can use Docker to containerize your Zenoh and ROS 2 environment. -This lets you quickly replicate the same runtime on any device without needing to rebuild from source. +## Install Docker on Raspberry Pi +To simplify this process and ensure consistency, you can use Docker to containerize your Zenoh and ROS 2 environment. This lets you quickly replicate the same runtime on any device without needing to rebuild from source. -This enables multi-node testing and real-world distributed communication scenarios. +This enables scalable, multi-node testing in realistic distributed environments. -First, install Docker on each of Raspberry Pi. +First, install Docker on each Raspberry Pi device: ```bash curl -fsSL get.docker.com -o get-docker.sh && sh get-docker.sh sudo usermod -aG docker $USER ; newgrp docker ``` -### Create a ROS2 + Zenoh Docker Image +## Create a Docker image with ROS 2 and Zenoh To ensure compatibility with ROS-related tools, create a `Dockerfile` based on `ros:galactic `, and use the official Rust installation method to build Zenoh, as shown below. @@ -83,7 +79,8 @@ WORKDIR /root/zenoh/target/release CMD ["/bin/bash"] ``` -Under the directory where the above Dockerfile exists, run the following command to generate the docker image. +From the directory containing the above Dockerfile, run the following command to generate the docker image: + ```bash docker build -t zenoh-node . @@ -106,12 +103,16 @@ You can pull it directly using: ```bash docker pull odinlmshen/zenoh-node ``` +{{% notice Tip %}} +Once built, you can reuse this Docker image across multiple Arm-based nodes, including Raspberry Pi, AWS Graviton, and Arm Virtual Hardware. +{{% /notice %}} + +## Transfer the Docker image to another Raspberry Pi -### Transfer the Docker image to the other Raspberry Pi There are two options to transfer the Docker image to your second device. Choose one of the following methods. -Option 1: Save and copy via file +1. Save and copy using the file: ```bash docker save zenoh-node > zenoh-node.tar @@ -124,11 +125,11 @@ On the target device: docker load < zenoh-node.tar ``` -Option 2: Push the image to a container registry such as Docker Hub +2. Push the image to a container registry such as Docker Hub: You can also push the image to Docker Hub or GitHub Container Registry and pull it on the second device. -### Run the Docker Image +## Run the Docker image Once the image is successfully loaded on the second device, you can run the container to start the Zenoh environment. @@ -138,15 +139,17 @@ docker run -it --network=host zenoh-node The Zenoh example binaries are now available within this container, allowing you to test pub/sub and query flows across devices. -## Run Zenoh in Multi-Node Environment +## Run Zenoh examples in a multi-node environment + +With Zenoh running inside containers across devices, you’re now ready to explore real-time communication using prebuilt examples. -You’re now ready to run and test Zenoh communication flows across distributed edge devices. +The following examples are written in Rust and precompiled in your container image. They're fully interoperable and can be used to demonstrate Zenoh's key capabilities across devices. The Rust binaries are available in the `$ZENOH_LOC/target/release/examples/` directory. If you haven't set `ZENOH_LOC`, they can be found under `~/zenoh/target/release/examples/`. + -The source of the examples written in Rust will be provided, and both are interoperable. The -Rust binaries are already available under: `$ZENOH_LOC/target/release/examples/` directory. +The following sections illustrate the procedures to run the Zenoh examples so as to demonstrate the primary capabilities of Zenoh: +- Basic pub/sub – for real-time message distribution +- Query and storage – for persisting and retrieving historical data +- Queryable – for enabling on-demand remote computation +- Dynamic queryable with computation – for executing dynamic logic across nodes -The following sections illustrate the procedures to run the Zenoh examples so as to demonstrate the primary capabilities of Zenoh -1. Basic Pub/Sub – for real-time message distribution -2. Query and Storage – to persist and retrieving historical data -3. Queryable – to enable on-demand remote computation -4. Dynamic Queryable with Computation +With your Zenoh examples running across containers, you’re now ready to build and benchmark real-time communication flows across nodes, which is ideal for simulating robotic fleets or industrial systems. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/4_zenoh-ex1-pubsub.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/4_zenoh-ex1-pubsub.md index 6075c7cf3e..08c889a2ef 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/4_zenoh-ex1-pubsub.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/4_zenoh-ex1-pubsub.md @@ -1,41 +1,46 @@ --- -title: Zenoh Example-1 Simple Pub/Sub +title: Run a simple Zenoh pub/sub example weight: 5 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Example 1: Simple pub/sub +## Test Zenoh pub/sub across two devices -This first test demonstrates the real-time publish/subscribe model using two Raspberry Pi devices. +This example demonstrates how to use Zenoh’s real-time publish/subscribe model to exchange messages between two Raspberry Pi devices over a local network. -The following command is to initiate a subscriber for a key expression `demo/example/**`, a set of topics starting with the path `demo/example`. +Pub/sub is a core Zenoh communication pattern where data producers (publishers) send values to a key, and data consumers (subscribers) receive updates for matching key expressions. In this case, the subscriber listens for all messages published under the key expression `demo/example/**`, which matches any topic that begins with `demo/example/`. This enables flexible topic filtering and lightweight data dissemination across distributed systems. -### Step 1: Run subscriber +## Start the subscriber node -Run the subscriber example on one of the Raspberry Pi systems. +Run the subscriber example on one of the Raspberry Pi systems: ```bash cd ~/zenoh/target/release/examples ./z_sub ``` -### Step 2: Run publisher +## Start the publisher node -Then, log in to the other Raspberry Pi and run the publisher. +Then, log in to the other Raspberry Pi and run the publisher: ```bash cd ~/zenoh/target/release/examples ./z_pub ``` -The result is shown below: +{{% notice Tip %}} +You can run both `z_sub` and `z_pub` on the same device for testing, but running them on separate Raspberry Pis demonstrates Zenoh’s distributed discovery and cross-node communication. +{{% /notice %}} -![img1 alt-text#center](zenoh_ex1.gif "Figure 1: Simple Pub/Sub") +## Observe the pub/sub data flow -The left-side window shows the `z_sub` program. +The results are shown below: -It receives values with the key `demo/example/zenoh-rs-pub` continuously published by `z_pub` running in the right-side window. +![img1 Zenoh subscriber receiving messages from a publisher in a two-terminal view#center](zenoh_ex1.gif "Simple Pub/Sub") + +The left-side window shows the `z_sub` program. It receives values with the key `demo/example/zenoh-rs-pub` continuously published by `z_pub` running in the right-side window. + +This example confirms that Zenoh’s zero-configuration peer discovery and real-time pub/sub communication are working correctly across physical nodes. -This basic example shows Zenoh's zero-config discovery and low-latency pub/sub across physical nodes. diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/5_zenoh-ex2-storagequery.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/5_zenoh-ex2-storagequery.md index 4de6612a67..c7bb8846ab 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/5_zenoh-ex2-storagequery.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/5_zenoh-ex2-storagequery.md @@ -1,18 +1,23 @@ --- -title: Zenoh Example-2 Storage and Query +title: Run a Zenoh storage and query example + weight: 6 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Example 2: Storage and query +## Query historical data using Zenoh’s storage engine + +This example demonstrates Zenoh's data storage and query model which enables nodes to retrieve previously published values—even after the original publisher goes offline. -The second example adds Zenoh's data storage and querying capabilities by enabling nodes to retrieve historical values on demand. +Building on the previous pub/sub example, you’ll now run a lightweight Zenoh daemon that stores key–value pairs in memory. Then, you’ll publish data with `z_put` and retrieve it using `z_get`. -Building on the previous pub/sub example, you’ll now explore how Zenoh supports persistent data storage and on-demand querying, a powerful feature for robotics and IIoT applications. +This pattern is ideal for robotics and IIoT scenarios where devices intermittently connect or request snapshots of remote state. -In a typical warehouse or factory scenario, autonomous robots may periodically publish sensor data such as position, temperature, battery level, and a central system or another robot—may later need to query the latest state of each unit. +For example, in a warehouse or factory: +- Robots can periodically publish position, temperature, or battery level. +- A central system or peer node can later query these values on demand. Unlike Pub/Sub, which requires live, real-time message exchange, Zenoh's storage and query model enables asynchronous access to data that was published earlier, even if the original publisher is no longer online. @@ -20,7 +25,7 @@ In this example, you’ll run the `zenohd` daemon with in-memory storage and use This is especially useful for distributed systems where nodes may intermittently connect or request snapshots of state from peers. -### Step 1: Start the Zenoh daemon with in-memory storage +## Start the Zenoh daemon with in-memory storage On one Raspberry Pi, launch the Zenoh daemon with a configuration that enables in-memory storage for keys in the `demo/example/**` directory. @@ -35,7 +40,7 @@ You should see log messages indicating that the storage_manager plugin is loaded If port 7447 is already in use, either stop any previous Zenoh processes or configure a custom port using the `listen.endpoints.router` setting. -### Step 2: Publish data +## Publish a value On 2nd Raspberry Pi device, use `z_put` to send a key-value pair that will be handled by the `zenohd` storage. @@ -46,7 +51,7 @@ cd ~/zenoh/target/release/examples This command stores the string `Hello from storage!` under the key demo/example/test1. -### Step 3: Query the data +## Query the stored value Back on first Raspberry Pi, you can now query the stored data from any Zenoh connected node. @@ -66,9 +71,18 @@ The result is shown below: ![img2 alt-text#center](zenoh_ex2.gif "Figure 2: Storage and Query") -{{% notice tip %}} +{{% notice Tip %}} If you have more than two Raspberry Pi devices, you can run the `z_get` command on a third device to validate that storage queries work seamlessly across a multi-node setup. {{% /notice %}} This example shows how Zenoh's storage with query model supports asynchronous data access and resilient state-sharing—critical capabilities in robotics and industrial IoT systems where network connectivity may be intermittent or system components loosely coupled. +## What's next? + +Now that you've seen how Zenoh handles pub/sub and storage-based queries, you're ready to build reactive and intelligent edge nodes. + +In the next example, you’ll implement a **Zenoh queryable** that responds to runtime parameters,such as battery level and temperature, by computing and returning a real-time health score. This showcases how Zenoh supports on-demand edge computation without needing to pre-store data. + + + + diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/6_zenoh-ex3-queryable.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/6_zenoh-ex3-queryable.md index ed9d07ea5e..8bd96ce9c0 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/6_zenoh-ex3-queryable.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/6_zenoh-ex3-queryable.md @@ -1,24 +1,25 @@ --- -title: Zenoh Example-3 Computation on Query using Queryable +title: Run a Zenoh queryable node for on-demand edge computation + weight: 7 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Example 3: Computation on query +## Computation on query -Next, you’ll explore Zenoh's queryable capability, which lets a node dynamically respond to data queries by executing a custom computation or data generation function. +This example shows how to use Zenoh’s queryable feature to return computed results in real time, instead of serving pre-stored values. -Unlike `zenohd` which simply returns stored data, a queryable node can register to handle a specific key expression and generate responses at runtime. This is ideal for distributed computing at the edge, where lightweight devices, such as Raspberry Pi nodes, can respond to requests with calculated values. This enables sensor fusion, AI inference results, and diagnostics. +Unlike `zenohd` which simply returns stored data, a queryable node can register to handle a specific key expression and generate responses at runtime. This is ideal for distributed computing at the edge, where lightweight devices, such as Raspberry Pi nodes, can respond to requests with calculated values. This pattern is ideal for sensor fusion, lightweight AI inference, and dynamic system diagnostics on resource-constrained edge devices. -### Use Case: On-Demand battery health estimation +## Use case: estimate battery health on demand Imagine a robot fleet management system where the central planner queries each robot for its latest battery health score, which is not published continuously but calculated only when queried. -This saves bandwidth and enables edge compute optimization using Zenoh's Queryable. +This reduces bandwidth usage and enables edge-side optimization using Zenoh's queryable feature. -### Step 1: Launch a queryable node +## Launch a queryable node On one Raspberry Pi device, run the `z_queryable` Zenoh example to register a queryable handler. @@ -37,7 +38,7 @@ Press CTRL-C to quit... The node is now ready to accept queries on the key `demo/example/zenoh-rs-queryable` and respond with a predefined message. -### Step 2: Trigger a query from another node +## Trigger a query from another node On the other Raspberry Pi device, run the `z_get` example. @@ -56,11 +57,11 @@ Sending Query 'demo/example/zenoh-rs-queryable'... The result is shown below: -![img3 alt-text#center](zenoh_ex3.gif "Figure 3: Computation on Query using Queryable") +![img3 Zenoh queryable node responding to on-demand queries from a remote device#center](zenoh_ex3.gif "Figure 3: Computation on Query using Queryable") -The value you receive comes not from storage, but from the computation inside the queryable handler. +The response is generated dynamically by the queryable handler, and not fetched from stored data. -### Real-world application: distributed inference and computation +## Real-world use cases This model enables edge-based intelligence, such as: - Executing custom logic in response to a query such as “calculate load average” @@ -69,4 +70,7 @@ This model enables edge-based intelligence, such as: Queryable is a key feature for data-in-use scenarios, allowing fine-grained, on-demand compute inside your Zenoh-powered architecture. -Next, you’ll extend this Queryable pattern to perform parameterized computation, simulating edge diagnostics and adaptive inference. \ No newline at end of file +## What's next? + +In the next example, you'll extend this queryable pattern to support **runtime parameters**, such as battery level and temperature, allowing each node to return a calculated health score on demand. + diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/7_zenoh-querycomp.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/7_zenoh-querycomp.md index 1afd3c1103..c017ec8ca1 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/7_zenoh-querycomp.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/7_zenoh-querycomp.md @@ -1,5 +1,5 @@ --- -title: Zenoh Example-4 Dynamic Queryable with Computation +title: Run a Zenoh queryable with parameterized Rust computation weight: 8 ### FIXED, DO NOT MODIFY @@ -14,7 +14,7 @@ You’ll learn how to use Zenoh's Queryable API in Rust to build a parameterized This extends a previous example by supporting runtime query parameters like battery level and temperature. -## Use Case: Real-time battery health via computation +## Use case: real-time battery health through on-demand computation In robotic fleet management, a central controller may need to assess each robot’s battery health on demand. @@ -22,7 +22,7 @@ Instead of streaming data continuously, robots expose a queryable endpoint that This saves bandwidth and enables lightweight edge-side decision-making. -### Step 1: Create a new Zenoh rust project +## Create a new Zenoh rust project On any Raspberry Pi: @@ -40,7 +40,7 @@ tokio = { version = "1", features = ["full"] } url = "2" ``` -### Step 2: Implement the queryable node +## Implement the queryable node Next, log in to the other Raspberry Pi. @@ -103,8 +103,14 @@ Inside the callback: - A health score is computed from these inputs. - The result is sent back to the querying client using query.reply(). +{{% notice Tip %}} +You can extend this queryable pattern to respond to other real-time diagnostics, such as CPU load, camera snapshots, or local ML inference results. +{{% /notice %}} + This design pattern enables efficient, on-demand data exchange with minimal bandwidth usage. This is ideal for edge computing scenarios where resources and connectivity are constrained. +## Battery health estimation formula + The health score is calculated using the following logic: ```rust @@ -115,16 +121,15 @@ This formula estimates battery health as a percentage, considering both battery - battery: Current battery level (default 50%) - temp: Current temperature (default 25°C) -The health estimation logic begins with the battery level as the baseline score. -If the temperature rises above 25°C, the score is adjusted downward—specifically, for every 2°C above this threshold, the health is reduced by 1%. +This logic computes battery health as a percentage, adjusting for elevated temperatures. If temperature exceeds 25°C, the score is reduced by 1% for every 2°C increase. To ensure the calculation remains safe even when the temperature is below 25°C, the code uses saturating_sub(25), which prevents the result from becoming negative and avoids potential underflow errors. -For example, if battery = 88 and temp = 32, then: +For example, if `battery = 88` and `temp = 32`, then: - Temperature offset = (32 - 25) / 2 = 3 - Health = 88 - 3 = 85% -### Step 3: Build and run +## Build and run ```bash cd $HOME/zenoh/zenoh_battery_estimator @@ -138,7 +143,7 @@ After the build process, you will see: Finished `release` profile [optimized] target(s) in 1m 22s ``` -### Step 4: Query it with parameters +## Query it with parameters Run it on the Raspberry Pi you used for the build run: @@ -156,9 +161,9 @@ cd ~/zenoh/target/release/examples The result is shown below: -![img4 alt-text#center](zenoh_ex4.gif "Figure 4: Dynamic Queryable with Computation") +![img4 Estimated battery health query#center](zenoh_ex4.gif "Figure 4: Dynamic Queryable with Computation") -The excepted output is: +The expected output is: ```output >> Received ('robot/battery/estimate': 'Estimated battery health: 85%') diff --git a/content/learning-paths/cross-platform/zenoh-multinode-ros2/_index.md b/content/learning-paths/cross-platform/zenoh-multinode-ros2/_index.md index 833978a14d..39f75a71da 100644 --- a/content/learning-paths/cross-platform/zenoh-multinode-ros2/_index.md +++ b/content/learning-paths/cross-platform/zenoh-multinode-ros2/_index.md @@ -1,22 +1,19 @@ --- -title: Scalable Networking for Industrial and Robotics with Zenoh on Raspberry Pi - -draft: true -cascade: - draft: true +title: Build and deploy multi-node Zenoh systems on Raspberry Pi minutes_to_complete: 45 -who_is_this_for: This Learning Path is for robotics developers, industrial automation engineers, and IoT system architects building distributed, scalable, and low-latency applications. Whether you are using Robot Operating System (ROS), developing autonomous systems, or designing multi-node communication frameworks, you can use Eclipse Zenoh on Arm-based platforms, both in the cloud and on local devices like Raspberry Pi. +who_is_this_for: This Learning Path is for robotics developers, industrial automation engineers, and IoT system architects who are building distributed, scalable, and low-latency applications. Whether you're using the Robot Operating System (ROS), developing autonomous systems, or designing multi-node communication frameworks, you can use Eclipse Zenoh on Arm-based platforms, both in the cloud and on local devices like Raspberry Pi. + learning_objectives: - - Understand Zenoh's architecture and its integration of pub/sub, storage, querying, and computation models. - - Build and run Zenoh examples on both Arm servers and Raspberry Pi. - - Set up and deploy a multi-node Zenoh system. + - Understand Zenoh's architecture and how it integrates pub/sub, storage, querying, and computation models + - Build and run Zenoh examples on both Arm servers and Raspberry Pi + - Set up and deploy a multi-node Zenoh system prerequisites: - - At least two local Cortex-A devices running Linux, such as Raspberry Pi 4 or Pi 5. You can also use Arm servers or cloud instances. - - Experience with ROS 2 applications. + - At least two local Cortex-A devices running Linux, such as Raspberry Pi 4 or Pi 5. You can also use Arm servers or cloud instances + - Experience with ROS 2 applications author: - Odin Shen @@ -32,6 +29,8 @@ tools_software_languages: - ROS2 - C - Raspberry Pi + - Zenoh + - Rust operatingsystems: - Linux @@ -50,6 +49,11 @@ further_reading: title: Eclipse Zenoh Github link: https://github.com/eclipse-zenoh/zenoh type: documentation + - resource: + title: Zenoh and ROS 2 Integration Guide + link: https://github.com/eclipse-zenoh/zenoh-plugin-ros2 + type: documentation + ### FIXED, DO NOT MODIFY diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/2-env-setup.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/2-env-setup.md index ed0a605229..2787107f19 100644 --- a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/2-env-setup.md +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/2-env-setup.md @@ -50,7 +50,7 @@ Run the commands below to set up the ExecuTorch internal dependencies: ```bash git submodule sync -git submodule update --init +git submodule update --init --recursive ./install_executorch.sh ``` @@ -70,7 +70,7 @@ pip list | grep executorch ``` ```output -executorch 0.6.0a0+3eea1f1 +executorch 0.8.0a0+92fb0cc ``` ## Next Steps diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/3-env-setup-fvp.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/3-env-setup-fvp.md index 43662d7525..bc80217465 100644 --- a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/3-env-setup-fvp.md +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/3-env-setup-fvp.md @@ -18,7 +18,7 @@ The Corstone reference system is provided free of charge, although you will have {{% notice macOS %}} -Setting up FVPs on MacOS requires some extra steps, outlined in GitHub repo [VPs-on-Mac](https://github.com/Arm-Examples/FVPs-on-Mac/). macOS users must do this first, before setting up the Corestone-320 FVP. +Setting up FVPs on MacOS requires some extra steps, outlined in GitHub repo [FVPs-on-Mac](https://github.com/Arm-Examples/FVPs-on-Mac/). macOS users must do this first, before setting up the Corstone-320 FVP. {{% /notice %}} @@ -49,4 +49,4 @@ Test that the setup was successful by running the `run.sh` script for Ethos-U85, You will see a number of examples run on the FVP. -This confirms the installation, so you can now proceed to the Learning Path [Build a Simple PyTorch Model](/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/4-build-model/). \ No newline at end of file +This confirms the installation, so you can now proceed to the next section. diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/4-how-executorch-works.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/4-how-executorch-works.md index e10107404d..e2061aa1e2 100644 --- a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/4-how-executorch-works.md +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/4-how-executorch-works.md @@ -8,7 +8,8 @@ weight: 5 # 1 is first, 2 is second, etc. layout: "learningpathall" --- -Source: [How ExecuTorch Works](https://docs.pytorch.org/executorch/stable/intro-how-it-works.html) (official PyTorch Documentation) +To get a better understanding of [How ExecuTorch Works](https://docs.pytorch.org/executorch/stable/intro-how-it-works.html) refer to the official PyTorch Documentation. A summary is provided here for your reference: + 1. **Export the model:** * Generate a Graph * A graph is series of operators (ReLU, quantize, etc.) eligible for delegation to an accelerator diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/5-configure-fvp-gui.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/5-configure-fvp-gui.md index 7da0db60cf..e3902cafd4 100644 --- a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/5-configure-fvp-gui.md +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/5-configure-fvp-gui.md @@ -12,15 +12,9 @@ layout: "learningpathall" Note down your computer's IP address: ```bash -ip addr show +ip addr show ``` -To help you, here are some common WiFi interface names on Linux: -|Interface Name|Meaning / Context| -|--------------|-----------------| -|wlan0|Legacy name (common on older systems)| -|wlp2s0, wlp3s0|Predictable network naming scheme (modern systems)| -|wlx|Some systems name interfaces after MAC addresses| -|wifi0, ath0|Very rare, specific to certain drivers (e.g., Atheros)| +Note down the IP address of your active network interface (inet) which you will use later to pass as an argument to the FVP. {{% notice macOS %}} @@ -34,7 +28,7 @@ ipconfig getifaddr en0 # Returns your Mac's WiFi IP address ## Enable the FVP's GUI -Edit the following parameters in [run_fvp.sh](https://github.com/pytorch/executorch/blob/d5fe5faadb8a46375d925b18827493cd65ec84ce/backends/arm/scripts/run_fvp.sh#L97-L102), to enable the Mobilenet V2 output on the FVP's GUI: +Edit the following parameters in your locally checked out [executorch/backends/arm/scripts/run_fvp.sh](https://github.com/pytorch/executorch/blob/d5fe5faadb8a46375d925b18827493cd65ec84ce/backends/arm/scripts/run_fvp.sh#L97-L102) file, to enable the Mobilenet V2 output on the FVP's GUI: ```bash -C mps4_board.subsystem.ethosu.num_macs=${num_macs} \ diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/6-evaluate-output.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/6-evaluate-output.md index b6567d743f..2ab22dbdf2 100644 --- a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/6-evaluate-output.md +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/6-evaluate-output.md @@ -161,3 +161,5 @@ I [executorch:arm_perf_monitor.cpp:184] ethosu_pmu_cntr4 : 130 |ethosu_pmu_cntr2|External DRAM read beats(ETHOSU_PMU_EXT_RD_DATA_BEAT_RECEIVED)|Number of data beats read from off-chip memory (e.g., DRAM). Often 0 if Sram_Only is used.|If non-zero, may indicate cache misses or large model size.| |ethosu_pmu_cntr3|External DRAM write beats(ETHOSU_PMU_EXT_WR_DATA_BEAT_WRITTEN)|Number of write data beats to external memory.|Helps detect offloading or insufficient SRAM.| |ethosu_pmu_cntr4|Idle cycles(ETHOSU_PMU_NPU_IDLE)|Number of cycles where the NPU had no work scheduled (i.e., idle).|High idle count = possible pipeline stalls or bad scheduling.| + +In this learning path you have successfully learned how to deploy a MobileNet V2 Model using ExecuTorch on Arm's Corstone-320 FVP. diff --git a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_index.md b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_index.md index 71ce781eab..613a355ee5 100644 --- a/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_index.md +++ b/content/learning-paths/embedded-and-microcontrollers/visualizing-ethos-u-performance/_index.md @@ -7,11 +7,11 @@ cascade: minutes_to_complete: 120 -who_is_this_for: This is an introductory topic for developers and data scientists new to Tiny Machine Learning (TinyML), who want to visualize ExecuTorch performance on a virtual device. +who_is_this_for: This is an introductory topic for developers and data scientists new to Tiny Machine Learning (TinyML), who want to understand and visualize ExecuTorch performance on a virtual device. learning_objectives: - Identify suitable Arm-based devices for TinyML applications. - - Optionally, install Fixed Virtual Platforms (FVPs) on MacOS. + - Install Fixed Virtual Platforms (FVPs). - Deploy a TinyML ExecuTorch model to a Corstone-320 FVP. - Observe model execution on the FVP's graphical user interface (GUI). diff --git a/content/learning-paths/iot/_index.md b/content/learning-paths/iot/_index.md index 90e093d397..35b89dabc2 100644 --- a/content/learning-paths/iot/_index.md +++ b/content/learning-paths/iot/_index.md @@ -39,5 +39,7 @@ tools_software_languages_filter: - Raspberry Pi: 4 - Remote.It: 1 - ROS2: 1 +- Rust: 1 - VS Code: 1 +- Zenoh: 1 --- diff --git a/content/learning-paths/laptops-and-desktops/_index.md b/content/learning-paths/laptops-and-desktops/_index.md index 8f16771eb3..47c7b9b83f 100644 --- a/content/learning-paths/laptops-and-desktops/_index.md +++ b/content/learning-paths/laptops-and-desktops/_index.md @@ -34,8 +34,8 @@ tools_software_languages_filter: - C/C++: 4 - CCA: 1 - Clang: 11 -- CMake: 2 - cmake: 1 +- CMake: 2 - Coding: 16 - CSS: 1 - Daytona: 1 diff --git a/content/learning-paths/mobile-graphics-and-gaming/_index.md b/content/learning-paths/mobile-graphics-and-gaming/_index.md index 0ea7407037..2721c997a3 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/_index.md +++ b/content/learning-paths/mobile-graphics-and-gaming/_index.md @@ -10,14 +10,14 @@ key_ip: maintopic: true operatingsystems_filter: - Android: 31 -- Linux: 27 -- macOS: 12 -- Windows: 11 +- Linux: 28 +- macOS: 13 +- Windows: 12 subjects_filter: - Gaming: 6 - Graphics: 6 - ML: 10 -- Performance and Architecture: 31 +- Performance and Architecture: 32 subtitle: Optimize Android apps and build faster games using cutting-edge Arm tech title: Mobile, Graphics, and Gaming tools_software_languages_filter: @@ -29,7 +29,7 @@ tools_software_languages_filter: - Android Studio: 10 - Arm Development Studio: 1 - Arm Mobile Studio: 1 -- Arm Performance Studio: 2 +- Arm Performance Studio: 3 - assembly: 1 - Bazel: 1 - C: 2 @@ -44,6 +44,7 @@ tools_software_languages_filter: - Frame Advisor: 1 - GCC: 10 - GenAI: 2 +- Godot: 1 - GoogleTest: 1 - Hugging Face: 5 - Java: 6 diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/_index.md b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/_index.md new file mode 100644 index 0000000000..9b272c76ab --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/_index.md @@ -0,0 +1,62 @@ +--- +title: Use the Arm Performance Studio Integration extension in Godot + +draft: true +cascade: + draft: true + +minutes_to_complete: 15 + +who_is_this_for: This is an introductory topic for Godot developers who are targeting Android devices and want to get more insight into how their game performs on devices with Arm CPUs and GPUs. + +learning_objectives: + - Install the Arm Performance Studio Integration extension in Godot + - Annotate your Godot game with markers that give context to a profile in Arm Performance Studio tools + +prerequisites: + - Familiarity with Godot + - Familiarity with Arm Performance Studio tools + +author: Albin Bernhardsson, Julie Gaskin + +### Tags +skilllevels: Introductory +subjects: Performance and Architecture +armips: + - Cortex-A + - Mali +tools_software_languages: + - Godot + - Arm Performance Studio +operatingsystems: + - Windows + - macOS + - Linux + + +further_reading: + - resource: + title: Get started with Streamline + link: https://developer.arm.com/documentation/102477/latest/ + type: documentation + - resource: + title: Android performance triage with Streamline + link: https://developer.arm.com/documentation/102540/latest/ + type: documentation + - resource: + title: Get started with Performance Advisor + link: https://developer.arm.com/documentation/102478/latest/ + type: documentation + - resource: + title: Arm Performance Studio + link: https://developer.arm.com/Tools%20and%20Software/Arm%20Performance%20Studio + type: website + + + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/_next-steps.md b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/_next-steps.md new file mode 100644 index 0000000000..c3db0de5a2 --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/arm_mobile_studio_integrations.md b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/arm_mobile_studio_integrations.md new file mode 100644 index 0000000000..f345ac8828 --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/arm_mobile_studio_integrations.md @@ -0,0 +1,155 @@ +--- +title: Arm Performance Studio Godot integrations +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Arm Performance Studio Godot integrations + +[Arm Performance Studio](https://developer.arm.com/Tools%20and%20Software/Arm%20Performance%20Studio) is a free suite of analysis tools to help you profile game performance on mobile devices with Arm CPUs and GPUs. Arm provides a Godot extension to make data from [Godot games](https://godotengine.org/) visible in the Arm Performance Studio tools, [Streamline](https://developer.arm.com/Tools%20and%20Software/Streamline%20Performance%20Analyzer) and [Performance Advisor](https://developer.arm.com/Tools%20and%20Software/Performance%20Advisor). + +This package provides a simple way to incorporate annotations into your Godot project. These annotations enable you to mark the timeline with events or custom counters which provides valuable context alongside the performance data in Streamline, so you can see what was happening in the game when bottlenecks occur. For example, here you can see markers that highlight where a wave of enemies is spawning: + +![Marker annotations in Streamline](sl_annotation.png "Figure 1. Marker annotations in Streamline") + +{{% notice Note %}} +The Arm Performance Studio Integration extension is supported with Godot version 4.3 and later. +{{% /notice %}} + +### How to install the Arm Performance Studio Integration extension + +1. In Godot, click **AssetLib** to see the available extensions. + +2. Find the **Arm Performance Studio Integration** extension, then double-click to open the extension. + +3. The extension opens in a dialog box. Click **Download**. + +![Installing the Arm Performance Studio Integration extension in Godot](godot_install_performance_studio_extension.png "Figure 2. Installing the Arm Performance Studio Integration extension in Godot") + +4. A new dialog box opens where you can change the install folder if required. Click Install. + +### Using the extension + +All functionality in the extension is provided by the PerformanceStudio class, so first create an instance of it: + +```console +var performance_studio = PerformanceStudio.new() +``` + +### Adding single markers to a Godot project + +The simplest annotations are single markers, which can have a name and a color. To use them in a Godot project where you have installed this extension, simply call into the Performance Studio library as follows: + +```console +performance_studio.marker("Game Started") +``` + +This will emit a timestamped marker with the label "Game Started". When you capture a profile in Streamline, you can see this marker along the top of the timeline at the point that the game starts. + +![Marker annotation in Streamline](sl_marker.png "Figure 4. Marker annotation in Streamline") + +To give the annotation a color, use the `marker_color` method: + +```console +performance_studio.marker_color("Game Started", Color8(0, 255, 0)) +``` + +### Defining regions in a Godot project + +To define regions of interest within the game, you can specify a pair of markers prefixed with “Region Start” and “Region End”, for example: + +```console +performance_studio.marker("Region Start Times Square") +# Do work +performance_studio.marker("Region End Times Square") +``` + +These regions are shown on the frame rate analysis chart in the Performance Advisor report. + +![Regions in Performance Advisor](pa_frame_rate_regions.png "Figure 5. Regions in Performance Advisor") + +Also, dedicated charts for each region are appended to the end of the report, so you can analyze each region independently. + +![Dedicated region charts in Performance Advisor](pa_dedicated_region_charts.png "Figure 6. Dedicated region charts in Performance Advisor") + +### Using channels in a Godot project + +Channels are custom event timelines associated with a software thread. You can create channels and place annotations within them. A channel annotation has a text label and a color but, unlike markers, they span a range of time. + +To create a channel called "Spawner" and insert an annotation called "Spawning Wave", with the color red: + +```console +var channel : PerformanceStudio_Channel + +func _ready() -> void: + channel = performance_studio.create_channel("Spawner") + +# Annotations can then be inserted into a channel: +func _on_new_wave_started() -> void: + channel.annotate_color("Spawning Wave", Color8(255, 0, 0)) + +func _on_wave_completed() -> void: + channel.end() +``` + +To see channels in Streamline, select the **Core Map** view, and expand the **VkThread** thread: + +![Channel annotations in Streamline](sl_channel.png "Figure 7. Channel annotations in Streamline") + +### Creating counters + +Counters are numerical data points that can be plotted as a chart in the Streamline timeline view. Counters can be created as either absolute counters, where every value is an absolute value, or as a delta counter, where values are the number of instances of an event since the last value was emitted. All values are floats and will be presented to 2 decimal places. + +When charts are first defined, you can specify a title and series name. The title names the chart, the series names the data series. + +Multiple counter series can use the same title, which means that they will be plotted on the same chart in the Streamline timeline. + +To create a counter: + +```console +var counter = performance_studio.create_counter("Title", "Series", false) +``` + +Counter values are set easily as shown below: + +```console +counter.setValue(42.2) +``` + +### Custom Activity Maps + +[Custom Activity Map (CAM)](https://developer.arm.com/documentation/101816/latest/Annotate-your-code/User-space-annotations/Custom-Activity-Map-annotations) views allow execution information to be plotted on a hierarchical set of timelines. Like channel annotations, CAM views plot jobs on tracks, but unlike channel annotations, CAM views are not associated with a specific thread. Each CAM view contains one or more tracks and each track contains one or more jobs. + +![Custom activity maps in Streamline](sl_cam.png "Figure 8. Custom activity maps in Streamline") + +To create a custom activity map and add tracks to it: + +```console +var game_cam : PerformanceStudio_CAM +var wave_track : PerformanceStudio_CAMTrack +var ui_track : PerformanceStudio_CAMTrack + +func _ready() -> void: + # Create the CAM + game_cam = performance_studio.create_cam("Game Activity") + + # Add tracks to the CAM + wave_track = game_cam.create_track("Wave Activity") + ui_track = game_cam.create_track("UI Activity") +``` + +To create a job within a track: + +```console +var wave_job : PerformanceStudio_CAMJob + +func _on_new_wave_started() -> void: + wave_job = wave_track.create_job("Spawning Wave", Color8(255, 0, 0)) + +func _on_wave_completed() -> void: + wave_job.stop() +``` + +You can now annotate your Godot game and analyze the performance with markers that give context to a profile in Arm Performance Studio tools. diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/godot_install_performance_studio_extension.png b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/godot_install_performance_studio_extension.png new file mode 100644 index 0000000000..e78b7ca25b Binary files /dev/null and b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/godot_install_performance_studio_extension.png differ diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/pa_dedicated_region_charts.png b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/pa_dedicated_region_charts.png new file mode 100644 index 0000000000..a2e73ab3f2 Binary files /dev/null and b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/pa_dedicated_region_charts.png differ diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/pa_frame_rate_regions.png b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/pa_frame_rate_regions.png new file mode 100644 index 0000000000..4e694733b5 Binary files /dev/null and b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/pa_frame_rate_regions.png differ diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_annotation.png b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_annotation.png new file mode 100644 index 0000000000..41a08d4066 Binary files /dev/null and b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_annotation.png differ diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_cam.png b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_cam.png new file mode 100644 index 0000000000..b47aa4ab16 Binary files /dev/null and b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_cam.png differ diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_channel.png b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_channel.png new file mode 100644 index 0000000000..2c50d3ec74 Binary files /dev/null and b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_channel.png differ diff --git a/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_marker.png b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_marker.png new file mode 100644 index 0000000000..9980bc6ec0 Binary files /dev/null and b/content/learning-paths/mobile-graphics-and-gaming/godot_packages/sl_marker.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/_index.md b/content/learning-paths/servers-and-cloud-computing/_index.md index 5176ae893e..2cf5628cc6 100644 --- a/content/learning-paths/servers-and-cloud-computing/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/_index.md @@ -8,7 +8,7 @@ key_ip: maintopic: true operatingsystems_filter: - Android: 2 -- Linux: 153 +- Linux: 154 - macOS: 10 - Windows: 14 pinned_modules: @@ -19,7 +19,7 @@ pinned_modules: - migration subjects_filter: - CI-CD: 6 -- Containers and Virtualization: 28 +- Containers and Virtualization: 29 - Databases: 15 - Libraries: 9 - ML: 28 @@ -53,10 +53,10 @@ tools_software_languages_filter: - AWS Elastic Container Service (ECS): 1 - AWS Elastic Kubernetes Service (EKS): 3 - AWS Graviton: 1 -- Azure CLI: 1 +- Azure CLI: 2 - Azure Portal: 1 -- bash: 2 - Bash: 1 +- bash: 2 - Bastion: 3 - BOLT: 2 - bpftool: 1 @@ -99,7 +99,7 @@ tools_software_languages_filter: - GoogleTest: 1 - HammerDB: 1 - Herd7: 1 -- Hugging Face: 9 +- Hugging Face: 10 - InnoDB: 1 - Intrinsics: 1 - iPerf3: 1 @@ -112,7 +112,6 @@ tools_software_languages_filter: - Libamath: 1 - libbpf: 1 - Linaro Forge: 1 -- Linux: 1 - Litmus7: 1 - Llama.cpp: 1 - LLM: 9 @@ -125,8 +124,8 @@ tools_software_languages_filter: - MongoDB: 2 - mpi: 1 - MySQL: 9 -- Neon: 3 - NEON: 4 +- Neon: 3 - Nexmark: 1 - Nginx: 3 - Node.js: 3 @@ -141,6 +140,7 @@ tools_software_languages_filter: - PostgreSQL: 4 - Python: 28 - PyTorch: 9 +- QEMU: 1 - RAG: 1 - Redis: 3 - Remote.It: 2 @@ -180,6 +180,6 @@ weight: 1 cloud_service_providers_filter: - AWS: 17 - Google Cloud: 13 -- Microsoft Azure: 9 +- Microsoft Azure: 10 - Oracle: 2 --- diff --git a/content/learning-paths/servers-and-cloud-computing/azure-vm/_index.md b/content/learning-paths/servers-and-cloud-computing/azure-vm/_index.md new file mode 100644 index 0000000000..8a7c2f5afa --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/azure-vm/_index.md @@ -0,0 +1,64 @@ +--- +title: Create an Azure Linux 3.0 virtual machine with Cobalt 100 processors + +draft: true +cascade: + draft: true + +minutes_to_complete: 120 + +who_is_this_for: This Learning Path explains how to create a virtual machine on Azure running Azure Linux 3.0 on Cobalt 100 processors. + + +learning_objectives: + - Use QEMU to create a raw disk image, boot a VM using an Aarch64 ISO, install the OS, and convert the raw disk image to VHD format. + - Upload the VHD file to Azure and use the Azure Shared Image Gallery (SIG) to create a custom image. + - Use the Azure CLI to create an Azure Linux 3.0 VM for Arm, using the custom image from the Azure SIG. + +prerequisites: + - A [Microsoft Azure](https://azure.microsoft.com/) account with permission to create resources, including instances using Cobalt 100 processors. + - A local Linux machine with [QEMU](https://www.qemu.org/download/) and the [Azure CLI](/install-guides/azure-cli/) installed and authenticated. + +author: Jason Andrews + +### Tags +skilllevels: Advanced +subjects: Containers and Virtualization +cloud_service_providers: Microsoft Azure + +armips: + - Neoverse + +tools_software_languages: + - QEMU + - Azure CLI + +operatingsystems: + - Linux + +further_reading: + - resource: + title: Azure Virtual Machines documentation + link: https://learn.microsoft.com/en-us/azure/virtual-machines/ + type: documentation + - resource: + title: Azure Shared Image Gallery documentation + link: https://learn.microsoft.com/en-us/azure/virtual-machines/shared-image-galleries + type: documentation + - resource: + title: QEMU User Documentation + link: https://wiki.qemu.org/Documentation + type: documentation + - resource: + title: Upload a VHD to Azure and create an image + link: https://learn.microsoft.com/en-us/azure/virtual-machines/linux/upload-vhd + type: documentation + + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- + diff --git a/content/learning-paths/servers-and-cloud-computing/azure-vm/_next-steps.md b/content/learning-paths/servers-and-cloud-computing/azure-vm/_next-steps.md new file mode 100644 index 0000000000..c3db0de5a2 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/azure-vm/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/azure-vm/azure-vm.md b/content/learning-paths/servers-and-cloud-computing/azure-vm/azure-vm.md new file mode 100644 index 0000000000..f7c1a0e446 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/azure-vm/azure-vm.md @@ -0,0 +1,70 @@ +--- +title: Create an Azure Linux image for Arm +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +You can view the Azure Linux 3.0 project on [GitHub](https://github.com/microsoft/azurelinux). There are links to the ISO downloads in the project README. + +Using QEMU, you can create a raw disk image and boot a virtual machine with the ISO to install the OS onto the disk. + +Once the installation is complete, you can convert the raw disk to a fixed-size VHD, upload it to Azure Blob Storage, and then use the Azure CLI to create a custom Arm image. + +## Download and create a virtual disk file + +Use `wget` to download the Azure Linux ISO image file. + +```bash +wget https://aka.ms/azurelinux-3.0-aarch64.iso +``` + +Use `qemu-img` to create a 32 GB raw disk image. + +This step creates a 32 GB empty raw disk image to install the OS. You can increase the disk size by modifying the value passed to `qemu-img`. + +```bash +qemu-img create -f raw azurelinux-arm64.raw 34359738368 +``` + +## Boot and install the OS + +Use QEMU to boot the operating system in an emulated Arm VM. + +```bash +qemu-system-aarch64 \ + -machine virt \ + -cpu cortex-a72 \ + -m 4096 \ + -nographic \ + -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd \ + -drive if=none,file=azurelinux-arm64.raw,format=raw,id=hd0 \ + -device virtio-blk-device,drive=hd0 \ + -cdrom azurelinux-3.0-aarch64.iso \ + -netdev user,id=net0 \ + -device virtio-net-device,netdev=net0 +``` + +Once the OS boots successfully, install the Azure Linux Agent for VM provisioning, and power off the VM. + +```bash +sudo dnf install WALinuxAgent -y +sudo systemctl enable waagent +sudo systemctl start waagent +sudo poweroff +``` + +## Convert the raw disk to VHD Format + +Now that the raw disk image is ready to be used, convert the image to fixed-size VHD, making it compatible with Azure. + +```bash +qemu-img convert -f raw -o subformat=fixed,force_size -O vpc azurelinux-arm64.raw azurelinux-arm64.vhd +``` + +{{% notice Note %}} +VHD files have 512 bytes of footer attached at the end. The `force_size` flag ensures that the exact virtual size specified is used for the final VHD file. Without this, QEMU may round the size or adjust for footer overhead (especially when converting from raw to VHD). The `force_size` flag forces the final image to match the original size. This flag helps make the final VHD size a clean, whole number in MB or GiB, which is required for Azure. +{{% /notice %}} + +Next, you can save the image in your Azure account. diff --git a/content/learning-paths/servers-and-cloud-computing/azure-vm/background.md b/content/learning-paths/servers-and-cloud-computing/azure-vm/background.md new file mode 100644 index 0000000000..dda13eaf11 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/azure-vm/background.md @@ -0,0 +1,46 @@ +--- +title: "About Azure Linux" + +weight: 2 + +layout: "learningpathall" +--- + +## What is Azure Linux 3.0? + +Azure Linux 3.0 is a Linux distribution developed and maintained by Microsoft, specifically designed for use on the Azure cloud platform. It is optimized for running cloud-native workloads, such as containers, microservices, and Kubernetes clusters, and emphasizes performance, security, and reliability. Azure Linux 3.0 provides native support for the Arm (AArch64) architecture, enabling efficient, scalable, and cost-effective deployments on Arm-based infrastructure within Azure. + +Currently, Azure Linux 3.0 is not available as a ready-made virtual machine image for Arm-based VMs in the Azure Marketplace. Only x86_64 images, published by Ntegral Inc., are offered. This means you cannot directly create an Azure Linux 3.0 VM for Arm from the Azure portal or CLI. + +However, you can still run Azure Linux 3.0 on Arm-based Azure VMs by creating your own disk image. Using QEMU, an open-source machine emulator and virtualizer, you can build a custom Azure Linux 3.0 Arm image locally. After building the image, you can upload it to your Azure account as a managed disk or custom image. This process allows you to deploy and manage Azure Linux 3.0 VMs on Arm infrastructure, even before official images are available. + +This Learning Path guides you through the steps to build an Azure Linux 3.0 disk image with QEMU, upload it to Azure, and prepare it for use in creating virtual machines. + +Following this process, you'll be able to create and run Azure Linux 3.0 VMs on Arm-based Azure infrastructure. + +To get started install the dependencies on your local Linux machine. The instructions work for both Arm or x86 running Ubuntu. + +```bash +sudo apt update && sudo apt install qemu-system-arm qemu-system-aarch64 qemu-efi-aarch64 qemu-utils ovmf -y +``` + +You also need to install the Azure CLI. Refer to [How to install the Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest). You can also use the [Azure CLI install guide](/install-guides/azure-cli/) for Arm Linux systems. + +Make sure the CLI is working by running the version command and confirm the version is printed. + +```bash +az version +``` + +You should see an output similar to: + +```output +{ + "azure-cli": "2.61.0", + "azure-cli-core": "2.61.0", + "azure-cli-telemetry": "1.1.0", + "extensions": {} +} +``` + +Continue to learn how to prepare the Azure Linux disk image. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/azure-vm/save-image.md b/content/learning-paths/servers-and-cloud-computing/azure-vm/save-image.md new file mode 100644 index 0000000000..a9f52f461b --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/azure-vm/save-image.md @@ -0,0 +1,128 @@ +--- +title: Transfer the image to Azure +weight: 4 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +You can now use the Azure CLI to create a disk image in Azure and copy the local image to Azure. + +## Prepare Azure resources for the image + +Before uploading the VHD file to Azure storage, set the environment variables for the Azure CLI. + +```bash +RESOURCE_GROUP="MyCustomARM64Group" +LOCATION="centralindia" +STORAGE_ACCOUNT="mycustomarm64storage" +CONTAINER_NAME="mycustomarm64container" +VHD_NAME="azurelinux-arm64.vhd" +GALLERY_NAME="MyCustomARM64Gallery" +IMAGE_DEF_NAME="MyAzureLinuxARM64Def" +IMAGE_VERSION="1.0.0" +PUBLISHER="custom" +OFFER="custom-offer" +SKU="custom-sku" +OS_TYPE="Linux" +ARCHITECTURE="Arm64" +HYPERV_GEN="V2" +STORAGE_ACCOUNT_TYPE="Standard_LRS" +VM_NAME="MyAzureLinuxARMVM" +ADMIN_USER="azureuser" +VM_SIZE="Standard_D4ps_v6" +``` + +{{% notice Note %}} +You can modify the environment variables such as RESOURCE_GROUP, VM_NAME, and LOCATION based on your naming preferences, region, and resource requirements. +{{% /notice %}} + +Create a new resource group. If you are using an existing resource group for the RESOURCE_GROUP environment variable you can skip this step. + +```bash +az group create --name "$RESOURCE_GROUP" --location "$LOCATION" +``` + +Create Azure blob storage. + +```bash +az storage account create \ + --name "$STORAGE_ACCOUNT" \ + --resource-group "$RESOURCE_GROUP" \ + --location "$LOCATION" \ + --sku Standard_LRS \ + --kind StorageV2 +``` + +Create a blob container in the blob storage account. + +```bash +az storage container create \ + --name "$CONTAINER_NAME" \ + --account-name "$STORAGE_ACCOUNT" +``` + +## Upload and save the image in Azure + +Upload the VHD file to Azure. + +```bash +az storage blob upload \ + --account-name "$STORAGE_ACCOUNT" \ + --container-name "$CONTAINER_NAME" \ + --name "$VHD_NAME" \ + --file ./azurelinux-arm64.vhd +``` + +You can now use the Azure console to see the image in your Azure account. + +Next, create a custom VM image from this VHD, using Azure Shared Image Gallery (SIG). + +```bash +az sig create \ + --resource-group "$RESOURCE_GROUP" \ + --gallery-name "$GALLERY_NAME" \ + --location "$LOCATION" +``` + +Create the image definition. + +```bash +az sig image-definition create + --resource-group "$RESOURCE_GROUP" + --gallery-name "$GALLERY_NAME" + --gallery-image-definition "$IMAGE_DEF_NAME" + --publisher "$PUBLISHER" + --offer "$OFFER" + --sku "$SKU" + --os-type "$OS_TYPE" + --architecture "$ARCHITECTURE" + --hyper-v-generation "$HYPERV_GEN" +``` + +Create the image version to register the VHD as a version of the custom image. + +```bash +az sig image-version create + --resource-group "$RESOURCE_GROUP" + --gallery-name "$GALLERY_NAME" + --gallery-image-definition "$IMAGE_DEF_NAME" + --gallery-image-version "$IMAGE_VERSION" + --location "$LOCATION" + --os-vhd-uri "[https://${STORAGE_ACCOUNT}.blob.core.windows.net/${CONTAINER_NAME}/${VHD_NAME](https://${storage_account}.blob.core.windows.net/$%7BCONTAINER_NAME%7D/$%7BVHD_NAME)}" + --os-vhd-storage-account "$STORAGE_ACCOUNT" + --storage-account-type "$STORAGE_ACCOUNT_TYPE" +``` + +Once the image has been versioned, you can retrieve the unique image ID for use in VM creation. + +```bash +IMAGE_ID=$(az sig image-version show + --resource-group "$RESOURCE_GROUP" + --gallery-name "$GALLERY_NAME" + --gallery-image-definition "$IMAGE_DEF_NAME" + --gallery-image-version "$IMAGE_VERSION" + --query "id" -o tsv) +``` + +Next, you can create a virtual machine with the new image using the image ID. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/azure-vm/start-vm.md b/content/learning-paths/servers-and-cloud-computing/azure-vm/start-vm.md new file mode 100644 index 0000000000..cb28839882 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/azure-vm/start-vm.md @@ -0,0 +1,41 @@ +--- +title: Start an Azure virtual machine with the new image +weight: 5 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Create a virtual machine using the new image + +You can now use the newly created Azure Linux image to create a virtual machine in Azure with Cobalt 100 processors. Confirm the VM is created by looking in your Azure account in the “Virtual Machines” section. + +```bash +az vm create \ + --resource-group "$RESOURCE_GROUP" \ + --name "$VM_NAME" \ + --image "$IMAGE_ID" \ + --size "$VM_SIZE" \ + --admin-username "$ADMIN_USER" \ + --generate-ssh-keys \ + --public-ip-sku Standard +``` + +After the VM is successfully created, retrieve the public IP address. + +```bash +az vm show \ + --resource-group "$RESOURCE_GROUP" \ + --name "$VM_NAME" \ + --show-details \ + --query "publicIps" \ + -o tsv +``` + +Use the public IP address to SSH to the VM. Replace `` with the IP returned by the previous command. + +```bash +ssh azureuser@ +``` + +You are ready to use your Azure Linux virtual machine. diff --git a/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/build-2.md b/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/build-2.md index 94a5ae8661..12e48a2332 100644 --- a/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/build-2.md +++ b/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/build-2.md @@ -21,10 +21,11 @@ Perform a build inside the container: ``` During the build you will see that TF-A, UEFI and SCP firmware are built using their own build systems. The build finishes with the following output: + ```output -output/bin/grub-mkimage: info: kernel_img=0x7f366cf65010, kernel_size=0x1a000. -output/bin/grub-mkimage: info: the core size is 0xa2a98. -output/bin/grub-mkimage: info: writing 0xa5000 bytes. +output/bin/grub-mkimage: info: kernel_img=0xf0a19d2c2010, kernel_size=0x1a000. +output/bin/grub-mkimage: info: the core size is 0xa3040. +output/bin/grub-mkimage: info: writing 0xa6000 bytes. Execute build for build-grub.sh on rdn2[rdn2][busybox] done. ----------------------------------- *********************************** diff --git a/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/environment-setup-1.md b/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/environment-setup-1.md index 0228a2b61e..4be732c0ec 100644 --- a/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/environment-setup-1.md +++ b/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/environment-setup-1.md @@ -39,28 +39,30 @@ repo version The output looks like this: ```output +/usr/bin/repo:681: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). + now = datetime.datetime.utcnow() -repo launcher version 2.17 +repo launcher version 2.36 (from /usr/bin/repo) -git 2.34.1 -Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] -OS Linux 6.2.0-1009-aws (#9~22.04.3-Ubuntu SMP Tue Aug 1 21:11:51 UTC 2023) -CPU x86_64 (x86_64) -Bug reports: https://bugs.chromium.org/p/gerrit/issues/entry?template=Repo+tool+issue +git 2.43.0 +Python 3.12.3 (main, Jun 18 2025, 17:59:45) [GCC 13.3.0] +OS Linux 6.8.0-1031-aws (#33-Ubuntu SMP Fri Jun 20 17:58:11 UTC 2025) +CPU aarch64 (aarch64) +Bug reports: https://issues.gerritcodereview.com/issues/new?component=1370071 ``` ### Fetch source code Create a new directory in to which you can download the source code, build the stack, and then obtain the manifest file. -To obtain the manifest, choose a tag of the platform reference firmware. [RD-INFRA-2023.09.29](https://neoverse-reference-design.docs.arm.com/en/latest/releases/RD-INFRA-2023.09.29/release_note.html) is used here, although it is recommended to use the latest version available. See the [release notes](https://neoverse-reference-design.docs.arm.com/en/latest/) for more information. +To obtain the manifest, choose a tag of the platform reference firmware. [RD-INFRA-2025.07.03](https://neoverse-reference-design.docs.arm.com/en/latest/releases/RD-INFRA-2025.07.03/release_note.html) is used here, although it is recommended to use the latest version available. See the [release notes](https://neoverse-reference-design.docs.arm.com/en/latest/) for more information. Specify the platform you would like with the manifest. In the [manifest repo](https://git.gitlab.arm.com/infra-solutions/reference-design/infra-refdesign-manifests) there are a number of available platforms. In this case, select `pinned-rdn2.xml`. ```bash mkdir rd-infra cd rd-infra/ -repo init -u https://git.gitlab.arm.com/infra-solutions/reference-design/infra-refdesign-manifests.git -m pinned-rdn2.xml -b refs/tags/RD-INFRA-2023.12.22 +repo init -u https://git.gitlab.arm.com/infra-solutions/reference-design/infra-refdesign-manifests.git -m pinned-rdn2.xml -b refs/tags/RD-INFRA-2025.07.03 ``` Now look at what the configured manifest contains: @@ -104,24 +106,24 @@ The contents of `pinned-rdn2.xml` are shown below: - - - - - - - - - - + + + + + + + + + + - - - + + + - + ``` @@ -133,18 +135,26 @@ repo sync -c -j $(nproc) --fetch-submodules --force-sync --no-clone-bundle ``` The output from running this command looks like: ```output -... A new version of repo (2.40) is available. -... New version is available at: /home/ubuntu/rd-infra/.repo/repo/repo + +... A new version of repo (2.54) is available. +... New version is available at: /home/ubuntu/rdn2-infra/.repo/repo/repo ... The launcher is run from: /usr/bin/repo !!! The launcher is not writable. Please talk to your sysadmin or distro !!! to get an update installed. -Fetching: 100% (17/17), done in 2m23.399s -Fetching: 100% (16/16), done in 26.300s -Fetching: 100% (8/8), done in 11.914s -Fetching: 100% (1/1), done in 0.592s -Updating files: 100% (79368/79368), done.testsUpdating files: 26% (21084/79368) -Checking out: 100% (42/42), done in 13.164s +Fetching: 100% (17/17), done in 8m16.653s +Fetching: 100% (17/17), done in 17.218s +Fetching: 100% (17/17), done in 14.719s +Fetching: 100% (7/7), done in 12.526s +Fetching: 100% (1/1), done in 0.385s +Updating files: 100% (84978/84978), done.ting files: 17% (14646/84978) +Checking out: 27% (16/59), done in 8.804s +Checking out: 28% (17/59), done in 0.779s +Updating files: 100% (18459/18459), done.ptoPkg/Library/OpensslLib/openssl/pyca-cryptographyUpdating files: 42% (10441/24326) +Updating files: 100% (24326/24326), done.ptoPkg/Library/OpensslLib/openssl/fuzz/corporaUpdating files: 54% (13137/24326) +Checking out: 30% (18/59), done in 1.675s +Checking out: 11% (7/59), done in 0.874s +Checking out: 1% (1/59), done in 0.004s repo sync has finished successfully. ``` Now you should have all the code. @@ -186,13 +196,19 @@ docker image list The output from this command looks like: ```output REPOSITORY TAG IMAGE ID CREATED SIZE -rdinfra-builder latest 8729adb0b96c 8 minutes ago 3.07GB +rdinfra-builder latest e11c8e27a4c6 7 days ago 8.12GB ubuntu jammy-20230624 5a81c4b8502e 6 months ago 77.8MB ``` The appearance of the output is like a standard ubuntu container based on the latest release with the rdinfra-builder container built on top. Let's run and enter the container: ```bash -docker run -it rdinfra-builder:latest /bin/bash +docker run --rm \ + -v /home/ubuntu/rd-infra/:/home/ubuntu/rd-infra/ \ + -w /home/ubuntu/rd-infra/ \ + -e ARCADE_USER=$(id -un) \ + -e ARCADE_UID=$(id -u) \ + -e ARCADE_GID=$(id -g) \ + -it rdinfra-builder bash ``` This command puts you in the running container where you can run `ls` to list the contents: diff --git a/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/test-with-fvp-3.md b/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/test-with-fvp-3.md index f8c0ae9b2d..d8467f5081 100644 --- a/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/test-with-fvp-3.md +++ b/content/learning-paths/servers-and-cloud-computing/refinfra-quick-start/test-with-fvp-3.md @@ -15,7 +15,8 @@ The firmware build can be executed on the Neoverse N2 Reference Design FVP that Download the FVP from the previous page, or directly with: ```bash -wget https://developer.arm.com/-/media/Arm%20Developer%20Community/Downloads/OSS/FVP/Neoverse-N2/Neoverse-N2-11-24-12/FVP_RD_N2_11.24_12_Linux64.tgz +wget https://developer.arm.com/-/cdn-downloads/permalink/FVPs-Neoverse-Infrastructure/RD-N2/FVP_RD_N2_11.25_23_Linux64.tgz + ``` Unpack the tarball and run the install script: