EMOS v0.7.6

Full Changelog: v0.7.5...v0.7.6

EMOS v0.7.0 — The Agentic Release

This is the biggest EMOS release to date. It brings the entire stack forward a full generation — a brand-new agentic planning layer, graph-backed spatio-temporal memory, live system visualization, process-level self-healing, and a GPU-accelerated navigation engine — alongside a matured, zero-touch CLI and dashboard.

Under the hood, the whole stack moves together: EmbodiedAgents 0.7.3, Kompass 0.5.0, Sugarcoat 0.7.0, and kompass-core 0.8.1.

---

⭐ Headline features

Cortex — an agentic harness for robots

The marquee addition. Cortex is a planner-executor component you drop on top of the rest of your recipe, and it turns the whole graph into a self-directed agent. It auto-discovers every other component's actions, exposes them to an LLM as callable tools, and runs a two-phase loop: it plans a task into ordered steps, then executes them — dispatching navigation goals, calling a VLM, speaking through TTS — watching feedback and replanning on failure. No event wiring, no hand-tuned prompts, no orchestration glue. Give it a goal in plain language and it figures out the rest.

📖 Cortex reference · Cortex agent tutorial · Cortex driving the full stack

Memory — spatio-temporal graph memory

A new Memory component (built on eMEM) gives robots persistent, queryable recall. Every detection, scene caption, and interoception reading folds into a graph indexed simultaneously by meaning, location, and time. Episodes consolidate into long-term gists; the same object is recognized across sessions; the whole state is a single file that survives reboots. Perception layers and internal-state (interoception) layers are distinguished via is_internal_state, and Cortex consumes Memory's retrieval tools during planning. (Replaces the deprecated MapEncoding component.)

📖 Memory reference · Memory + Cortex tutorial · Spatio-temporal memory recipe

Live System Graph in the dashboard

The Dynamic Web UI now renders a draggable, resizable node-graph of your entire running recipe — components as nodes, topics/events/actions as edges, with a click-through event-detail card. See exactly how your robot is wired, live, while it runs.

📖 Visualizing the System Graph · Dynamic Web UI

Self-healing robots

Two new layers of resilience: process-level auto-respawn via Launcher.on_process_fail() (the launcher relaunches a component whose process actually crashes) and the safe_restart() context manager for atomic in-place reconfigure-and-restart. Layered under the existing component-level fallback system, this keeps long-running deployments alive through model-server outages, segfaults, and OS hiccups with nobody in the loop.

📖 Status & fallbacks · Multiprocessing & fault tolerance

Events from plain Python conditions

You can now build event triggers from any Python method, polled at a configurable check_rate, with no topic or publisher required. Want the robot to react when an internal-state condition goes true — overheating, idle too long, battery low and far from base? It's just a function that returns a bool.

📖 Internal-state events · Events & actions

---

🧠 Intelligence layer (EmbodiedAgents 0.7.3)

• Component actions as LLM tools — components expose @component_action/@component_fallback methods that Cortex (or any LLM component) can call as tools, executed through service calls with outputs captured for subsequent steps.
• VLM actions — the Vision component gains describe and track actions; a new VisionLanguageAction message carries feedback.
• Async action monitoring — action clients run and are monitored on the main action loop, with helpers to watch and cancel in-flight goals — the foundation for Cortex's long-horizon tasks.
• System-tool registration — components automatically register their additional ROS entrypoints as tools, with auto-generated descriptions.
• VLA refinements — the Vision-Language-Action component (shipped in the previous cycle) gained a clearer action name and locking of unimplemented methods.
• Numerous fixes: STT empty-buffer inference, TTS say abrupt-stop, LLM string-arg tool calling, topic validation now compares ROS 2 message types, callback disambiguation on layer topics.

📖 Intelligence overview · AI components

🧭 Navigation (Kompass 0.5.0)

A big release for navigation, spanning a new controller, GPU-accelerated perception, and tighter integration with the intelligence layer:

• Pure Pursuit controller — a new full path follower with obstacle avoidance and curvature-based speed regulation, alongside the existing DWA planner.
• GPU-accelerated local mapping — the pointcloud → laserscan → occupancy-grid pipeline now runs entirely on-device; the 100k-point mapper benchmark clocks ~0.57 ms on an RTX A5000.
• Smarter DWA — an adaptive prediction horizon that shortens lookahead on tightly curved paths, and an arc-length-based goal cost (CPU and GPU) for more sensible path tracking.
• Richer goal sources — the Planner now accepts Detections, Trackings, and PointsOfInterest as goal-point inputs (multiple topics allowed), so an intelligence component can hand navigation a structured target instead of a raw pose. Backed by a new on-device depth detector that does 2D→3D bounding-box estimation with outlier rejection.
• Vision-following refactor — RGB and RGBD followers are now co-equal sibling controllers (ControllersID.VISION_IMG / VISION_DEPTH) selected by algorithm, with the depth follower cleanly separated into a pure depth-tracking controller and new correctness tests.
• Component introspection — components now describe their available actions, which is what lets Cortex discover and drive the navigation stack.
• Cleaner architecture under the hood — modularized controller lifecycle/status/obstacle logic, GPU kernel optimizations (trajectory/obstacle tiling, atomics replaced by sub-group reductions, a direct-lidar critical-zone checker), split robot-state/sensor-data updates, quaternion goal-orientation fix, and Rolling/Jazzy compatibility.

⚠️ Breaking: the kompass.launcher module was removed — Launcher now comes from kompass.ros (re-exporting the unified Sugarcoat launcher).

📖 Planning · Vision tracking with depth

🏗️ Foundation layer (Sugarcoat 0.7.0)

• System Graph visualization (see above), action-server logs streamed into the main logging card, and a geometry display element that auto-renders Point/Pose values when a recipe has no map.
• Process auto-respawn and safe_restart() (see above).
• Generic events from Python methods (see above).
• execute_method service now returns the called method's response — this is what makes Cortex's component-action calls observable.
• executor_spin_timeout config decouples callback-blocking time from loop_rate.
• Component introspection + JSON tool descriptions; health monitoring now on for all components by default.

---

🛠️ EMOS CLI & zero-touch dashboard (0.7.0)

• Four install modes — container (no ROS needed), native (builds into your ROS 2), licensed (private deployment image), and the new pixi mode (root-less, Docker-less, any Linux distro).
• Zero-touch dashboard (emos serve) — a browser console with pairing-code + QR onboarding, mDNS discovery (http://emos.local:8765), optional self-signed TLS with fingerprint verification, and emos serve install-service to run it at boot via a hardened systemd unit.
• emos uninstall — a proper, mode-aware inverse of install, with --keep-data / --keep-config / --remove-image and a path-safety guard.
• Mode-aware emos update — self-updates the CLI binary from GitHub releases, then updates the install per its mode (pull image / rebuild native / refresh pixi workspace).
• Recipe lifecycle — emos recipes, emos pull, emos ls, emos info (auto-extracts sensor/topic requirements from recipe.py), emos run (with --rmw and --skip-sensor-check); recipes you drop in ~/emos/recipes/ appear in the dashboard automatically.
• Device & pairing management — emos config with rotate-pairing (hot-reloads the running daemon), tokens, revoke-token, tls-fingerprint/tls-regenerate, and reset.
• Mapping tools — emos map record / install-editor / edit.

📖 Installation · CLI reference · Dashboard · Running recipes

🐛 Nota...

EMOS v0.6.1

EMOS v0.6.1 — pixi Install Mode & Documentation

pixi Install Mode (Experimental)

EMOS can now be installed entirely in userspace using pixi — no root, no Docker, no pre-installed ROS2. Works on any Linux distro (amd64/arm64), including Ubuntu 22.04 where Jazzy isn't officially supported.

curl -fsSL https://pixi.sh/install.sh | bash
git clone --recurse-submodules https://github.com/automatika-robotics/emos.git
cd emos && pixi install && pixi run setup

ROS2 Jazzy and all dependencies are pulled as pre-built packages from RoboStack and conda-forge. The CLI (emos status, emos run, emos update) fully supports the new mode.

Documentation

• Local Models tutorial — new recipe page showing how to run LLM, VLM, STT, and TTS entirely on-device with enable_local_model=True
• Hardware setup guide — per-mode instructions (Container/Native/Pixi) for installing and verifying sensor drivers
• Troubleshooting page — common errors (sensor timeouts, import errors, Zenoh port conflicts, plugin issues) with diagnosis and fixes
• Expanded emos run docs — pipeline explanation, --skip-sensor-check guidance, pre-flight checklist
• Local model fallback docs — updated fallback recipes with the new fallback_to_local() API
• Built-in Local Models section in models reference

Fixes

• Fixed pip build isolation issue for Ubuntu < 24.04
• CI workflow for pixi install (Ubuntu 22.04 + 24.04)

Links

• Docs: emos.automatikarobotics.com
• Discord: discord.gg/B9ZU6qjzND

EMOS v0.6.0

EMOS v0.6.0 — Built-in Local Models & Native Install Hardening

EMOS v0.6.0 focuses on offline resilience and installation reliability. Robots can now fall back to built-in local models when remote servers are unavailable — no cloud access, no external dependencies, just one line of code. The native installer and CLI have also been substantially hardened.

---

Built-in Local Models

EmbodiedAgents now ships with lightweight models that run directly on-device. No model server required.

Component	Local Model	Framework	Default Checkpoint
LLM	LocalLLM	llama-cpp-python	Qwen3 0.6B (GGUF)
VLM	LocalVLM	llama-cpp-python	Moondream2 (GGUF)
SpeechToText	LocalSTT	sherpa-onnx	Whisper tiny.en
TextToSpeech	LocalTTS	sherpa-onnx	Kokoro EN
Vision	LocalVision	onnxruntime	DEIM detector

Models are auto-downloaded from HuggingFace on first use. GPU-accelerated variants available for llama-cpp-python and onnxruntime.

Zero-Config Fallback

Two lines turn any remote-backed component into a self-healing one:

switch_to_local = Action(method=llm_component.fallback_to_local)
llm_component.on_component_fail(action=switch_to_local, max_retries=3)

If the cloud API drops, the component automatically switches to its built-in local model and keeps running. Works for LLM, VLM, STT, and TTS components.

Alternatively, enable local models directly in config with enable_local_model=True, or point to a custom checkpoint via local_model_path.

CLI Self-Update

emos update now checks for newer CLI releases on GitHub and replaces its own binary before updating the stack. No more re-running the install script to get CLI fixes.

Native Install Improvements

The native installer (emos install --mode native) received significant hardening:

• Two-phase colcon build matching the Docker build pattern — localization dependencies first, then EMOS packages
• Submodule handling — proper --recurse-submodules on clone and submodule update on pull
• Package validation — verifies package.xml exists before attempting build
• kompass-core GPU install integrated into the native flow

Enhanced emos status

Native mode status now performs deep health checks:

• Verifies Python package imports (ros_sugar, agents, kompass, kompass_core) inside the ROS environment
• Lists installed ROS packages (automatika_ros_sugar, automatika_embodied_agents, kompass, kompass_interfaces)
• Clear per-package pass/fail output

Docker Images

Local model dependencies (llama-cpp-python, sherpa-onnx, onnxruntime, huggingface-hub) are now pre-installed in all EMOS Docker images. Multi-arch (amd64 + arm64) for Humble, Jazzy, and Kilted:

docker pull ghcr.io/automatika-robotics/emos:jazzy-latest

CI

New native-install.yml workflow tests the full native installation pipeline across Humble, Jazzy, and Kilted on every push to stack/.

Stack Updates

• EmbodiedAgents: Local model support (LLM, VLM, STT, TTS, Vision), think-token stripping, developer docs
• Kompass: v0.4.1, developer docs reorganization

Links

• Documentation: emos.automatikarobotics.com
• Quick Start: Installation Guide
• Discord: discord.gg/B9ZU6qjzND

EMOS v0.5.0

EMOS v0.5.0 — The Embodied Operating System

EMOS is the missing operating system for Physical AI. It's the open-source layer that turns any robot — quadrupeds, humanoids, mobile platforms — into an intelligent agent that can see, think, move, and adapt. All from a single Python script.

Today we're releasing EMOS as a unified open-source platform for the first time.

---

The Problem

Building intelligent robots today means stitching together a dozen frameworks: one for perception, another for navigation, a third for manipulation, plus launch files, lifecycle management, failure recovery, and deployment tooling. Each piece speaks a different dialect. And when something fails at 2 AM on a security patrol, your robot just... stops.

The EMOS Approach

Write a Recipe. Deploy it on any robot. No code changes.

A Recipe is a pure Python script that describes a complete robot behavior — perception, reasoning, navigation, manipulation — wired together through a declarative component graph:

from agents.components import VLM, SpeechToText, TextToSpeech
from agents.clients.ollama import OllamaClient
from agents.models import OllamaModel, Whisper, SpeechT5
from agents.ros import Topic, Launcher

# A conversational robot that listens, sees, and speaks
audio_in = Topic(name="audio0", msg_type="Audio")
image_in = Topic(name="image_raw", msg_type="Image")
text_out = Topic(name="text1", msg_type="String")

stt = SpeechToText(inputs=[audio_in], outputs=[query], model_client=whisper_client, trigger=audio_in)
vlm = VLM(inputs=[query, image_in], outputs=[text_out], model_client=ollama_client, trigger=query)
tts = TextToSpeech(inputs=[text_out], outputs=[audio_out], model_client=tts_client, trigger=text_out)

launcher = Launcher()
launcher.add_pkg(components=[stt, vlm, tts])
launcher.bringup()

Under the hood, each component runs as a managed ROS2 lifecycle node with health monitoring, automatic fallbacks, and event-driven reconfiguration. If the cloud API goes down, the system switches to a local model. If the navigation controller gets stuck, an event fires a recovery maneuver. Failure is a control flow state, not a crash.

What's Inside

EMOS unifies three battle-tested open-source frameworks into a single stack:

Layer	Package	Highlights
Intelligence	EmbodiedAgents	LLMs, VLMs, VLAs, speech-to-text, text-to-speech, vision, semantic memory, semantic routing, tool calling
Navigation	Kompass	GPU-accelerated planning & control (up to 3,106x faster than CPU). Cross-vendor GPU support via SYCL — runs on NVIDIA, AMD, Intel
Foundation	Sugarcoat	Lifecycle-managed components, parallel event engine with microsecond reaction times, Pythonic launch API that replaces XML

Each of these has been shipping independently for months. This release brings them together under one roof with a unified CLI and documentation.

The CLI

One binary. Full lifecycle management.

# Install
curl -sSL https://raw.githubusercontent.com/automatika-robotics/emos/main/stack/emos-cli/scripts/install.sh | sudo bash

# Set up EMOS (container mode — no ROS needed, or native mode for full integration)
emos install

# Discover, inspect, and run recipes
emos recipes
emos info vision_follower    # AST-based sensor introspection — NEW in v0.5.0
emos run vision_follower

Two deployment modes:

• Container — Docker-based, no ROS 2 required. Pull and run in minutes.
• Native — Builds and installs directly into /opt/ros/{distro}/. After setup, run recipes with just python3 recipe.py.

20+ Ready-to-Use Recipes

The documentation includes complete, working recipes for:

• Conversational agents with speech I/O
• Visual question answering with prompt engineering
• Semantic routing for multi-capability agents
• Spatio-temporal semantic mapping
• Tool calling and function execution
• Vision-Language-Action (VLA) manipulation
• Point-to-point navigation with GPU-accelerated planning
• Vision-based target following (RGB and RGBD)
• Runtime model fallbacks and self-healing agents
• Event-driven cognition loops
• ...and more

Every recipe runs as-is. No glue code. No launch files.

Auto-Generated Web UI

Every recipe automatically gets a fully functional web dashboard — real-time telemetry, video feeds, component settings, and controls. Zero frontend code required.

AI-Friendly

EMOS publishes an llms.txt covering the full documentation. Feed it to Claude, GPT, or your preferred coding assistant and have it write Recipes for you.

Get Started

curl -sSL https://raw.githubusercontent.com/automatika-robotics/emos/main/stack/emos-cli/scripts/install.sh | sudo bash
emos install

• Documentation: emos.automatikarobotics.com
• Quick Start: emos.automatikarobotics.com/getting-started/quickstart
• GitHub: github.com/automatika-robotics/emos
• Discord: discord.gg/B9ZU6qjzND

Docker Images

Multi-arch (amd64 + arm64) for Humble, Jazzy, and Kilted:

docker pull ghcr.io/automatika-robotics/emos:jazzy-latest

License

MIT. Built in collaboration between Automatika Robotics and Inria. Contributions welcome.