ros 2 robot operating system

ROS 2 (Robot Operating System)

ROS 2 is a framework designed to help developers build and manage robotic applications. Despite its name, it is not a real operating system like Windows or Linux. Instead, it acts as a communication bridge that allows all the different parts of a robot-like cameras, motors, and AI brains, to talk to each other easily. It provides pre-built libraries and tools so you don't have to write everything from scratch.

In short, ROS 2 is a powerful system that connects a robot's hardware and software together, making it easier to build smart and complex machines.

To make working with ROS 2 even easier, this project uses Vulcanexus. Vulcanexus is a specialized, "all-in-one" distribution of ROS 2 created by eProsima. Normally, setting up ROS 2 can be complicated because you have to manually install and configure many different communication tools. Vulcanexus solves this by coming pre-packaged with the best communication engine (called Fast DDS) and essential development tools ready to use right out of the box.

In short, Vulcanexus is a ready-to-use version of ROS 2 that makes setting up, developing, and running robotic applications faster and much more stable.

What is a ROS 2 Node?

Imagine you are building a robot with LEGO bricks. In the world of ROS 2 (Robot Operating System), each "brick" is a Node.

One node might control the wheels.
One node might read a sensor.
Our node handles the "Eyes" (the camera).

Nodes talk to each other by sending messages on Topics (like radio channels). This project is designed to be one "brick" in a larger system, providing vision and intelligence to your robot.

Shapes blocks

In shape we have many blocks for ROS2:

Topic blocks:
- Subscribe
- Pubblish
- List
Messages blocks:
- Message type
- Message String
- Message Bool
- Message Int64
- Message Float64
Node blocks:
- Command
- Node Ready
- List

The On Topic Block (Subscribe)

The On topic block is an "Event Listener." It waits for a specific message to arrive from the ROS 2 network before triggering an action.

How does it works:

Topic & Type: You define the "channel" (Topic) to listen to and the message format (Type).
Payload: When a message arrives, the block stores all its data into a dictionary variable called the payload (JSON).
Extraction: You use the "Get item" block to pick specific values from the payload and save them into variables for your program.

Example Exercise: Controlling a Robotic Arm

To show the true power of this system, we created a shape macro that controls a robotic arm.

The goal of this exercise is to make the robot move objects between a right and left position, controlled entirely by your finger. The robot operates using a simple State Machine based on a variable called holding

The arm starts in an empty state (holding = false).
We point at a marker (id0 = Right, id1 = Left) to tell the robot where to pick up an object.
Once picked up, the robot switches to the full state (holding = true).
We point at a marker again (0 or 1) to tell the robot where to drop the object.
After dropping it, the state goes back to empty (holding = false), and the cycle repeats.

Looking at the full shape, we can divide the program into three main blocks:

1. Setup Block (Initialization)

When the program starts, this block prepares the robot. It opens the gripper, moves the arm to a safe "home" position (x: 0, y: 10, z: 5), closes the gripper, and sets the starting state: holding = false.

2. The ROS Listener Block (On topic)

This block is the bridge between our AI Camera Node and the Robot.

It constantly listens to the /camera_tracking/pointing topic.
When you point at a marker, it receives the message payload, which should be something like this:

   {
       'id': 0,
       'p1': camera_tracking_msgs.msg.Point2D(x=218, y=199),
       'p2': camera_tracking_msgs.msg.Point2D(x=172, y=201),
       'p3': camera_tracking_msgs.msg.Point2D(x=174, y=156),
       'p4': camera_tracking_msgs.msg.Point2D(x=220, y=156)
   }

It extracts the id of the marker (e.g., 0 or 1) and saves it into a variable called marker_pointed_id.

3. The Main Loop Block (Action)

This block runs continuously and executes actions based on the holding state and the marker_pointed_id.

Phase A - Picking up (if not holding):
If the hand is empty and the user points at Marker 0, the robot moves to the right coordinates (-200, 50, 0), grabs the object, and returns to the home position. It then sets holding= true. (The same happens for Marker 1 on the left, with the opposite X coordinates).
Phase B - Dropping (else if holding):
If the hand is holding something and the user points at Marker 0, the robot moves to the right, opens the gripper to drop the object, and goes back home. It then sets holding = false. (The same happens for Marker 1 on the left, with the opposite X coordinates).

Note:
After every pickup or drop action, the program sets marker_pointed_id to 999. This is a "dummy" value used to reset the memory. It prevents the robot from getting stuck in an infinite loop of repeating the same action.

Here's a quick video of the exercise working:

Braccio_camera_tracking.mp4

###The Publish Block

The Publish block is the tool that allows your program to "talk" to the rest of the robotic world.

Topic (Channel): Think of a topic as a radio station (e.g., /test). The block broadcasts on that frequency. Anyone "tuned in" to that specific name will receive the information.
Message Type: This defines what kind of data we are sending. In this exercise, we use a String, which is simply a piece of text.
Data (Content): This is the actual message being sent (e.g., "Hello World!").

1. ROS 2 Network is Ready?

Function: This block checks if the ROS 2 network has successfully discovered external entities (like a camera container or a robot driver). It distinguishes between internal system entities and external ones.
Return Value: Boolean (True or False).
Logic & Fail-safe:
- It returns False as long as only the default internal entities (e.g., ShapeNode, /rosout) are detected.
- It returns True as soon as at least one external node or topic is discovered.
- Timeout: If no external entities are found after 5 seconds, the block will force a True result. This is a safety feature to prevent the program from getting stuck in an infinite loop if a sensor is offline.
Usage: Place this block inside a While loop at the beginning of your Setup block. This ensures your script waits for the hardware to connect before proceeding.

2. List ROS 2 Topics

Function: This block queries the background cache and returns a list of all currently active communication channels and their respective message types.
Return Value: Array of Arrays [[Topic1, TypeA], [Topic2, TypeB], ...]
Real-time Updates: Because the discovery process runs continuously in the background, this list updates automatically. If a new node starts publishing while your program is running, the new topic will appear in the list almost instantly.
Usage: Use this to verify that sensors or nodes are successfully broadcasting data.

3. List ROS 2 Nodes

Function: This block retrieves the names of all active processes (nodes) currently connected to the ROS 2 graph.
Return Value: Array of Strings (e.g., ['CameraTrackingNode', 'web_video_server', 'ShapeNode']).
Real-time Updates: Just like with the Topics block, the discovery block runs in the background, updating the list of active nodes immediately.
Usage: Use this to confirm that your custom nodes or external drivers have fully joined the network.

Shapes nodes

Shapes comes boundled with a few nodes:

Camera tracking node
Example node

Camera Tracking Node

This project is an AI-powered vision system. It combines two powerful technologies:

MediaPipe AI: It recognizes your hand and calculates the exact position of your fingers in 3D space.
ArUco Markers: These are special square patterns (like QR codes) that the camera can identify.

So by combining these two things, when you point your index finger at a specific Marker, the system understands what you are pointing at and sends this information to a specific topic.

To make this project more user-friendly, we added a Web Stream feature. Inside the Tactigon Shapes web page, you will find a Camera Icon. Clicking it opens a page where you can choose which live feed to watch. One is /image_raw which, as the name says, will shows the raw, untouched, feed from the camera. The other live feed combines the raw feed with the marker and index detection, showing colored dots on your hand and green boxes around the detected markers. This is very useful to position the camera and check if the marker or fingers get detected.

Prerequisites

Operating System: Native Ubuntu is recommended. However, it is fully compatible with Windows using WSL2 (Windows Subsystem for Linux), provided you share the USB camera correctly.
Docker & Docker Compose: To run everything in isolated containers.
Hardware: A standard USB webcam or integrated laptop camera.

(For WSL users only: You must use the usbipd-win tool in PowerShell to pass your physical camera from Windows to the WSL environment, for more info about this, check the Prerequisites page in the Tactigon Shapes wiki).

Custom Messages (`msgs/camera_tracking_msgs`)

Standard ROS 2 doesn't have a message type for "Markers". So, we created our own package to describe exactly what the AI sees:

Point2D: Represents a single coordinate on the screen.
- int32 x
- int32 y
Marker: Represents a detected ArUco square.
- uint32 id (The number inside the ArUco)
- Point2D p1, p2, p3, p4 (The 4 corners of the square)
MarkerList:
- Marker[] markers (An array of all visible markers)

The Node (`nodes/camera_tracking`)

The core of the logic lives here. To build this robust system, we used:

OpenCV (cv2): For image processing and drawing the debug elements.
ROS 2 usb_cam driver: Chosen over standard drivers because of its excellent support for compressed MJPEG streams, which ensures widespread compatibility and smooth framerates even on WSL/Virtual Machines.
Google MediaPipe (Tasks API): The latest AI engine for hand tracking. We specifically configured the Dockerfile to use Python 3.12 with a locked numpy < 2.0.0 environment to guarantee stability with MediaPipe's dependencies.

Configuration

You can customize the node's behavior using the camera_tracking.json file.

{
    "debug": true,
    "camera_topic": "/image_raw",
    "tracking_topic": "/camera_tracking/image",
    "marker_topic": "/camera_tracking/markers",
    "pointing_topic": "/camera_tracking/pointing",
    "marker_definition": "ARUCO_4X4_100",
    "marker_detection_margin": 50
}

With this configuration you can easily remap the inputs/outputs without touching the code.

Topics: Think of these as the "names of the radio stations" where the data is being broadcast.

Warning: Changing the camera_topic from "/image_raw" will cause errors.
Marker Definition: Defines the types of markers or QR codes being used.
Margin (50): Adds a virtual "padding" around the marker. It makes it easier to trigger the interaction even if your finger is slightly outside the physical printed border.

ROS Topics (Inputs and Outputs)

Topic Name	Type	Direction	Description
`/image_raw`	`sensor_msgs/Image`	INPUT	The raw video from the webcam.
`/camera_tracking/markers`	`MarkerList`	OUTPUT	List of all visible markers and their coordinates.
`/camera_tracking/pointing`	`Marker`	OUTPUT	Emitted ONLY when the user's index finger is pointing inside a marker area.
`/camera_tracking/image`	`sensor_msgs/Image`	OUTPUT	The visual debug stream (used by the Web UI).

Tactigon-Shapes ROS2 Message Integration

To allow the main application to understand our new custom messages, we updated the central registry in tactigon-shape/modules/ros2/models.py:

# We added our custom types to the system
CameraTrackingMessageTypes = Point2D | Marker | MarkerList

# The main list now includes them
RosMessageTypes = StrMessageTypes | InterfacesMessageTypes | CameraTrackingMessageTypes 

def get_message_type_by_name(name: str) -> RosMessageTypes:
    type_index =[t.__name__ for t in RosMessageTypes.__args__].index(name)    
    return RosMessageTypes.__args__[type_index]

Why did we do this? The web interface talks to the Python backend using strings (text). This function acts as a translator: if the web UI requests to listen to "Marker", this code translates the string into the actual Python Class, allowing the ROS 2 system to subscribe to our custom topics correctly.

How to run Ros2 environment

We have 2 main ways:

Docker
Local installation

1. Docker (Quick & Easy)

To see the messages being sent by your blocks, open a terminal on your PC and enter this command:

docker exec -it tactigon-shapes bash

You can skip the Full Installation and go to the environment activation.

2. Full Guide & Installation (Longer but works everywhere on the PC)

If you want to replicate this environment on your local PC without using Docker, you need to install Vulcanexus Jazzy (based on ROS 2).

Official Guide Link: Vulcanexus Installation (if you use a different version like Kilted or Humble remember to replace the name during the installation)

Activate the Vulcanexus environment

Once you installed Vulcanexus you'll need to Activate ROS2 and subscibe to the topic:

Activate ROS 2:

source /opt/vulcanexus/jazzy/setup.bash

If you have to listen to custom messages (like with camera_tracking) you must build your packages with colcon and source the message definition:

# Example
colcon build --packages-select camera_tracking_msgs

source install/setup.bash

Start Listening (Subscribe):

ros2 topic echo /test

Now you're listening to the /test topic, run the exercise on Shapes to see the messages appear in your terminal.

Adding Nodes to Shapes

This section provides a comprehensive overview of creating, configuring, and deploying custom ROS 2 nodes within the Tactigon-Shapes ecosystem using Docker.

Package Folder Structure

In ROS 2 (Python), the directory structure is strictly defined. To create a package named example_node, follow this exact layout:

Tactigon-Shapes/
┣━ 📁 ros/
┃  ┣━ 📁 msgs/                  # Custom ROS 2 message definitions
┃  ┗━ 📁 nodes/                 # ROS 2 Nodes source code
┃     ┗━ 📁 camera_tracking
┃     ┗━ 📁 example_node/
┃        ┣━ 📁 example_node/    # Python package file
┃        ┃  ┣━ 📄 __init__.py
┃        ┃  ┣━ 📄 main.py
┃        ┃  ┣━ 📄 models.py
┃        ┃  ┗━ 📄 node.py
┃        ┣━ 📁 resource/
┃        ┃  ┗━ 📄 example_node  # Empty file (marker for the colcon build system)
┃        ┣━ 📄 config.json      # Config file
┃        ┣━ 📄 CMakeLists.txt   # CMake file to build the Ros2 node
┃        ┣━ 📄 Dockerfile
┃        ┣━ 📄 entrypoint.sh    # File to run commands inside of Docker
┃        ┣━ 📄 package.xml      # Metadata and dependencies
┃        ┣━ 📄 setup.cfg        # Configuration for script executables
┃        ┗━ 📄 setup.py         # Installation instructions for Python/ROS
┗━ ... (other tactigon-shape files)

You can already see an example in this project of the example node's code, ready to go

Docker compose modifications

To run the new node in it's container you must add it to the docker-compose.yml file:

  example-node:
      build:
        context: ./ros
        dockerfile: nodes/example_node/Dockerfile
      container_name: example_node_container
      restart: unless-stopped
      environment:
        - ROS_DOMAIN_ID=0

ros 2 robot operating system

ROS 2 (Robot Operating System)

What is a ROS 2 Node?

Shapes blocks

The On Topic Block (Subscribe)

How does it works:

Example Exercise: Controlling a Robotic Arm

1. Setup Block (Initialization)

2. The ROS Listener Block (On topic)

3. The Main Loop Block (Action)

1. ROS 2 Network is Ready?

2. List ROS 2 Topics

3. List ROS 2 Nodes

Shapes nodes

Camera Tracking Node

Prerequisites

Custom Messages (msgs/camera_tracking_msgs)

The Node (nodes/camera_tracking)

Configuration

ROS Topics (Inputs and Outputs)

Tactigon-Shapes ROS2 Message Integration

How to run Ros2 environment

1. Docker (Quick & Easy)

2. Full Guide & Installation (Longer but works everywhere on the PC)

Activate the Vulcanexus environment

Adding Nodes to Shapes

Package Folder Structure

You can already see an example in this project of the example node's code, ready to go

Docker compose modifications

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

Custom Messages (`msgs/camera_tracking_msgs`)

The Node (`nodes/camera_tracking`)