Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


The RealAnt robot platform from Ote Robotics is designed for real-world reinforcement learning research and development. It is a complete solution with a web camera based tracking system. It is aimed to be a low-cost starting point for anybody interested in bringing reinforcement learning to practical real robots.


This repository provides 3D models and build instructions for making your own RealAnt, firmware code for the robot microcontroller, and Python code for the robot interface and ArUco tag based pose estimation.

If you want to get the robot ready assembled, you can buy them from

For an example implementation of reinforcement learning that learns to stand, turn and walking, see

Getting Started

You can get to reinforcement learning in the real world with five steps:

  1. Get or build your own RealAnt.
  2. Calibrate and test the web camera for pose estimation.
  3. Setup testing scene with adequate lighting.
  4. Start Python Server scripts.
  5. Start experiments.

Get Your RealAnt

If you want to get the robot ready assembled, you can buy them from

To build your own, you need to obtain 8 Robotis Dynamixel AX-12A's and one Robotis OpenCM9.04A board + accessory set or Ote Robotics RealAnt Main Board v1.2 with Arduino Nano 33 IoT (see board_1.2 folder for PCB gerbers, BOM, layout and schematics PDFs).

For building, you need additionally some two-wire cable for power, a soldering iron, side cutters, a Phillips screwdriver and thread-locking fluid (such as Loctite blue).

The 3D model files are under stl folder. You need to print two body torso plates and four leg assemblies.

For PrusaSlicer, use 20% gyroid infill, 0.2mm layer height, no top and bottom layers.

RealAnt Main Board Firmware

For the OpenCM9.04A board, install Arduino and setup Robotis OpenCM9.04 board support, and then upload the OpenCM9.04 firmware from ant11_cmd_dxl folder.

For the Ote Robotics RealAnt Main Board v1.2 with Arduino Nano 33 IoT, install Arduino and use firmware from ant14_cmd_dxl_nano33iot folder. You also need to install the forked DynamixelSDK from this folder into Arduino libraries.

Pose Estimation Calibration and Testing

Print calibration chessboard from markers on e.g. A4 and glue them to a flat surface (e.g. piece of cardboard). Print the calibration chessboard at 100% scale, this produces 4cm chessboard pattern. (If you use a different scaling, change script's marker size accordingly.) Clipping at the edges of the outermost squares doesn't matter, as only the inner corners of the squares are used.

Capture various poses of the chessboard (15+ images) with and then run Move the position of the chessboard around, so that it would cover the whole field of view of the camera. Tilting the chessboard also can yield better calibration. Make sure your chessboard poses are free from motion blur, delete the ones that are blurry.

After this, run to obtain cam_calib.pkl file.

# python ../
# python ../ 

After calibrating the camera, adjust such that it uses the correct cam_calib.pkl file.

Print and attach reference and moving agent markers to floor and the ant, adjust their sizes in the script and run the script to estimate distance. Make sure to use correct camera calibration file.

# python

Setup The Test Scene

Make sure you have adequate lighting so that there is no motion blur in pose estimation during robot movement.

For example, 2x 50W LED floodlights rated at 3800 lumens mounted 1 m above the robot should yield around 3000 lux illuminance at the robot level, which is enough for reinforcement learning purposes with Logitech Brio 4K USB camera.

Python Interface Server

For starting trials, start the following scripts in their own terminal windows:

# python         # zmq pub-sub communication proxy
# python       # server for serial communication to the physical ant
# python  # pose estimation

You might need to adjust for the correct serial port device. It defaults to /dev/ttyACM0 on Linux.

Test scripts:

# python     # send simple test movements to ant

Reinforcement Learning

For an example implementation of reinforcement learning that learns to stand, turn and walking, see

Clone that repository, and then run the scripts:

#             # random exploration & agent evaluation
# --task walk   # train for walking

Further Interface Implementation Details

The main components are

  • (the main pub-sub network proxy, this should work as-is, as it just receives and forwards packages as a message hub — just run this on the background),
  • (this sends orientation and location info) and
  • (this processes the s1...s8 servo commands, and sends joint position values and (optional) foot sensors).

The pub-sub network protocol is very simple. Each message is a list of byte strings. The beginning of the first string is the "topic" in ZMQ terms (used with zmq.SUBSCRIBE to filter in messages, "" receives everything):

  • "cmd" is a command that the receives. This takes a multipart string argument that is forwarded to the firmware. Currently supported commands are "sN X sM Y ..." where N,M = servo number in [1,8] and X,Y = angle setpoint in [224...800], 512 = middle, and "reset", "attach_servos" and "detach_servos".
  • "{" which is json data, which is used both for all measurements (and recorded setpoints) from and also for measurement packets.

RealAnt Revision History

Version Description
RealAnt v1 First version with FP04-2 hip joints and OpenCM9.04 microcontroller board
RealAnt v1.1 Hip joints replaced with 3D printed ones, OpenCM9.04 microcontroller board
RealAnt v1.2 Microcontroller board changed to Arduino Nano 33 IoT based one

Copyright and License

Unless otherwise noted, all source code, documentation and data in this repository is Copyright (c) 2020 Ote Robotics Ltd and is licensed under MIT license.

See LICENSE for details.


RealAnt robot platform for low-cost, real-world reinforcement learning








No releases published


No packages published