Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

TORSO-21 Dataset: Typical Objects in RoboCup Soccer 2021

This repository contains the scripts and addtional information for the TORSO-21 Dataset. This is a dataset for the RoboCup Humanoid Soccer domain consisting of images of the Humanoid League as well as the Standard Platform League. We provide two image collections. The first one consists of images from various real-world locations, recorded by different robots. It includes annotations for the ball, goalposts, robots, lines, field edge, and three types of line intersections. The second collection is generated in the Webots simulator which is used for the official RoboCup Virtual Humanoid Soccer Competition. Additionally to the labels of the first collection, labels for the complete goal, depth images, 6D poses for all labels, as well as the camera location in the field of play, are provided.

Meta Data

# of real-world Images 10464
# of Balls 10959
# of Robots 14383
# of Goalposts 12780
# of L-Intersections 15458
# of T-Intersection 13479
# of X-Intersections 13447
# of Field Segmentations 10464
# of Line Segmentations 10464

# of simulated images: 24.000

Example images




With annotations marked


With segmentations of lines and field




With annotations, segmentation mask and depth image


Download Dataset and Labels

Manual Download

The images and annotations can be downloaded here:

Automated Download

The data can also be downloaded with the following script (use --help for further options):

./scripts/ --all

YOLO Labelform

If you want to train a YOLO, you can use the script provided in this repository to generate the labels.


The repository structure is as follows:

├── data                          # contains the annotations and images
│   ├── reality                   # the images recorded in reality
│   │   ├── train                 # the training set
│   │   │   ├── annotations.yaml  # the annotations in yaml format
│   │   │   ├── images/           # a folder containing all the images of the training set
│   │   │   └── segmentations/    # a folder containing all the segmentation masks of the training set
│   │   └── test                  # the test set
│   │       └── ...               # it is structured in the same way as the training set
│   └── simulation                # the images recorded in simulation
│       ├── train                 # the training set
│       │   ├── annotations.yaml  # the annotations in yaml format
│       │   └── depth/            # a folder containing all the depth images of the training set
│       │   ├── images/           # a folder containing all the images of the training set
│       │   └── segmentations/    # a folder containing all the segmentation masks of the training set
│       └── test                  # the test set
│           └── ...               # it is structured in the same way as the training set
└── scripts                       # some useful scripts, see below for details
    └── ...

The annotations are in the following format:

    width: 1920
    height: 1080
      - blurred: true
        concealed: true
        in_image: true
        type: robot
        - - 42 # x value
          - 26 # y value
        - - 81
          - 98
        pose: # The pose of the annotated object, only available in simulation
            x: 0
            y: 0
            z: 0
            x: 0
            y: 0
            z: 0
            w: 0
          motion: standing
      - in_image: false
        type: ball
    metadata: # The keys should be like this but do not need to be present for all images
      fov: 42
      location: "foobay"
      tags: ["natural_light", "telstar18", "do_not_use"]
      imageset_id: 130
      camera_pose: # The pose of the annotated object, only available in simulation
          x: 0
          y: 0
          z: 0
          x: 0
          y: 0
          z: 0
          w: 0
      Natural light: False
      League: HSL


Set up environment

Follow these instructions to set up the dependencies for the scripts used for visualization and creation of the dataset.

# Clone the repository
git clone
cd TORSO_21_dataset

# Install poetry
pip3 install poetry --user

# Install dependencies
poetry install --no-dev

# Optionally install dependencies including those for the dataset creation
poetry install


To run the tools you need to source the poetry environment in your shell.

# Source the virtualenv
poetry shell

Instead, you can also use poetry run ./scripts/... to run the scripts without sourcing.

Visualize annotations

To visualize the annotations, run the following two commands to pickle and show the annotations in the poetry environment:

./scripts/ data/reality/train/annotations.yaml
./scripts/ data/reality/train/annotations.pkl

Statistics and checks

Generates metadata statistics from an annotations file, i.e. how often which metadata type occurs.

This script is used to generate statistics about the annotations, i.e. how often each annotation occurs per image. Its first argument is the annotation file to generate annotations for.

Sanity-checks the annotations, i.e. checks if some labels are marked as in image and not in image and if the field boundary is contained.

YOLO Evaluation

Simple script that runs a YOLO model against the test dataset and calculates the IOU metrics.

./scripts/ --yolo-path /path/to/yolo_folder --collection data/reality/test

Further scripts

To use these scripts, make sure to install all dependencies with poetry install.

This script downloads multiple image sets and annotations from the ImageTagger. The imagesets and the annotation format are defined at the top of the file. Its output is a folder data_raw in the root of this repository that contains all image files. To avoid conflicting names, every filename is prepended with its dataset id. Additionally, a file annotations.yaml is created that contains a dict mapping set ids to their metadata and a dict mapping image names to their labels.

This is just a verbatim copy of the ImageTagger download script. Its API is used by, it it not necessary to use this script directly.

This script filters the annotations contained in data_raw/annotations.yaml to only include the images in the data folder and creates a data/annotations.yaml file.

This script prepares the files in data for the ImageTagger, i.e. zips the images and converts the annotations to the upload format.

This script can be used to label lines.

This script converts labels from the Pascal VOC XML format to the yaml format as defined above.

Creates the file data/annotations_with_metadata.yaml from data/annotations.yaml and data/metadata.csv. annotations.yaml can be downloaded from the ImageTagger, metadata.csv has to be manually created.

This script was used to resolve an issue regarding the segmentation images of the reality collection. This fixes the issue of incorrect color values of the class field caused by anti-aliasing and a old bug in the line_label_tool.

Variational Autoencoder

The variational autoencoder, we have used, is based on noctrog's conv-vae.

The training code for the autoencoder is located in scripts/vae/.


This file runs the training of the vae. More details are avalible by running vae/ -h.


This script runs the autoencoder on a given input and shows the recnstruction of the image. More details are avalible by running vae/ -h.


This script runs the vae recursivly on all image inside a given folder and saves their latent space representation und reconstruction errors inside a file. More details are avalible by running vae/ -h.


Plots n'th neigbours in the latent space of a given image. More details are avalible by running vae/ -h.


Creates a yaml file containing three lists containing:

  • The images that survived the pruning
  • The images that got selected due to the high reconstruction error
  • The images that will be removed from the dataset

More details are avalible by running vae/ -h.


Loads an embeddings file and plots the reconstruction errors.


The PyTorch model definition.


The PyTorch dataset definition.


# Layer (type) Output Shape Param #
Input (3, 128, 112) 0
1 Conv2d (32, 64, 56) 896
2 BatchNorm2d (32, 64, 56) 64
3 LeakyReLU (32, 64, 56) 0
4 Conv2d (64, 32, 28) 18,496
5 BatchNorm2d (64, 32, 28) 128
6 LeakyReLU (64, 32, 28) 0
7 Conv2d (64, 16, 14) 36,928
8 BatchNorm2d (64, 16, 14) 128
9 LeakyReLU (64, 16, 14) 0
10 Conv2d (64, 8, 7) 36,928
11 BatchNorm2d (64, 8, 7) 128
12 LeakyReLU (64, 8, 7) 0
13 Linear (300) 1,075,500
14 LeakyReLU (300) 0
15 Dropout (300) 0
16 Linear (300) 1,075,500
17 LeakyReLU (300) 0
18 Dropout (300) 0
19 Linear (3584) 1,078,784
20 LeakyReLU (3584) 0
21 Dropout (3584) 0
22 UpsamplingNearest2d (64, 16, 14) 0
23 ConvTranspose2d (64, 16, 14) 36,928
24 BatchNorm2d (64, 16, 14) 128
25 LeakyReLU (64, 16, 14) 0
26 psamplingNearest2d (64, 32, 28) 0
27 ConvTranspose2d (64, 32, 28) 36,928
28 BatchNorm2d (64, 32, 28) 128
29 LeakyReLU (64, 32, 28) 0
30 UpsamplingNearest2d (64, 64, 56) 0
31 ConvTranspose2d (32, 64, 56) 18,464
32 BatchNorm2d (32, 64, 56) 64
33 LeakyReLU (32, 64, 56) 0
34 UpsamplingNearest2d (32, 128, 112) 0
35 ConvTranspose2d (3, 128, 112) 867
36 Sigmoid (3, 128, 112) 0

Generation of Simulation Data

The code for generating the simulation data can be found here

TODO better documentation


Visualization of the position density of the respective annotations in the imagespace over all images of the real-world collection:



When you use our dataset or related software, please cite it as follows:

TORSO-21 Dataset: Typical Objects in RoboCup Soccer 2021

We present a dataset specifically designed to be used as a benchmark to compare vision systems in the RoboCup Humanoid Soccer domain. The dataset is composed of a collection of images taken in various real-world locations as well as a collection of simulated images. It enables comparing vision approaches with a meaningful and expressive metric. The contributions of this paper consist of providing a comprehensive and annotated dataset, an overview of the recent approaches to vision in RoboCup, methods to generate vision training data in a simulated environment, and an approach to increase the variety of a dataset by automatically selecting a diverse set of images from a larger pool. Additionally , we provide a baseline of YOLOv4 and YOLOv4-tiny on this dataset.

[ResearchGate] [Download]

author = {Bestmann, Marc and Engelke, Timon and Fiedler, Niklas and Güldenstein, Jasper and Gutsche, Jan and Hagge, Jonas and Vahl, Florian},
year = {2021},
title = {{TORSO-21 Dataset: Typical Objects in RoboCup Soccer 2021}},
booktitle={RoboCup 2021: Robot World Cup XXIV}

NOTE: You can get various citation types in the right sidebar on GitHub "Cite this repository"...


July 12, 2021

  • Replacement of the segmentations in the reality collection (using the The update towards the publication introduced incorrect color values of the field class in the segmentation images.

June 27, 2021

  • Publication


No description, website, or topics provided.







No releases published


No packages published