Antillia Realistic AutoAnnotation Tool (Updated: 2023/02/22)

This is an experimental project to implement Antillia Realistic AutoAnnotation Tool(ARAAT) for Object Detection.

It generates a realistic image dataset for training and validation, which is artificially made from a small real or realistic images, and annotates those images automatically for object detection.
For example, imagine to take a lot of real roadsigns pictures in US in a real world for training of object detection task. Probably, it is difficult to gather enough roadsigns images, because the classes of roadsigns in US is more than 160. One of other approaches to alleviate the difficulties is
to generate a lot of realistic and artificial images from some real or realistic object images including illustration by using any image augmentation methods. In this project, we try to design and implement a Realistic AutoAnnotation Tool, which generates a realistic images dataset, and annotation files to those images automatically,without any manual mouse operations of a GUI-based annotation tool.

1 Antillia Realistic AutoAnnotation Tool
2 Generate Resized images
3 Generate Enhanced images
4 Create YOLO dataset
5 Create TFRecord dataset
6 Create COCO dataset
7 Run create command
8 US_RoadSigns_160classes
9 US_RoadSigns_86classes
10 UK_RoadSigns_94classess
11 FR_RoadSigns_152classess
12 DE_RoadSigns_108classes
13 IT_RoadSigns_120classes
14 CA_RoadSigns_78classes

Added a sample project UK_RoadSigns_94classess (2022/07/07).

Added a sample project FR_RoadSigns_152classess (2022/07/07).

Added a sample project DE_RoadSigns_108classess (2022/07/14).

Added a sample project IT_RoadSigns_120classess (2023/02/18).

Fixed bugs in create_images_* methods of YOLOTrainDatasetCreator.py. (2023/02/18).

Added YOLOMixedSizeDatasetCreator.py. (2023/02/21).

Added a sample project CA_RoadSigns_120classess (2023/02/22).

1 Antillia Realistic AutoAnnotation Tool

We have been using tensorflow 2.4.0 and Python 3.8 environment on Windows11.
Please clone this repository to your local machine.
git clone https://github.com/sarah-antillia/Realistic-AutoAnnotation-Tool.git
You can see the following tree structure by tree command, and the projects folder contains two sample project Japanese-RoadSigns-90classes and USA_RoadSigns_160classes.

1.1 Folder tree structure

├─background
│  └─roadsigns
│      ├─test
│      ├─train
│      └─valid
├─batch_templates
├─config_templates
└─projects
    ├─CA_RoadSigns_78classes
    │  ├─configs
    │  └─PNG_CA_RoadSigns_78classes_Master
    ├─DE_RoadSigns_108classes
    │  ├─configs
    │  └─PNG_DE_RoadSigns_108classes_Master
    ├─FR_RoadSigns_152classes
    │  ├─configs
    │  └─PNG_FR_RoadSigns_152classes_Master
    ├─IT_RoadSigns_120classes
    │  ├─configs
    │  └─PNG_IT_RoadSigns_120classes_Master
    ├─JP_RoadSigns_90classes
    │  ├─configs
    │  └─PNG_JP_RoadSigns_90classes_Master
    ├─UK_RoadSigns_94classes
    │  ├─configs
    │  └─PNG_UK_RoadSigns_94classes_Master
    ├─US_RoadSigns_160classes
    │  ├─configs
    │  └─PNG_US_Roadsigns_160classes_Master
    └─US_RoadSigns_86classes
        ├─configs
        └─PNG_US_RoadSigns_86classes_Master

1.2 Prepare a set of object and background images

If you would like to get started your own project, you have to do:
1. Create your own project under the projects folder.
2. Prepare a minimum set of object images of some classes.
Those images must be PNG format, and image size less than 240x240.
3. Prepare a set of background images to create train and valid image dataset.
Those backgrounds must have same image size 512x512.
4. Prepare a set of background images to create test image dataset.
Those backgrounds must have same image size 1280x720.

The minimum image set will be augmented by our augmentation tool(python scripts), and the augmented images will be pasted on train, valid, test background images to generate realistic dataset for object detection.

1.3 Create a project

Please use ProjectCreator.py to create your own project.

python ProjectCreator.py category dataset_name project_name

Please run the following command to create JP_RoadSigns_90classes project:

python ProjectCreator.py roadsigns jp_roadsigns Japanese_RoadSigns_90classes

Sample images of PNG_JP_RoadSigns_Master

Those PNG files have been take from the following websites:
en.wikipedia.org:Road signs in Japan
commons.wikimedia.org: Road signs in Japan
The following background folders contain background jpg images files, which will be used as the background images to generate test, train and valid dataset.

└─background
    └─roadsigns
        ├─test
        ├─train
        └─valid

The ProjectCretator.py generates configs folder, which contain the following configuration files.

0_image_resizer.conf
1_image_enhancer.conf
2_yolo_train_dataset_creator.conf
3_yolo_test_dataset_creator.conf
4_yolo2tfrecord_converter.conf
5_yolo2coco_converter.conf
6_yolo2pascalvoc_converter.conf
color_enhancer.conf
warp_parallelogramer_medium.conf
warp_parallelogramer_small.conf
warp_rotator_medium.conf
warp_rotator_small.conf
warp_trapezoider_medium.conf
warp_trapezoider_small.conf

, and the following bat files under your project folder.

0_image_resizer.bat
1_image_enhancer.bat
2_yolo_train_dataset_creator.bat
3_yolo_test_dataset_creator.bat
4_yolo2tfrecord_converter.bat
5_yolo2coco_converter.bat
6_yolo2pascalvoc_converter.bat
create.bat
tfrecord_inspector.bat

2 Generate Resized images

Please move to your project directory, and run the following 0_image_enhancer.bat to resize original PNG roadsigns images:

./0_image_resizer.bat

, which is the following:

rem 0_image_resizer.bat
python ../../PNGImageResizer.py ./configs/0_image_resizer.conf

, and the 0_image_enhancer.conf in configs folder.

;0_image_resizer.conf
[configs]
version     = "2.2"
master_dir  = "./PNG_JP_RoadSigns_90classes_Master"
medium_dir  = "./PNG_JP_RoadSigns_90classes_Medium_Mixed"
small_dir   = "./PNG_JP_RoadSigns_90classes_Small_Mixed"
ratios      = [0.5, 1.0, 1.5, 2.0, ]

This will generate the following resized image folders:

├─PNG_JP_RoadSigns_90classes_Medium_Mixed
└─PNG_JP_RoadSigns_90classes_Small_Mixed

3 Generate Enhanced images

Please move to your project directory, and run the following 1_image_enhancer.bat to augment PNG roadsings images generated by 0_image_resizer.bat

./1_image_enhancer.bat

, which is the following:

python ../../ImageEnhancer.py ./configs/1_image_enhancer.conf train
python ../../ImageEnhancer.py ./configs/1_image_enhancer.conf test

, and the image_enhancer.conf in configs folder.

;1_image_enhander.conf
;2022/06/23/ Added medium and small sections.
[configs]
version                 = "2.2"

[medium]
warp_rotator_config     = "./configs/warp_rotator_medium.conf"
warp_trapezoider_config = "./configs/warp_trapezoider_medium.conf"
warp_parallelogramer    = "./configs/warp_parallelogramer_medium.conf"
enhanced_images_dir     = "./Enhanced_images_medium"

[small]
warp_rotator_config     = "./configs/warp_rotator_small.conf"
warp_trapezoider_config = "./configs/warp_trapezoider_small.conf"
warp_parallelogramer    = "./configs/warp_parallelogramer_small.conf"
enhanced_images_dir     = "./Enhanced_images_small"

This will generate the following enhanced image folders:

├─Enhanced_images_medium_test
├─Enhanced_images_medium_train
├─Enhanced_images_small_test
└─Enhanced_images_small_train

4 Create YOLO dataset

4.1 Create train and valid dataset

In your project folder, please run the following 2_yolo_train_dataset_creator.bat.

./2_yolo_train_dataset_creator.bat

, which is the following:

python ../../YOLOTrainDatasetCreator.py ./configs/2_yolo_train_dataset_creator.conf master

, and yolo_train_dataset_creator.conf

; 2_yolo_train_dataset_creator.conf
[configs]
version         = "2.1"
[dataset]
name            = "jp_roadsigns"
copyright       = "antillia.com"
version         = "1.1"
background_size = [512,512]
max_image_size  = [250,250]
classes         = "./classes.txt"
auto_splitter   = True
[master]
backgrounds_dir   = "../../background/roadsigns/train"
medium_images_dir = "./Enhanced_images_medium_train"
small_images_dir  =  "./Enhanced_images_small_train"
output_dir        = "./YOLO_JP_RoadSigns_90classes/master"
[train]
backgrounds_dir   = "../../background/roadsigns/train"
images_dir        = "./Enhanced_images_train"
output_dir        = "./YOLO_JP_RoadSigns_90classes/train"
[valid]
backgrounds_dir   = "../../background/roadsigns/valid"
images_dir        = "./Enhanced_images_valid"
output_dir        = "./YOLO_JP_RoadSigns_90classes/valid"

This bat file will generate YOLO_JP_RoadSigns_90classes folder, which contains train and valid dataset(images and annotation files).

└─YOLO_JP_RoadSigns_90classes
    ├─master
    ├─train
    └─valid

The train and valid dataset will be generated automatically from the master by a splitting tool.

Sample images of YOLO master dataset

4.2 Create realistic test dataset

In your project folder, please run the following 3_yolo_test_dataset_creator.bat to create realistic test dataset.

./3_yolo_test_dataset_creator.bat

, which is the following:

python ../../YOLOTestDatasetCreator.py ./configs/3_yolo_test_dataset_creator.conf

, and 3_yolo_test_dataset_creator.conf.

; 3_yolo_test_dataset_creator.conf
[configs]
version         = "2.0"
[dataset]
name            = "jp_roadsigns"
copyright       = "antillia.com"
version         = "1.0"
background_size = [1280,720]
max_image_size  = [240, 240]
classes         = "./classes.txt"
[test]
backgrounds_dir   = "../../background/roadsigns/test"
images_dir      = "./Enhanced_images_small_test"
output_dir      = "./realistic_test_dataset"
num_test_dataset= 100

This will generate realistic_test_dataset folder, which contain test dataset(images and annotation files).

└─realistic_test_dataset

You can download realistic_test_dataset from Japanese-RoadSigns-90classes-V5.

Sample images of realistic_test_dataset

5 Create TFRecord dataset

In your project folder, please run the following 4_yolo2tfrecord_converter.bat to convert YOLO annotation dataset to TFRecord:

./4_yolo2tfrecord_converter.bat 

, which is the following.

python ../../YOLO2TFRecordConverter.py ./configs/4_yolo2tfrecord_converter.conf 

, and 4_yolo2tfrecord_converter.conf.

; 4_yolo2tfrecord_converter.conf
[configs]
version      = "2.0"

[dataset]
name         = "jp_roadsigns"
copyright    = "antillia.com"
version      = "2.0"
classes      = "./classes.txt"

tfrecord_dir = "./TFRecord_JP_RoadSigns_90classes"

label_map_pbtxt = "./TFRecord_JP_RoadSigns_90classes/label_map.pbtxt"
label_map_yaml  = "./TFRecord_JP_RoadSigns_90classes/label_map.yaml"

[train]
images_dir   = "./YOLO_JP_RoadSigns_90classes/train"
anno_dir     = "./YOLO_JP_RoadSigns_90classes/train"

[valid]
images_dir   = "./YOLO_JP_RoadSigns_90classes/valid"
anno_dir     = "./YOLO_JP_RoadSigns_90classes/valid"

This wll generate TFRecord_JP_RoadSigns_90classes folder, which contain train and valid tfrecords.

└─TFRecord_JP_RoadSigns_90classes
    ├─train
    └─valid

Please run the following bat file to inspect the generated tfrecord.
tfrecord_inspect.bat

python ../../TFRecordInspector.py ^
  ./TFRecord_JP_RoadSigns_90classes/train/train.tfrecord ^
  ./TFRecord_JP_RoadSigns_90classes/label_map.pbtxt ^
  ./Inspector/train

Objects count in train.tfrecord:


Sample images in train.tfrecord:

Label map label_map.pbtxt

6 Create COCO dataset

In your project directory, please run the following 5_yolo2coco_converter.bat to convert YOLO annotation dataset to COCO annotation dataset:

,/5_yolo2coco_converter.bat

, which is the following:

python ../../YOLO2COCOConverter.py ./configs/5_yolo2coco_converter.conf 

,and 5_yolo2coco_converter.conf.

; 5_yolo2coco_converter.conf
[configs]
version      = "2.0"
[dataset]
name         = "jp_roadsigns"
copyright    = "antillia.com"
version      = "2.0"
classes      = "./classes.txt"
[train]
images_dir  = "./YOLO_JP_RoadSigns_90classes/train"
output_dir  = "./COCO_JP_RoadSigns_90classes/train"
[valid]
images_dir  = "./YOLO_JP_RoadSigns_90classes/valid"
output_dir  = "./COCO_JP_RoadSigns_90classes/valid"

You can download COCO Dataset from Japanese-RoadSigns-90classes-V5.

7 Run create command

If you would like to create YOLO, TFRecord and COCO dataset from your master image dataset at once, please run the following batch processing command:

./create.bat

, which is the following.

call 0_image_resizer.bat
call 1_image_enhancer.bat
call 2_yolo_train_dataset_creator.bat
call 3_yolo_test_dataset_creator.bat
call 4_yolo2tfrecord_converter.bat
call 5_yolo2coco_converter.bat

You can download the following dataset created by the command above:

COCO_JP_RoadSigns_90classes.zip
TFRecord_JP_RoadSigns_90classes.zip
realistic_test_dataset.zip
YOLO_JP_RoadSigns_90classes.zip

from here JP_RoadSigns_90classes_V7.

See also:EfficientDet-Slightly-Realistic-Japanese-RoadSigns-90classes
Inference example of realistic_test_dataset generated by this tool:

Inference example of real_japanese_roadsigns:

8 US_RoadSigns_160classes

8.1. Create US_RoadSigns_160classes project

You can create US_RoadSigns_160classes project by running the following command.

python ProjectCreator.py roadsigns us_roadsigns US_RoadSigns_160classes

Sample images of PNG_US_RoadSigns_160classes_Master

The PNG files in US_RoadSigns_160classes_Master have been take from the following websites:
Road signs in the United States

8.2. Run create command

Please run the following batch processing command to create YOLO, TFRecord and COCO dataset from your base image dataset at once,

./create.bat

See also:EfficientDet-Slightly-Realistic-USA-RoadSigns-160-classes
Inference example of realistic_test_dataset generated by this tool:

9 US_RoadSigns_86classes

9.1. Create US_RoadSigns_86classes project

You can create US_RoadSigns_86classes project by running the following command.

python ProjectCreator.py roadsigns us_roadsigns US_RoadSigns_86classes

Sample images of PNG_US_RoadSigns_86classes_Master

The PNG files in US_RoadSigns_86classes_Master have been take from the following websites:
Road signs in the United States

9.2. Run create command

Please run the following batch processing command to create YOLO, TFRecord and COCO dataset from your base image dataset at once,

./create.bat

See also:EfficientDet-Slightly-Realistic-US-RoadSigns-86classes
Inference example of realistic_test_dataset generated by this tool:

10 UK_RoadSigns_94classes

10.1. Create UK_RoadSigns_94classes project

You can create UK_RoadSigns_94classes project by running the following command.

python ProjectCreator.py roadsigns uk_roadsigns UK_RoadSigns_94classes

Sample images of PNG_UK_RoadSigns_94classes_Master

The PNG files in UK_RoadSigns_94classes_Master have been take from the following websites:
Road signs in the United Kingdom

10.2. Run create command

Please run the following batch processing command to create YOLO, TFRecord and COCO dataset from your base image dataset at once,

./create.bat

See also:EfficientDet-Slightly-Realistic-UK-RoadSigns-94classes
Inference example of realistic_test_dataset generated by this tool:

11 FR_RoadSigns_152classes

11.1. Create FR_RoadSigns_152classes project

You can create FR_RoadSigns_152classes project by running the following command.

python ProjectCreator.py roadsigns fr_roadsigns FR_RoadSigns_152classes

Sample images of PNG_FR_RoadSigns_152classes_Master

The PNG files in FR_RoadSigns_152classes_Master have been take from the following websites:
Road signs in France

11.2. Run create command

Please run the following batch processing command to create YOLO, TFRecord and COCO dataset from your base image dataset at once,

./create.bat

See also:EfficientDet-Slightly-Realistic-FR-RoadSigns-152classes
Inference example of realistic_test_dataset generated by this tool:

12 DE_RoadSigns_108classes (Updated: 2022/07/15)

12.1. Create DE_RoadSigns_108classes project

You can create DE_RoadSigns_108classes project by running the following command.

python ProjectCreator.py roadsigns de_roadsigns DE_RoadSigns_108classes

Sample images of PNG_DE_RoadSigns_108classes_Master

The PNG files in DE_RoadSigns_108classes_Master have been take from the following websites:
Comparison of European road signs

12.2. Run create command

Please run the following batch processing command to create YOLO, TFRecord and COCO dataset from your base image dataset at once,

./create.bat

See also:EfficientDet-Slightly-Realistic-DE-RoadSigns-108classes
Inference example of realistic_test_dataset generated by this tool:

13 IT_RoadSigns_120classes (Updated: 2023/02/18)

13.1. Create IT_RoadSigns_120classes project

You can create IT_RoadSigns_120classes project by running the following command.

python ProjectCreator.py roadsigns it_roadsigns IT_RoadSigns_120classes

Sample images of PNG_IT_RoadSigns_120classes_Master

The PNG files in IT_RoadSigns_120classes_Master have been take from the following websites:
Road signs in Italy

13.2. Run create command

Please run the following batch processing command to create YOLO, TFRecord and COCO dataset from your base image dataset at once,

./create.bat

See also:EfficientDet-Slightly-Realistic-IT-RoadSigns-120classes
Inference example of realistic_test_dataset generated by this tool:

14 CA_RoadSigns_78classes (Updated: 2023/02/22)

14.1. Create CA_RoadSigns_78classes project

You can create CA_RoadSigns_78classes project by running the following command.

python ProjectCreator.py roadsigns ca_roadsigns CA_RoadSigns_78classes

Sample images of PNG_CA_RoadSigns_78classes_Master

The PNG files in CA_RoadSigns_78classes_Master have been take from the following websites:
Road signs in Canada

14.2. Run create command

Please run the following batch processing command to create YOLO, TFRecord and COCO dataset from your base image dataset at once,

./create.bat

See also:EfficientDet-Slightly-Realistic-CA-RoadSigns-78classes
Inference example of realistic_test_dataset generated by this tool: