FreeYOLO

FreeYOLO is inspired by many other excellent works, such as YOLOv7 and YOLOX. Achieving the SOTA performance is not my purpose, which requires much computing resource. I just believe Mr. Feynman's famous saying: Learning by doing. I will never understand the YOLO detector until I achieve it.

I have tried my best to design FreeYOLO. Although there is still much room for improvement, my few GPU devices are not enough to support me to continue to optimize it, which is a pity for me.

Nevertheless, in this project, you will enjoy:

FreeYOLO on COCO for general object detection.
FreeYOLO on WiderFace for face detection.
FreeYOLO on CrowdHuman for person detection.

Besides the detection task, I also apply FreeYOLO to the multi-object tracking task.

FreeTrack

My FreeTrack consists of the FreeYOLO object detector and ByteTrack tracker. Note that my FreeTrack is just a A+B work, so it is not novel.

Requirements

We recommend you to use Anaconda to create a conda environment:

conda create -n yolo python=3.6

Then, activate the environment:

conda activate yolo

Requirements:

pip install -r requirements.txt

My environment:

PyTorch = 1.9.1
Torchvision = 0.10.1

At least, please make sure your torch is version 1.x.

Tricks

Mosaic Augmentation
Mixup Augmentation
Multi scale training
Cosine Annealing Schedule

Training Configuration

Configuration
Per GPU Batch Size	16 (8 for FreeYOLO-Huge)
Init Lr	0.01
Lr Scheduler	Cos
Optimizer	SGD
ImageNet Predtrained	True
Multi Scale Train	True
Mosaic	True
Mixup	True

Experiments

COCO

Download COCO.

cd <FreeYOLO_HOME>
cd dataset/scripts/
sh COCO2017.sh

Check COCO

cd <FreeYOLO_HOME>
python dataset/coco.py

Train on COCO

For example:

python train.py --cuda -d coco -v yolo_free_nano -bs 16 --max_epoch 300 --wp_epoch 1 --eval_epoch 10 --fp16 --ema --root path/to/COCO

Main results on COCO-val:

Model	Scale	FPS^2080ti	FLOPs	Params	AP^val 0.5:0.95	AP^{test 0.5:0.95}	Weight
FreeYOLO-Nano	640	50	4.6 G	2.0 M	30.5	31.1	github
FreeYOLO-Tiny	640	66	13.9 G	6.2 M	34.4	35.2	github
FreeYOLO-Large	640	50	144.8 G	44.1 M	48.6	49.0	github
FreeYOLO-Huge	640	34	257.8 G	78.9 M	50.0	50.0	github

WiderFace

Download WiderFace.
Prepare WiderFace

WiderFace
|_ WIDER_train
|  |_ images
|     |_ 0--Parade
|     |_ ...
|_ WIDER_tval
|  |_ images
|     |_ 0--Parade
|     |_ ...
|_ wider_face_split
|_ eval_tools

Convert WiderFace to COCO format.

cd <FreeYOLO_HOME>
python tools/convert_widerface_to_coco.py --root path/to/WiderFace

Check WiderFace

cd <FreeYOLO_HOME>
python dataset/widerface.py

Train on WiderFace For example:

python train.py --cuda -d widerface --root path/to/WiderFace -v yolo_free_nano -bs 16 -lr 0.001 -mlr 0.05 --max_epoch 100 --wp_epoch 1 --eval_epoch 10 --fp16 --ema --pretrained path/to/coco/yolo_free_nano_ckpt --mosaic 0.5 --mixup 0.0 --min_box_size 1

Main results on WiderFace-val:

Model	Scale	AP	AP50	Weight
FreeYOLO-Nano	640	26.4	51.6	github
FreeYOLO-Tiny	640	30.1	57.4	github
FreeYOLO-Large	640	35.7	64.6	github
FreeYOLO-Huge	640	35.8	64.8	github

CrowdHuman

Download CrowdHuman.

CrowdHuman
|_ CrowdHuman_train01.zip
|_ CrowdHuman_train02.zip
|_ CrowdHuman_train03.zip
|_ CrowdHuman_val.zip
|_ annotation_train.odgt
|_ annotation_val.odgt

Prepare CrowdHuman

CrowdHuman
|_ CrowdHuman_train
|  |_ Images
|     |_ 273271,1a0d6000b9e1f5b7.jpg
|     |_ ...
|_ CrowdHuman_val
|  |_ Images
|     |_ 273271,1b9330008da38cd6.jpg
|     |_ ...
|_ annotation_train.odgt
|_ annotation_val.odgt

Convert CrowdHuman to COCO format.

cd <FreeYOLO_HOME>
python tools/convert_crowdhuman_to_coco.py --root path/to/CrowdHuman

Check CrowdHuman

cd <FreeYOLO_HOME>
python dataset/crowdhuman.py

Train on CrowdHuman For example:

python train.py --cuda -d crowdhuman -v yolo_free_nano -bs 16 -lr 0.001 -mlr 0.05 --max_epoch 100 --wp_epoch 1 --eval_epoch 10 --fp16 --ema --root path/to/CrowdHuman --pretrained path/to/coco/yolo_free_nano_ckpt

Main results on CrowdHuman-val:

Model	Scale	AP	AP50	Weight
FreeYOLO-Nano	640	31.3	67.2	github
FreeYOLO-Tiny	640	34.7	70.4	github
FreeYOLO-Large	640	43.1	76.5	github
FreeYOLO-Huge	640	44.8	78.9	github

MOT17

Download MOT17, then you will get a ```MOT17.zip` file.
Prepare MOT17

MOT17
|_ train
|  |_ MOT17-02-DPM
|     |_ det
|     |_ gt
|     |_ img1
|     |_ ...
|  ...
|_ test
|  |_ MOT17-01-DPM
|     |_ det
|     |_ img1
|     |_ ...
|  ...

Convert MOT17 to COCO format.

cd <FreeYOLO_HOME>
python tools/convert_mot17_to_coco.py --root path/to/MOT17

Check MOT17

cd <FreeYOLO_HOME>
python dataset/mot17.py

Train on MOT17 half

For example:

python train.py --cuda -d mot17_half -v yolo_free_nano -bs 16 --max_epoch 100 --wp_epoch 1 --eval_epoch 10 --fp16 --ema --root path/to/MOT17 --pretrained path/to/coco/yolo_free_nano_ckpt

Main results on MOT17 val-half (trained on MOT17 train-half):

Model	Scale	AP	AP50	Weight
FreeYOLO-Nano	640
FreeYOLO-Tiny	640
FreeYOLO-Large	640
FreeYOLO-Huge	640

Train on MOT17

For example:

python train.py --cuda -d mot17 -v yolo_free_nano -bs 16 --max_epoch 100 --wp_epoch 1 --fp16 --ema --root path/to/MOT17 --pretrained path/to/coco/yolo_free_nano_ckpt

Pretrained weights on MOT17 train split (fully train, not train-half):

MOT20

Download MOT20, then you will get a ```MOT20.zip` file.
Prepare MOT20 Similar to MOT17
Convert MOT20 to COCO format.

cd <FreeYOLO_HOME>
python tools/convert_mot20_to_coco.py --root path/to/MOT20

Check MOT20

cd <FreeYOLO_HOME>
python dataset/mot20.py

Train on MOT20 half

For example:

python train.py --cuda -d mot20_half -v yolo_free_nano -bs 16 --max_epoch 100 --wp_epoch 1 --eval_epoch 10 --fp16 --ema --root path/to/MOT20 --pretrained path/to/coco/yolo_free_nano_ckpt

Main results on MOT20 val-half (trained on MOT20 train-half):

Model	Scale	AP	AP50	Weight
FreeYOLO-Nano	640
FreeYOLO-Tiny	640
FreeYOLO-Large	640
FreeYOLO-Huge	640

Train on MOT20

For example:

python train.py --cuda -d mot20 -v yolo_free_nano -bs 16 --max_epoch 100 --wp_epoch 1 -- eval_epoch 10 --fp16 --ema --root path/to/MOT20 --pretrained path/to/coco/yolo_free_nano_ckpt

Pretrained weights on MOT20 train split (fully train, not train-half):

Train

Single GPU

sh train.sh

You can change the configurations of train.sh, according to your own situation.

Multi GPUs

sh train_ddp.sh

You can change the configurations of train_ddp.sh, according to your own situation.

In the event of a training interruption, you can pass --resume the latest training weight path (None by default) to resume training. For example:

python train.py \
        --cuda \
        -d coco \
        -v yolo_free_large \
        --ema \
        --fp16 \
        --eval_epoch 10 \
        --resume weights/coco/yolo_free_large/yolo_free_large_epoch_151_39.24.pth

Then, training will continue from 151 epoch.

Test

python test.py -d coco \
               --cuda \
               -v yolo_free_large \
               --img_size 640 \
               --weight path/to/weight \
               --root path/to/dataset/ \
               --show

Evaluation

python eval.py -d coco-val \
               --cuda \
               -v yolo_free_large \
               --img_size 640 \
               --weight path/to/weight \
               --root path/to/dataset/ \
               --show

Demo

I have provide some images in data/demo/images/, so you can run following command to run a demo:

python demo.py --mode image \
               --path_to_img data/demo/images/ \
               -v yolo_free_large \
               --img_size 640 \
               --cuda \
               --weight path/to/weight

If you want run a demo of streaming video detection, you need to set --mode to video, and give the path to video --path_to_vid。

python demo.py --mode video \
               --path_to_img data/demo/videos/your_video \
               -v yolo_free_large \
               --img_size 640 \
               --cuda \
               --weight path/to/weight

If you want run video detection with your camera, you need to set --mode to camera。

python demo.py --mode camera \
               -v yolo_free_large \
               --img_size 640 \
               --cuda \
               --weight path/to/weight

Train on ourself dataset

Besides the popular datasets, we can also train the model on ourself dataset. To achieve this goal, you should follow these steps:

Step-1: Prepare the images (JPG/JPEG/PNG ...) and use labelimg to make XML format annotation files.

OurDataset
|_ train
|  |_ images     
|     |_ 0.jpg
|     |_ 1.jpg
|     |_ ...
|  |_ annotations
|     |_ 0.xml
|     |_ 1.xml
|     |_ ...
|_ val
|  |_ images     
|     |_ 0.jpg
|     |_ 1.jpg
|     |_ ...
|  |_ annotations
|     |_ 0.xml
|     |_ 1.xml
|     |_ ...
|  ...

You can refer the format of dataset/OurDataset/ which has been provided in this project.

Step-2: Convert ourdataset to COCO format.

cd <FreeYOLO_HOME>
cd tools
# convert train split
python convert_ours_to_coco.py --root path/to/OurDataset/ --split train
# convert val split
python convert_ours_to_coco.py --root path/to/OurDataset/ --split val

Then, we can get a train.json file and a val.json file, as shown below.

OurDataset
|_ train
|  |_ images     
|     |_ 0.jpg
|     |_ 1.jpg
|     |_ ...
|  |_ annotations
|     |_ 0.xml
|     |_ 1.xml
|     |_ ...
|     |_ train.json
|_ val
|  |_ images     
|     |_ 0.jpg
|     |_ 1.jpg
|     |_ ...
|  |_ annotations
|     |_ 0.xml
|     |_ 1.xml
|     |_ ...
|     |_ val.json
|  ...

Step-3 Define our class labels.

Please open dataset/ourdataset.py file and change our_class_labels = ('cat',) according to our definition of categories.

Step-4 Check

cd <FreeYOLO_HOME>
cd dataset
# convert train split
python ourdataset.py --root path/to/OurDataset/ --split train
# convert val split
python ourdataset.py --root path/to/OurDataset/ --split val

Step-5 Train

For example:

cd <FreeYOLO_HOME>
python train.py --root path/to/OurDataset/ -d ourdataset -v yolo_free_nano -bs 16 -p path/to/COCO-pretrained-model

Step-6 Test

For example:

cd <FreeYOLO_HOME>
python test.py --root path/to/OurDataset/ -d ourdataset -v yolo_free_nano --weight path/to/checkpoint --show

Step-7 Eval

For example:

cd <FreeYOLO_HOME>
python eval.py --root path/to/OurDataset/ -d ourdataset -v yolo_free_nano --weight path/to/checkpoint

Name		Name	Last commit message	Last commit date
Latest commit History 707 Commits
config		config
dataset		dataset
deployment		deployment
evaluator		evaluator
img_files		img_files
models		models
tools		tools
utils		utils
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
benchmark.py		benchmark.py
demo.py		demo.py
engine.py		engine.py
eval.py		eval.py
requirements.txt		requirements.txt
test.py		test.py
train.py		train.py
train.sh		train.sh
train_ddp.sh		train_ddp.sh

License

yjh0410/FreeYOLO

Folders and files

Latest commit

History

Repository files navigation

FreeYOLO

Requirements

Tricks

Training Configuration

Experiments

COCO

WiderFace

CrowdHuman

MOT17

MOT20

Train

Single GPU

Multi GPUs

Test

Evaluation

Demo

Train on ourself dataset

Deployment

About

Resources

License

Stars

Watchers

Forks

Languages