Playing-Card-Detection---YOLO-V4-

Detect playing cards using yolo v4. And Deploy it in android.

Convert Darknet YOLOv4 or YOLOv3 to TensorFlow Model

YOLOv4, YOLOv4-tiny Implemented in Tensorflow 2.3.

This repository is created for the channel TheCodingBug.

This repository shows how to convert YOLO v4, YOLOv3, YOLO tiny .weights to .pb, .tflite and trt format for tensorflow, tensorflow lite, tensorRT.

1. Setting Up Environment
   • Using Conda
   • Using Pip
2. Convert YOLOv4 to TensorFlow
   • Run Object Detection
3. Convert YOLOv4 to tflite
   • Run Objection Detection using TFLite Model
4. FPS Comparison

Setting Up Environment

Using Conda

CPU

# CPU
conda env create -f conda-cpu.yml


# activate environment on Windows or Linux
conda activate tf-cpu

# activate environment on Mac
source activate tf-cpu

GPU

# GPU
conda env create -f conda-gpu.yml

# activate environment on Windows or Linux
conda activate tf-gpu

# activate environment on Mac
source activate tf-gpu

Using Pip

# CPU
pip install -r requirements.txt

# GPU
pip install -r requirements-gpu.txt

Note: If installing GPU version with Pip, you need to install CUDA and cuDNN in your system. You can find the tutorial for Windows here.

Performance

Download Weights File

Download yolov4.weights file 245 MB: yolov4.weights (Google-drive mirror yolov4.weights )

If using tiny version, download yolov4-tiny.weights file instead. I am using original yolov4 because its more accurate (although slower).

If you have a custom model then replace the (.name) file in the ./data/classes/ and replace the (.weight) file in ./data

Then you need to edit the "./data/classes/coco.names" coco.names name with your own custom (.names). You will find in the config.py file which is in ./core/ folder.

Convert YOLOv4 to TensorFlow

# Convert darknet weights to tensorflow
## yolov4
python save_model.py --weights ./data/yolov4.weights --output ./checkpoints/yolov4-416 --input_size 416 --model yolov4 

## yolov4-tiny
python save_model.py --weights ./data/yolov4-tiny.weights --output ./checkpoints/yolov4-tiny-416 --input_size 416 --model yolov4 --tiny

If you want to run yolov3 or yolov3-tiny change --model yolov3 in command and also download corresponding YOLOv3 weights and and change --weights to ./data/yolov3.weights

Run Object Detection

# Run yolov4 on image
python detect.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --image ./data/kite.jpg

python detect.py --weights ./checkpoints/yolov4-tiny-416 --size 416 --model yolov4 --image ./data/kite.jpg --tiny

# Run on multiple images
python detect.py --weights ./checkpoints/yolov4-tiny-416 --size 416 --model yolov4 --images "./data/kite.jpg, ./data/girl.jpg"

# Run yolov4 on video
python detect_video.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --video ./data/japan.mp4 --output ./detections/video_output.avi

# Run yolov4 on webcam
python detect_video.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --video 0 --output ./detections/webcam_output.avi

Output

Yolov4 original weight

Convert to tflite

# Save tf model for tflite converting
python save_model.py --weights ./data/yolov4.weights --output ./checkpoints/yolov4-416 --input_size 416 --model yolov4 --framework tflite

# yolov4
python convert_tflite.py --weights ./checkpoints/yolov4-416 --output ./checkpoints/yolov4-416.tflite

# yolov4 quantize float16
python convert_tflite.py --weights ./checkpoints/yolov4-416 --output ./checkpoints/yolov4-416-fp16.tflite --quantize_mode float16

# yolov4 quantize int8
python convert_tflite.py --weights ./checkpoints/yolov4-416 --output ./checkpoints/yolov4-416-int8.tflite --quantize_mode int8

Run Objection Detection using TFLite Model

# Run demo tflite model
python detect.py --weights ./checkpoints/yolov4-416.tflite --size 416 --model yolov4 --image ./data/kite.jpg --framework tflite

# Run demo tflite on videos
python detect_video.py --weights ./checkpoints/yolov4-416.tflite --size 416 --model yolov4 --video ./data/japan.mp4 --output ./detections/video_output.avi --framework tflite

Yolov4 tflite int8 weight

Yolov4 and Yolov4-tiny int8 quantization have some issues. I will try to fix that. You can try Yolov3 and Yolov3-tiny int8 quantization

Convert to TensorRT

Similar to above method, you can convert darknet yolov3 or yolov4 models to tensorflow and then to TensorRT.

# yolov3
python save_model.py --weights ./data/yolov3.weights --output ./checkpoints/yolov3.tf --input_size 416 --model yolov3
python convert_trt.py --weights ./checkpoints/yolov3.tf --quantize_mode float16 --output ./checkpoints/yolov3-trt-fp16-416

# yolov3-tiny
python save_model.py --weights ./data/yolov3-tiny.weights --output ./checkpoints/yolov3-tiny.tf --input_size 416 --tiny
python convert_trt.py --weights ./checkpoints/yolov3-tiny.tf --quantize_mode float16 --output ./checkpoints/yolov3-tiny-trt-fp16-416

# yolov4
python save_model.py --weights ./data/yolov4.weights --output ./checkpoints/yolov4.tf --input_size 416 --model yolov4
python convert_trt.py --weights ./checkpoints/yolov4.tf --quantize_mode float16 --output ./checkpoints/yolov4-trt-fp16-416

FPS Comparison

We use japan.mp4 for all the experiments and report following results.

OpenCV GPU: 9.96 FPS

Darknet GPU: 13.5 FPS

TensorFlow GPU: 11.5 FPS

TFLite CPU: 2 FPS

TFLite GPU (On Android): To be tested

Evaluate on COCO 2017 Dataset

# run script in /script/get_coco_dataset_2017.sh to download COCO 2017 Dataset
# preprocess coco dataset
cd data
mkdir dataset
cd ..
cd scripts
python coco_convert.py --input ./coco/annotations/instances_val2017.json --output val2017.pkl
python coco_annotation.py --coco_path ./coco 
cd ..

# evaluate yolov4 model
python evaluate.py --weights ./data/yolov4.weights
cd mAP/extra
python remove_space.py
cd ..
python main.py --output results_yolov4_tf

mAP50 on COCO 2017 Dataset

Detection	512x512	416x416	320x320
YoloV3	55.43	52.32
YoloV4	61.96	57.33

Traning your own model

# Prepare your dataset
# If you want to train from scratch:
In config.py set FISRT_STAGE_EPOCHS=0 
# Run script:
python train.py

# Transfer learning: 
python train.py --weights ./data/yolov4.weights

The training performance is not fully reproduced yet, so I recommended to use Alex's Darknet to train your own data, then convert the .weights to tensorflow or tflite.

TODO

• Convert YOLOv4 to TensorRT
• YOLOv4 tflite on android
• YOLOv4 tflite on ios
• Training code
• Update scale xy
• ciou
• Mosaic data augmentation
• Mish activation
• yolov4 tflite version
• yolov4 in8 tflite version for mobile