This reference kit provides an end-to-end (E2E) Machine Learning (ML) workflow for traffic management using Pytorch*, Intel® Neural Compressor, OpenVINO* and Intel® Optimizations for PyTorch*. Traffic management is an important issue plaguing well-established and rapidly growing cities. Bad traffic management and accidents impact performance at work, but mainly quality of life.
Check out more workflow examples in the Developer Catalog.
Traffic accidents are dangerous and often result in fatalities. The time taken to respond to accidents and send medical aid depends on multiple human factors. Often a lack of timely response impacts the likelihood of survival.
Intelligent traffic management systems leveraging video surveillance, automated accident detection, and prediction solutions will go a long way in improving safety and traffic flow in cities. These solutions need to provide real-time insights and recommendations to be effective.
Leveraging the power of edge computing, communication between devices with very low latency can be achieved. An example scenario includes traffic signals and vehicles exchanging information about pedestrians on crosswalks, communication between surveillance cameras and vehicles regarding proximity, and the possibility of an accident with very low latency; thus enabling preventive action in near real-time.
Deep Learning (DL) algorithms can predict traffic accidents based on live traffic camera feeds. Computer vision tasks and complex feature analysis can be accomplished easily with high performance by leveraging DL algorithms.
Artificial Intelligence (AI) based detection algorithms deployed at the edge enable real-time analytics of video feeds and detection of accidents and other issues, thus improving overall traffic management. AI-enabled traffic camera imaging aid helps address traffic management challenges by reducing congestion on road, improving the accuracy of pedestrian/vehicle identification, improving device-2-device communication, and helping reduce accidents.
This workflow implementation is a reference solution to the described use case that includes an optimized reference E2E architecture enabled with Intel® Optimizations for PyTorch* and Intel® Neural Compressor available as part of Intel® AI Tools optimizations and OpenVINO*:
-
Intel® Optimizations for PyTorch*
PyTorch* is an AI and machine learning framework popular for both research and production usage. This open source library is often used for deep learning applications whose compute-intensive training and inference test the limits of available hardware resources. Intel releases its newest optimizations and features in Intel® Optimizations for PyTorch* before upstreaming them into open source PyTorch.
-
Intel® Neural Compressor
Intel® Neural Compressor performs model compression to reduce the model size and increase the speed of deep learning inference for deployment on CPUs or GPUs. This open source Python* library automates popular model compression technologies, such as quantization, pruning, and knowledge distillation across multiple deep learning frameworks.
-
OpenVINO*
OpenVINO* is an open-source toolkit for optimizing and deploying AI inference.
- Boost deep learning performance in computer vision, automatic speech recognition, natural language processing and other common tasks.
- Use models trained with popular frameworks like TensorFlow, PyTorch and more.
- Reduce resource demands and efficiently deploy on a range of Intel® platforms from edge to cloud.
The dataset to be used is the Pascal VOC dataset. It will be downloaded automatically when running the train.py script and divided into folders for training and validation.
The training data consists of a set of images; each image has an annotation file giving a bounding box and object class label for each object in one of the twenty classes present in the image. Multiple objects from multiple classes may be present in the same image.
Note: Please see this dataset applicable license for terms and conditions. Intel®Corporation does not own the rights to this dataset and does not confer any rights to it.
Folder structure looks as below after data downloaded:
data/
├── images
│ ├── VOCdevkit
│ │ ├── VOC2007
│ │ │ ├── Annotations
│ │ │ ├── ImageSets
│ │ │ │ ├── Layout
│ │ │ │ ├── Main
│ │ │ │ └── Segmentation
│ │ │ ├── JPEGImages
│ │ │ ├── SegmentationClass
│ │ │ └── SegmentationObject
│ │ └── VOC2012
│ │ ├── Annotations
│ │ ├── ImageSets
│ │ │ ├── Action
│ │ │ ├── Layout
│ │ │ ├── Main
│ │ │ └── Segmentation
│ │ ├── JPEGImages
│ │ ├── SegmentationClass
│ │ └── SegmentationObject
│ ├── test2007
│ ├── train2007
│ ├── train2012
│ ├── val2007
│ └── val2012
└── labels
├── test2007
├── train2007
├── train2012
├── val2007
└── val2012
Intel® oneAPI is used to achieve quick results even when the data for a model is huge. It provides the capability to reuse the code present in different languages so that the hardware utilization is optimized to provide these results.
| Recommended Hardware | Precision |
|---|---|
| CPU: Intel® 2nd Gen Xeon® Platinum 8280 CPU @ 2.70GHz or higher | FP32, INT8, BF16 |
| RAM: 187 GB | |
| Recommended Free Disk Space: 20 GB or more |
Note: BF16 can be enabled on Intel® Fourth Gen Xeon®, previous generations of Xeon® might not be compatible.
Code was tested on Ubuntu* 22.04 LTS.
The following diagram shows the traffic camera object detection E2E workflow:
The following is an example of expected input and output:
| Input | Output |
|---|---|
| Traffic Camera Live Feed | Detected Objects(Vehicle/Pedestrians) and Alarm(High Risk/Low Risk) for possible accident scenarios |
 |
 |
This reference kit uses a general detection model capable of distinguishing objects that would be relevant to traffic cameras. It preprocesses a Pascal VOC dataset by combining it with COCO classes using OpenCV*. A transfer learning approach is performed using an advanced pre-trained real-time object detection YOLOv5 model, which is further trained to detect vehicles and pedestrians. The dataset is first preprocessed using OpenCV* and NumPy, and then NumPy based postprocessing is performed using Non-Maxima Suppression (NMS) and centroid-based distance calculations for possible collision detection, which could be used for example to warn vehicle drivers via device-2-device communication.
Since GPUs are typically the choice for Deep Learning and AI processing to achieve a higher Frames Per Second (FPS) rate, to offer a more cost-effective option leveraging a CPU, we use the quantization technique, leveraging Intel® AI Tools, to achieve high FPS by performing vectorized operations on CPUs itself.
By quantizing/compressing the model (from floating point to integer model), while maintaining a similar level of accuracy as the floating point model, we demonstrate efficient utilization of underlying resources when deployed on edge devices with low processing and memory capabilities. Model has been quantized using Intel® Neural Compressor and Intel® Distribution of OpenVINO* which has shown high-performance vectorized operations on Intel® platforms.
Start by defining an environment variable that will store the workspace path, these directories will be created in further steps and will be used for all the commands executed using absolute paths.
export WORKSPACE=$PWD/traffic-camera-object-detection
export DATA_DIR=$WORKSPACE/data
export OUTPUT_DIR=$WORKSPACE/output
export CONFIG_DIR=$WORKSPACE/config
export YOLO_DIR=$WORKSPACE/src/yolov5Create a working directory for the workflow and clone the main repository into your working directory.
mkdir -p $WORKSPACE && cd $WORKSPACEgit clone https://github.com/oneapi-src/traffic-camera-object-detection.git $WORKSPACEmkdir -p $DATA_DIR $OUTPUT_DIR/models $OUTPUT_DIR/images $YOLO_DIRTo learn more, please visit install anaconda on Linux.
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
rm Miniconda3-latest-Linux-x86_64.shThe conda environment dependencies are kept in $WORKSPACE/env/intel_env.yml.
| Package | Version |
|---|---|
| python | 3.9 |
| opencv | 4.8.0 |
| intel-aikit-pytorch | 2024.0 |
| neural-compressor | 2.3.1 |
| torchvision | 0.15.2 |
| seaborn | 0.13.0 |
| tqdm | 4.66.1 |
| tensorboard | 2.15.1 |
| pillow | 10.0.1 |
| ultralytics | 8.0.227 |
| gitpython | 3.1.40 |
| pip | 23.3.1 |
| opencv-python-headless | 4.8.1.78 |
| thop | 0.1.1-2209072238 |
| ipython | 8.18.1 |
| openvino-dev[pytorch,onnx] | 2023.2.0 |
| onnx | 1.14.1 |
To install the conda environment follow the next instructions:
# If you have conda 23.10.0 or greater you can skip the following two lines
# since libmamba is already set as the default solver.
conda install -n base conda-libmamba-solver -y
conda config --set solver libmamba
conda env create -f $WORKSPACE/env/intel_env.ymlEnvironment setup is required only once. Make sure no conda environment exists with the same name since this step does not cleanup/overwrite the existing environment. During this setup a new conda environment will be created with the dependencies listed in the YAML file.
Once the appropriate environment is created, activate it using the conda command given below:
conda activate traffic_detection_intelData will be downloaded automatically through the script while running train.py; users can change the data download path by setting the value of $DATA_DIR. In order to download the datasets curl must be installed.
sudo apt install curl -yYou can execute the reference pipeline using the following environments:
- Bare Metal
Our examples use the conda package and environment on your local computer. If you don't already have conda installed or the conda environment created, go to Set Up Conda* or see the Conda* Linux installation instructions.
YOLOv5 is needed to run the workflow, download it by executing the following instructions:
git clone https://github.com/ultralytics/yolov5.git $YOLO_DIR
cd $YOLO_DIR
git reset --hard 63555c8e2230328585d09fdc50a6601822a70ded
# Intel® Extension for PyTorch training patch
git apply --reject --whitespace=fix $CONFIG_DIR/training.patch
# Copying required files to the cloned repo
cp $CONFIG_DIR/data/VOC.yaml $YOLO_DIR/data/
cp $CONFIG_DIR/deploy.yaml $YOLO_DIR
cp $WORKSPACE/src/run* $YOLO_DIR
cp -r $WORKSPACE/src/openvino $YOLO_DIR
#Changes default data download path from `../data/VOC` to `$DATA_DIR`.
sed -i "s+../data/VOC+$DATA_DIR+g" $YOLO_DIR/data/VOC.yamlTo start training run train.py python script inside src/yolov5. The script downloads the dataset, preprocesses it and runs the training routine. The trained model will be saved in output/models/train/exp{}/weights folder.
usage: train.py [-h] [--weights WEIGHTS] [--cfg CFG] [--data DATA] [--hyp HYP] [--epochs EPOCHS] [--batch-size BATCH_SIZE] [--imgsz IMGSZ]
[--rect] [--resume [RESUME]] [--nosave] [--noval] [--noautoanchor] [--noplots] [--evolve [EVOLVE]] [--bucket BUCKET]
[--cache [CACHE]] [--image-weights] [--device DEVICE] [--multi-scale] [--single-cls] [--optimizer {SGD,Adam,AdamW}]
[--sync-bn] [--workers WORKERS] [--project PROJECT] [--name NAME] [--exist-ok] [--quad] [--cos-lr]
[--label-smoothing LABEL_SMOOTHING] [--patience PATIENCE] [--freeze FREEZE [FREEZE ...]] [--save-period SAVE_PERIOD]
[--seed SEED] [--local_rank LOCAL_RANK] [--entity ENTITY] [--upload_dataset [UPLOAD_DATASET]]
[--bbox_interval BBOX_INTERVAL] [--artifact_alias ARTIFACT_ALIAS] [--bf16]
optional arguments:
-h, --help show this help message and exit
--weights WEIGHTS initial weights path
--cfg CFG model.yaml path
--data DATA dataset.yaml path
--hyp HYP hyperparameters path
--epochs EPOCHS total training epochs
--batch-size BATCH_SIZE
total batch size for all GPUs, -1 for autobatch
--imgsz IMGSZ, --img IMGSZ, --img-size IMGSZ
train, val image size (pixels)
--rect rectangular training
--resume [RESUME] resume most recent training
--nosave only save final checkpoint
--noval only validate final epoch
--noautoanchor disable AutoAnchor
--noplots save no plot files
--evolve [EVOLVE] evolve hyperparameters for x generations
--bucket BUCKET gsutil bucket
--cache [CACHE] image --cache ram/disk
--image-weights use weighted image selection for training
--device DEVICE cuda device, i.e. 0 or 0,1,2,3 or cpu
--multi-scale vary img-size +/- 50%
--single-cls train multi-class data as single-class
--optimizer {SGD,Adam,AdamW}
optimizer
--sync-bn use SyncBatchNorm, only available in DDP mode
--workers WORKERS max dataloader workers (per RANK in DDP mode)
--project PROJECT save to project/name
--name NAME save to project/name
--exist-ok existing project/name ok, do not increment
--quad quad dataloader
--cos-lr cosine LR scheduler
--label-smoothing LABEL_SMOOTHING
Label smoothing epsilon
--patience PATIENCE EarlyStopping patience (epochs without improvement)
--freeze FREEZE [FREEZE ...]
Freeze layers: backbone=10, first3=0 1 2
--save-period SAVE_PERIOD
Save checkpoint every x epochs (disabled if < 1)
--seed SEED Global training seed
--local_rank LOCAL_RANK
Automatic DDP Multi-GPU argument, do not modify
--entity ENTITY Entity
--upload_dataset [UPLOAD_DATASET]
Upload data, "val" option
--bbox_interval BBOX_INTERVAL
Set bounding-box image logging interval
--artifact_alias ARTIFACT_ALIAS
Version of dataset artifact to use
--bf16 Enable only on Intel® Fourth Gen Xeon, BF16An example of how to use the above command is provided in the following code block. It will automatically download the dataset and yolov5s.pt model, preprocess the dataset, run the training script for 10 epochs using the values stored in yolov5s.pt for fine-tunning and $YOLO_DIR/data/VOC.yaml as configuration parameters. Output models will be stored at $OUTPUT_DIR/models/train.
python $YOLO_DIR/train.py --weights $OUTPUT_DIR/models/yolov5s.pt --data $YOLO_DIR/data/VOC.yaml --epochs 10 --project $OUTPUT_DIR/models/traintrain.py script also includes a command line flag --bf16 that enables bf16 mixed precision training (on CPUs that support it) along with the optimizations.
The training process for Intel® Optimizations for PyTorch* along with bf16 mixed precision training can be enabled using the train.py script as:
python $YOLO_DIR/train.py --weights $OUTPUT_DIR/models/yolov5s.pt --data $YOLO_DIR/data/VOC.yaml --epochs 10 --project $OUTPUT_DIR/models/train --bf16Note: You can enable bf16 training by setting the bf16. Please note that this flag MUST be enabled only on Intel® Fourth Gen Xeon® Scalable processors codenamed Sapphire Rapids that has bf16 training support and optimizations to utilize AMX, the latest ISA introduced in this family of processors.
To run inference use the run_inference.py python script inside src/yolov5.
usage: run_inference.py [-h] [-c CONFIG] [-d DATA_YAML] [-b BATCHSIZE] [-w WEIGHTS] [-int8inc] [-qw QUANT_WEIGHTS] [-si]
[-sip SAVE_IMAGE_PATH]
optional arguments:
-h, --help show this help message and exit
-c CONFIG, --config CONFIG
Yaml file for quantizing model, default is "./deploy.yaml"
-d DATA_YAML, --data_yaml DATA_YAML
Absolute path to the data yaml file containing configurations
-b BATCHSIZE, --batchsize BATCHSIZE
batchsize for the dataloader....default is 1
-w WEIGHTS, --weights WEIGHTS
Model Weights ".pt" format
-int8inc Run INC quantization when present
-qw QUANT_WEIGHTS, --quant_weights QUANT_WEIGHTS
Quantization Model Weights folder containing ".pt" format model
-si, --save_image Save images in the specified save_image_path when present.
-sip SAVE_IMAGE_PATH, --save_image_path SAVE_IMAGE_PATH
Path to save images after post processing/ detected resultsAn example of how to use the inference script is provided in the following code block. The script can be run in an Intel® environment using different batch sizes -b:{1/8/16/32/64/128/256}. To set the output folder path to save the output images --save_image and --save_image_path {path/to/folder} must be specified.
python $YOLO_DIR/run_inference.py -c $CONFIG_DIR/deploy.yaml -d $YOLO_DIR/data/VOC.yaml -b 1 -w $OUTPUT_DIR/models/yolov5s.pt --save_image --save_image_path $OUTPUT_DIR/images/fp32Intel® Neural Compressor is used to quantize the FP32 Model to the INT8 Model. Optimized model is used here for evaluating and timing analysis.
Intel® Neural Compressor supports many optimization methods, for this case post-training quantization with the Accuracy Aware Mode method is used to quantize the FP32 model.
usage: run_inc_quantization.py [-h] [-o OUTPATH] [-c CONFIG] [-d DATA_YAML] [-w WEIGHTS]
optional arguments:
-h, --help show this help message and exit
-o OUTPATH, --outpath OUTPATH
absolute path to save quantized model. By default it will be saved in "./inc_compressed_model/output" folder
-c CONFIG, --config CONFIG
Yaml file for quantizing model, default is "./deploy.yaml"
-d DATA_YAML, --data_yaml DATA_YAML
Absolute path to the data yaml file containing configurations
-w WEIGHTS, --weights WEIGHTS
Model Weights ".pt" formatAn example of how to quantize trained models is provided in the following code block. Quantized model will be saved by default in $OUTPUT_DIR/models/inc_compressed_model/ folder as best_model.pt
python $YOLO_DIR/run_inc_quantization.py -o $OUTPUT_DIR/models/inc_compressed_model/ -c $CONFIG_DIR/deploy.yaml -d $YOLO_DIR/data/VOC.yaml -w $OUTPUT_DIR/models/yolov5s.ptAn example of how to run inference with quantize models is provided in the following code block.
python $YOLO_DIR/run_inference.py -c $CONFIG_DIR/deploy.yaml -d $YOLO_DIR/data/VOC.yaml -b 1 -w $OUTPUT_DIR/models/yolov5s.pt -int8inc -qw $OUTPUT_DIR/models/inc_compressed_model/best_model.pt --save_image --save_image_path $OUTPUT_DIR/images/int8When it comes to the deployment of this model on Edge devices, with less computing and memory resources, we further need to explore options for quantizing and compressing the model which brings out the same level of accuracy and efficient utilization of underlying computing resources. Intel® Distribution of OpenVINO* Toolkit facilitates the optimization of a deep learning model from a framework and deployment using an inference engine on such computing platforms based on Intel hardware accelerators. This section covers the steps to use this toolkit for model quantization and measure its performance.
Below script is used to convert FP32 model to ONNX model representation.
usage: convert_to_onnx.py [-h] [-o OUTPATH] [-w WEIGHTS] [-mname MODEL_NAME]
optional arguments:
-h, --help show this help message and exit
-o OUTPATH, --outpath OUTPATH
absolute path to save converted model. By default it will be saved in "./openvino/openvino_models/openvino_onnx" folder
-w WEIGHTS, --weights WEIGHTS
Model Weights in ".pt" format
-mname MODEL_NAME, --model_name MODEL_NAME
Name of the model to be created in ".onnx" format, default "TrafficOD"Here is an example of how to use convert_to_onnx.py:
python $YOLO_DIR/openvino/convert_to_onnx.py -o $OUTPUT_DIR/models/openvino_models/openvino_onnx -w $OUTPUT_DIR/models/yolov5s.ptThe converted model will be saved to the $OUTPUT_DIR/models/openvino_models/openvino_onnx in .onnx format.
Below command is used to convert the onnx model to OpenVINO* IR model format. mo has more arguments for the user to use, but for this case only --input_model and --output_dir are needed. For more options check --help.
mo --input_model <onnx model> --output_dir <output dir path to save the IR model>
arguments:
--input_model onnx model
--output_dir path of the folder to save the OpenVINO IR model formatThe above command will generate bin and xml files as output which can be used for OpenVINO* inference, with FP32 as default precision.
Here is an example of how to use mo:
mo --input_model $OUTPUT_DIR/models/openvino_models/openvino_onnx/TrafficOD_Onnx_Model.onnx --output_dir $OUTPUT_DIR/models/openvino_models/openvino_irPost-training Optimization Tool (POT) is designed to accelerate the inference of deep learning models by applying special methods without model retraining or fine-tuning. One such method is post-training quantization.
benchmark_app has more arguments for the user to use, but for this case only --modelpath is needed. For more options check --help.
benchmark_app -m <path of onnx model>
argument:
-m,--modelpath path of model in onnx formatThe example below is used to run the benchmark tool for the ONNX model generated.
benchmark_app -m $OUTPUT_DIR/models/openvino_models/openvino_onnx/TrafficOD_Onnx_Model.onnx -api async -niter 120 -nireq 1 -b 1 -nstreams 1 -nthreads 8 -hint noneBelow example is used to run the benchmark tool for the OpenVINO* IR model.
benchmark_app -m $OUTPUT_DIR/models/openvino_models/openvino_ir/TrafficOD_Onnx_Model.xml -api async -niter 120 -nireq 1 -b 1 -nstreams 1 -nthreads 8 -hint noneopenvino_quantization.py is used to convert OpenVINO* IR model to OpenVINO* INT8 model representation.
usage: openvino_quantization.py [-h] [-m FPIR_MODELPATH] [-o OUTPATH] [-d DATA_YAML] [-b BATCHSIZE]
optional arguments:
-h, --help show this help message and exit
-m FPIR_MODELPATH, --FPIR_modelpath FPIR_MODELPATH
FP32 IR Model absolute path without extension
-o OUTPATH, --outpath OUTPATH
default output quantized model will be save in path specified by outpath
-d DATA_YAML, --data_yaml DATA_YAML
Absolute path to the yaml file containing paths data/ download data
-b BATCHSIZE, --batchsize BATCHSIZE
batch size used for loading the dataRun the following example to run OpenVINO* quantization.
python $YOLO_DIR/openvino/openvino_quantization.py -o $OUTPUT_DIR/models/openvino_models/openvino_quantized -d $YOLO_DIR/data/VOC.yaml -b 1 -m $OUTPUT_DIR/models/openvino_models/openvino_ir/After running the above command, we can verify that bin, xml and mapping files (quantized model) got generated on $OUTPUT_DIR/models/openvino_models/openvino_quantized path.
Use the example below to run the benchmark tool for the Quantized OpenVINO* IR model.
benchmark_app -m $OUTPUT_DIR/models/openvino_models/openvino_quantized/torch_jit.xml -api async -niter 120 -nireq 1 -b 1 -nstreams 1 -nthreads 8 -hint noneBefore proceeding to the cleaning process, it is strongly recommended to make a backup of the data that the user wants to keep. To clean the previously downloaded and generated data, run the following commands:
conda activate base
conda env remove -n traffic_detection_intelrm -rf $DATA_DIR $OUTPUT_DIR $YOLO_DIRTo remove WORKSPACE:
rm -rf $WORKSPACEA successful execution of train.py should return similar results as shown below:
Epoch GPU_mem box_loss obj_loss cls_loss Instances Size
9/9 0G 0.03788 0.03211 0.01635 46 640: 100%|██████████| 1035/1035 [21:51<00:00, 1.27s/it]
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 155/155 [02:50<00:00, 1.10s/it]
all 4952 12032 0.723 0.68 0.74 0.459
10 epochs completed in 3.938 hours.
Optimizer stripped from /workspace/traffic-detection/output/models/train/exp/weights/last.pt, 14.8MB
Optimizer stripped from /workspace/traffic-detection/output/models/train/exp/weights/best.pt, 14.8MB
Validating /workspace/traffic-detection/output/models/train/exp/weights/best.pt...
Fusing layers...
Model summary: 157 layers, 7225885 parameters, 0 gradients, 16.4 GFLOPs
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 155/155 [02:33<00:00, 1.01it/s]
all 4952 12032 0.757 0.75 0.807 0.547
person 4952 4528 0.832 0.838 0.898 0.602
bicycle 4952 337 0.785 0.774 0.862 0.579
car 4952 1201 0.827 0.883 0.92 0.667
motorbike 4952 325 0.862 0.786 0.881 0.557
aeroplane 4952 285 0.897 0.786 0.885 0.562
bus 4952 213 0.78 0.878 0.92 0.724
train 4952 282 0.832 0.841 0.889 0.599
boat 4952 263 0.594 0.662 0.659 0.359
bird 4952 459 0.686 0.743 0.772 0.495
cat 4952 358 0.815 0.691 0.817 0.548
dog 4952 489 0.783 0.685 0.785 0.526
horse 4952 348 0.899 0.853 0.904 0.635
sheep 4952 242 0.718 0.822 0.849 0.628
cow 4952 244 0.736 0.77 0.828 0.59
bottle 4952 469 0.623 0.825 0.79 0.53
chair 4952 756 0.632 0.589 0.628 0.412
sofa 4952 239 0.689 0.653 0.717 0.535
pottedplant 4952 480 0.694 0.425 0.546 0.277
diningtable 4952 206 0.721 0.651 0.736 0.52
tvmonitor 4952 308 0.732 0.842 0.85 0.593
Results saved to /workspace/traffic-detection/output/models/train/expA successful execution of run_inference.py should return similar results as shown below:
Mean Average Precision for all images is 0.5856925404676505
Batch Size used here is 1
Average Inference Time Taken --> 0.016135698727872794 for images :: 1011A successful execution of run_inc_quantization.py should return similar results as shown below:
2023-12-22 01:11:12 [INFO] Tune 6 result is: [Accuracy (int8|fp32): 0.5543|0.5541, Duration (seconds) (int8|fp32): 158.9356|255.7827], Best tune result is: [Accuracy: 0.5543, Duration (seconds): 158.9356]
2023-12-22 01:11:12 [INFO] |***********************Tune Result Statistics**********************|
2023-12-22 01:11:12 [INFO] +--------------------+-----------+---------------+------------------+
2023-12-22 01:11:12 [INFO] | Info Type | Baseline | Tune 6 result | Best tune result |
2023-12-22 01:11:12 [INFO] +--------------------+-----------+---------------+------------------+
2023-12-22 01:11:12 [INFO] | Accuracy | 0.5541 | 0.5543 | 0.5543 |
2023-12-22 01:11:12 [INFO] | Duration (seconds) | 255.7827 | 158.9356 | 158.9356 |
2023-12-22 01:11:12 [INFO] +--------------------+-----------+---------------+------------------+
2023-12-22 01:11:12 [INFO] Save tuning history to /workspace/traffic-detection/src/yolov5/nc_workspace/2023-12-22_00-53-31/./history.snapshot.
2023-12-22 01:11:12 [INFO] Specified timeout or max trials is reached! Found a quantized model which meet accuracy goal. Exit.
2023-12-22 01:11:12 [INFO] Save deploy yaml to /workspace/traffic-detection/src/yolov5/nc_workspace/2023-12-22_00-53-31/deploy.yaml
2023-12-22 01:11:12 [INFO] Save config file and weights of quantized model to /workspace/traffic-detection/output/models/inc_compressed_model.
******************************
Succesfully Quantized model and saved at : /workspace/traffic-detection/output/models/inc_compressed_model/A successful execution of convert_to_onnx.py should return similar results as shown below:
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================A successful execution of mo for this workflow should return similar results as shown below:
[ INFO ] Generated IR will be compressed to FP16. If you get lower accuracy, please consider disabling compression explicitly by adding argument --compress_to_fp16=False.
Find more information about compression to FP16 at https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_FP16_Compression.html
[ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO* Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11.
Find more information about API v2.0 and IR v11 at https://docs.openvino.ai/2023.0/openvino_2_0_transition_guide.html
[ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /workspace/traffic-detection/output/models/openvino_models/openvino_ir/TrafficOD_Onnx_Model.xml
[ SUCCESS ] BIN file: /workspace/traffic-detection/output/models/openvino_models/openvino_ir/TrafficOD_Onnx_Model.binA successful execution of benchmark_app for this workflow should return similar results as shown below:
[Step 11/11] Dumping statistics report
[ INFO ] Execution Devices:['CPU']
[ INFO ] Count: 120 iterations
[ INFO ] Duration: 2144.58 ms
[ INFO ] Latency:
[ INFO ] Median: 17.50 ms
[ INFO ] Average: 17.72 ms
[ INFO ] Min: 17.06 ms
[ INFO ] Max: 29.55 ms
[ INFO ] Throughput: 55.95 FPSA successful execution of openvino_quantization.py should return similar results as shown below:
[2023-12-22 17:24:55][INFO] Step 1/9: Load the model
[2023-12-22 17:24:57][INFO] Step 2/9: Initialize the data loader
val: Scanning /workspace/traffic-detection/data/labels/test2007.cache... 4952 images, 0 backgrounds, 0 corrupt: 100%|██████████| 4952/4952 [00:00<?, ?it/s]
[2023-12-22 17:24:57][INFO] Step 3/9: Initialize the metric
[2023-12-22 17:24:57][INFO] Step 4/9: Initialize the engine for metric calculation and statistics collection
[2023-12-22 17:24:57][INFO] This will take time, please wait!
[2023-12-22 17:24:57][INFO] Step 5/9: Create a pipeline of compression algorithms
[2023-12-22 17:29:52][INFO] This will take time, please wait!
[2023-12-22 17:29:52][INFO] Step 6/9: Execute the pipeline
[2023-12-22 17:34:29][INFO] Step 7/9: Compress model weights quantized precision in" " order to reduce the size of final .bin file
[2023-12-22 17:34:32][INFO] Step 8/9: Save the compressed model and get the path to the model
[2023-12-22 17:34:36][INFO] The quantized model is stored in /workspace/traffic-detection/output/models/openvino_models/openvino_quantized/torch_jit.xml
[2023-12-22 17:34:36][INFO] Step 9 (Optional): Evaluate the original and compressed model. Print the results
[2023-12-22 17:37:11][INFO] MeanAP of the quantized model: 0.57575
[2023-12-22 17:37:11][INFO] MeanAP of the original model: 0.58256The ML pipeline can be break down into the following main tasks:
- Preprocessing (normalization and resizing) of VOC dataset using COCO classes.
- Set yolov5 weights for fine-tuning by using
yolov5s.ptpretrained model and run training using the preprocessed dataset. - Run yolov5 inference on a subset of images.
- The YOLO results are post processed; by using NMS and centroid-based distance calculation of detected objects possible collision can be detected.
- Provide a risk assessment as output.
This exercise for traffic camera object detection can be used as a reference implementation across similar use cases with Intel AI optimizations enabled to accelerate the E2E process.
For more information about or to read about other relevant workflow examples, see these guides and software resources:
The End-to-end Traffic Camera Object Detection team tracks both bugs and enhancement requests using GitHub issues. Before submitting a suggestion or bug report, see if your issue has already been reported.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
To the extent that any public or non-Intel datasets or models are referenced by or accessed using tools or code on this site those datasets or models are provided by the third party indicated as the content source. Intel does not create the content and does not warrant its accuracy or quality. By accessing the public content, or using materials trained on or with such content, you agree to the terms associated with that content and that your use complies with the applicable license.
Intel expressly disclaims the accuracy, adequacy, or completeness of any such public content, and is not liable for any errors, omissions, or defects in the content, or for any reliance on the content. Intel is not liable for any liability or damages relating to your use of public content.
*Names and brands that may be claimed as the property of others. Trademarks.