Add support for baseline evaluation and model optimization

[nik1806]

This contribution add following capabilities:

• Convert a model to tflite format
• Evaluate the model in terms of - model size, MSE and inference time.
• It also provide support to optimize models using 10 strategy based on tensorflow frame work.
• Include additional dependencies in requirements.txt

For more in-depth details about concepts and direction - please refer to blog https://theroboticsclub.github.io/gsoc2022-Nikhil_Paliwal/gsoc/Coding-Period-Week-3/.

To get more information about supported commandline arguments:

python3 optimize_models.py --help

We get output:

usage: optimize_models.py [-h] [--data_dir DATA_DIR] [--preprocess PREPROCESS] [--data_augs DATA_AUGS] [--img_shape IMG_SHAPE]
                          [--batch_size BATCH_SIZE] [--model_path MODEL_PATH] [--model_name MODEL_NAME] [--learning_rate LEARNING_RATE]
                          [--eval_base EVAL_BASE] [--tech TECH]

options:
  -h, --help            show this help message and exit
  --data_dir DATA_DIR   Directory to find Data
  --preprocess PREPROCESS
                        preprocessing information: choose from crop/nocrop and normal/extreme
  --data_augs DATA_AUGS
                        Data Augmentations: 0=No / 1=Normal / 2=Normal+Weather changes
  --img_shape IMG_SHAPE
                        Image shape
  --batch_size BATCH_SIZE
                        Batch size
  --model_path MODEL_PATH
                        Path to directory containing pre-trained models
  --model_name MODEL_NAME
                        Name of model
  --learning_rate LEARNING_RATE
                        Learning rate
  --eval_base EVAL_BASE
                        If set to True, it will calculate accuracy, size and inference time for original model.
  --tech TECH           Techniques to apply for model compression. Options are: 'dynamic_quan', 'int_quan', 'int_flt_quan', 'float16_quan',
                        'quan_aware', 'prune', 'prune_quan' and 'all' .

Example execution command:

python3 optimize_models.py --data_dir ../datasets_opencv/ \
	--preprocess crop \
	--preprocess extreme \
	--data_augs 2 \
	--img_shape "200,66,3" \
	--batch_size 1024 \
	--model_path ../trained_models/pilotnet.h5 \
	--model_name pilotnet \
	--eval_base True \
 	--tech dynamic_quan 

[nik1806]

You can find the optimized models here - https://drive.google.com/drive/folders/1j2nnmfvRdQF5Ypfv1p3QF2p2dpNbXzkt?usp=sharing

[sergiopaniego]

Thanks for the PR!
It would be interesting to update the current README that you have modified https://github.com/nik1806/DeepLearningStudio/tree/main/Formula1-FollowLine/tensorflow with the direct link to download the model, so it's easier to understand

[sergiopaniego]

Another point to consider is that the models that I have tested on my end output warnings in the console. Could we remove these warnings? Otherwise, we need to understand the cause.
Example:

'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'
'Interpreter' object has no attribute 'layers'

Using pilotnet_pruned.tflite with updated brains/f1/brain_f1_keras_opencv_dataset.py

[sergiopaniego]

Update README with direct download links

[nik1806]

I have added the download links. I will make changes for 'Interpreter' object has no attribute 'layers' warning in the respective PR of Behavior Metric.