How to really use exported Tensorflow Lite model?

[pyro000]

❔Question

Following this tutorial: https://towardsdatascience.com/using-tensorflow-lite-for-object-detection-2a0283f94aed

I get with interpreter.get_tensor(output_details[1]['index'])[0] Index Error, because this model doesn't have labels or classes inside it.

Trying to understand detect.py and general.py seems really dificult for me.
Is there any introduction of how to display results with this model and include the labels from yolov5 with Python? or what source do I have to use to make this?

Additional context

Using detect.py works as intended but instead of naming the objects, it appears as "class0", "class20"...

[github-actions]

👋 Hello @pyro000, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov5
$ cd yolov5
$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[pyro000]

I have this colab notebook to show what I've done by now, if anybody can help me or link me anything useful I would be very grateful.

[glenn-jocher]

@pyro000 exported YOLOv5 models do not contain names arrays, only PyTorch models do. You can manually specify a names array in detect.py to replace the default names = ['class0', 'class1', ...]:

yolov5/detect.py

Line 78 in cce7e78

stride, names = 64, [f'class{i}' for i in range(1000)] # assign defaults

@zldrobit this issue seems to come up often, users are understandably confused when detection names are not shown with TFLite detect.py inference. Is there any way to embed names arrays in a TFLite model?

EDIT: @zldrobit I know previously you defaulted to COCO names, but users will arrive here with custom-trained models also.

[pyro000]

The problem is that with the example provided by the tutorial and the labels that I defined manually in labelmap.txt I keep getting the error IndexError: list index out of range in output_details[1] and above, when it is supposed to exist in the yolov5.tflite model.

Or am I missing something? Any help is really appreciated.

[glenn-jocher]

@pyro000 your link goes to a 3rd party that I don't recognize. The official YOLOv5 colab is
https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb

Running inference with TFLite models is very easy:

python export.py --weights yolov5s.pt --include tflite
python detect.py --weights yolov5s-fp16.tflite

[pyro000]

That's the code I did to make the predictions of the model in Python, loading the model with tf.lite.Interpreter, what I'm trying is to use a custom model made by yolov5, export it as tflite then use it with Tensorflow object detection API.

@pyro000 your link goes to a 3rd party that I don't recognize. The official YOLOv5 colab is https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb

Running inference with TFLite models is very easy:

python export.py --weights yolov5s.pt --include tflite
python detect.py --weights yolov5s-fp16.tflite

Yes I'm aware that with detect.py works perfectly but how do I make my own code only for tflite model? The methods in the TF docs doesn't work.

[glenn-jocher]

@pyro000 oh I see. I don't have experience with the TF OD API, I'm sorry. You should try reposting this issue on the https://github.com/tensorflow/models repository to see if you can get help there.

[pyro000]

Yes I just realised this is TF issue, I'll close this.

[zldrobit]

@glenn-jocher We may embed metadata in TFLite, e.g. https://www.tensorflow.org/lite/convert/metadata_writer_tutorial#object_detectors for object detection model. But this requires the TFLite model integrate with the NMS op. Considering the support of NMS in TFLite/TF.js, not all use cases could be covered by any type of the NMS ops. So I suggest we keep the NMS op in TFLite as an experimental branch for now.

[zldrobit]

@pyro000 After exported the TFLite model, you could use my fork repo to run the model on Android devices.

[glenn-jocher]

@zldrobit got it, thanks! Will leave this alone for now then.

[EdjeElectronics]

An example of how to use a yolov5.tflite model inside a TFLite application (besides just using detect.py) would be super useful. I'm also confused trying to figure how to deploy my TFLite model in my own application.

[glenn-jocher]

@EdjeElectronics We plan to release an example for using yolov5.tflite in a TFLite app soon. In the meantime, I recommend referring to the TFLite image classification example on the TensorFlow website as a starting point for TFLite deployment.