-
-
Notifications
You must be signed in to change notification settings - Fork 15.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How can I adjust the output of my TFLite exported model in order to make it work with the official TFLite android app #8874
Comments
I got passed this error by custom modifications to x = x[0][0] # [x(1,6300,85), ...] to x(6300,85)
xywh = x[..., :4] # x(6300,4) boxes
conf = x[..., 4:5] # x(6300,1) confidences
cls = tf.reshape(tf.cast(tf.argmax(x[..., 5:], axis=1), tf.float32), (-1, 1)) # x(6300,1) classes
xywh = tf.expand_dims(
xywh, 0
)
conf = K.permute_dimensions(conf, (1,0))
cls = K.permute_dimensions(cls, (1,0))
cls = tf.expand_dims(
cls, 0
)
return xywh, cls, conf, cls Now I get: java.lang.IllegalArgumentException: Error occurred when initializing ObjectDetector: Expected BoundingBoxProperties for tensor number of detections, found FeatureProperties.
at org.tensorflow.lite.task.vision.detector.ObjectDetector.initJniWithModelFdAndOptions(Native Method)
at org.tensorflow.lite.task.vision.detector.ObjectDetector.access$000(ObjectDetector.java:88)
at org.tensorflow.lite.task.vision.detector.ObjectDetector$1.createHandle(ObjectDetector.java:156)
at org.tensorflow.lite.task.vision.detector.ObjectDetector$1.createHandle(ObjectDetector.java:149)
at org.tensorflow.lite.task.core.TaskJniUtils$1.createHandle(TaskJniUtils.java:70) Any ideas? It seems like a metadata issue |
@mikel-brostrom I'm not sure. In our past conversations with Google they indicated their app was only meant for Google models unfortunately. |
Ok, I guess that is why it cannot find any helpful source related to my issue. It surprises me though that they keep the TFLite app limited to their own models. Thanks for your time again @glenn-jocher! Keep the good work up! |
Hi. Thanks ` @mikel-brostrom hints, I managed to integrate a Yolo5s custom model in the TensorFlow Lite Object Detection Android Demo
The fourth returned parameter should be the number of detections (and not the classes. This probably explains your error : "for tensor number of detections"). According to the ObjectDetector class definition, The boundingbox type should be BOUNDARIES (xyxy: upper_left_x, upper_left_y, width, height) and not CENTER (xywh: center_x, center_y, width, height (ref: metadata schema The tflite file metadata should be adapted accordingly (output_location_meta.content.contentProperties.index = [0, 1, 2, 3])
I can run my model on the app but I'm still testing the results and accuracy and I'm facing performance issue if I'm using the default export scripts ( |
👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs. Access additional YOLOv5 🚀 resources:
Access additional Ultralytics ⚡ resources:
Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed! Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐! |
Nice @dvigouro! What kind of performance issues are you experiencing? I am interested in knowing if it runs much faster than in other Yolov5 apps like: https://github.com/lp6m/yolov5s_android? |
👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs. Access additional YOLOv5 🚀 resources:
Access additional Ultralytics ⚡ resources:
Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed! Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐! |
@dvigouro can you give me the repo for the android app? i have convert the model to tflite models, but i got error when running on the official example android app |
Hi, I was wondering if there is any continuation of this current part of the discussion, I encountered the same problem with the Error output from tflite using the official |
@fdff87554 hi! If you're encountering issues with the TFLite export using the official git clone https://github.com/ultralytics/yolov5 # clone
cd yolov5
pip install -r requirements.txt # install If the problem persists, could you please provide more details about the error message and the steps you followed? This will help in diagnosing the issue more effectively. Thanks! |
Hi, glad to receive your reply. Let me quickly explain the current situation. I am trying to convert the customized trained model into TFLite format. One of the biggest problems encountered is the Missing of the Metadata of the TFLite Model. I had a quick overview of the I keep getting the following Error while reading a TFLite Model in my Android Application.
It was confirmed that the reason is that TFLite Model will contain metadata in a specific format, and additional location/category/score/number of detections information will be output during Output. However, when I looked at the tflite model exported by yolov5, there was no metadata in the model. But this is interesting, because I have seen that after the tflite model is generated, there is an --- [Updated] --- I have found that the model file after
And cause the following problems:
But the officially expected metadata should be as expected. Example .tflite file output
I carefully observed the source code and learned that compared with the official demonstration, there is no difference between the current |
Hi @fdff87554, thanks for the detailed explanation! It seems like the issue revolves around the metadata not being correctly populated in the TFLite model, which is crucial for compatibility with certain applications, like the TFLite Object Detection Android app. The metadata should indeed include detailed information about input/output tensors, including their types, expected content, and associated files for labels. The discrepancy you're seeing in the metadata format might be due to how the metadata is being attached in the Here’s a quick suggestion: You might need to manually adjust the metadata population script to ensure all necessary fields are correctly specified. This includes setting the right content properties for each tensor and ensuring the associated files (like label maps) are correctly linked. If you're comfortable modifying the script, you can try to explicitly define the metadata as per the structure you've found to be expected. If this sounds a bit complex or if you're unsure how to proceed, could you share the relevant portion of your Thanks for your patience, and looking forward to getting this resolved! 🚀 |
This is the current
To comply with android usage, the period should conform to the following format:
I don't have a clear direction on how to modify it at the moment, I'm experimenting but would love to see what you think. I just tried to make some simple adjustments, but it still doesn't meet expectations. I'd like to ask you to help me and take a look.
|
Hi @fdff87554, thanks for sharing the details and your efforts in adjusting the metadata! It looks like the metadata structure needs to be more explicitly defined to match the expected format for the TFLite Object Detection model. Here's a simplified approach to adjust the metadata in your
Here's a basic example to guide you on how to structure the metadata: from tflite_support import metadata as _metadata
from tflite_support import metadata_schema_py_generated as _metadata_fb
from tflite_support import flatbuffers
def create_metadata():
# Create model metadata.
model_meta = _metadata_fb.ModelMetadataT()
model_meta.name = "ObjectDetector"
model_meta.description = "Model to detect objects in an image."
model_meta.version = "v1"
model_meta.author = "YOLOv5 Team"
model_meta.license = "Public Domain"
# Create input tensor metadata.
input_meta = _metadata_fb.TensorMetadataT()
input_meta.name = "image"
input_meta.description = "Input image to be processed by the model"
input_meta.content = _metadata_fb.ContentT()
input_meta.content.contentProperties = _metadata_fb.ImagePropertiesT()
input_meta.content.contentProperties.colorSpace = _metadata_fb.ColorSpaceType.RGB
input_normalization = _metadata_fb.ProcessUnitT()
input_normalization.optionsType = _metadata_fb.ProcessUnitOptions.NormalizationOptions
input_normalization.options = _metadata_fb.NormalizationOptionsT()
input_normalization.options.mean = [127.5]
input_normalization.options.std = [127.5]
input_meta.processUnits = [input_normalization]
input_stats = _metadata_fb.StatsT()
input_stats.max = [255]
input_stats.min = [0]
input_meta.stats = input_stats
# Create output tensor metadata.
output_meta = _metadata_fb.TensorMetadataT()
output_meta.name = "detection_boxes"
output_meta.description = "Locations of detected objects"
output_meta.content = _metadata_fb.ContentT()
output_meta.content.contentProperties = _metadata_fb.BoundingBoxPropertiesT()
output_meta.content.contentProperties.index = [0, 1, 2, 3]
output_meta.content.contentProperties.type = _metadata_fb.BoundingBoxType.BOUNDARIES
output_meta.content.contentProperties.coordinateType = _metadata_fb.CoordinateType.RATIO
# Assign the metadata to the model.
subgraph = _metadata_fb.SubGraphMetadataT()
subgraph.inputTensorMetadata = [input_meta]
subgraph.outputTensorMetadata = [output_meta]
model_meta.subgraphMetadata = [subgraph]
b = flatbuffers.Builder(0)
b.Finish(model_meta.Pack(b), _metadata.MetadataPopulator.METADATA_FILE_IDENTIFIER)
metadata_buf = b.Output()
return metadata_buf
def add_metadata_to_tflite(file_path, metadata_buffer):
populator = _metadata.MetadataPopulator.with_model_file(file_path)
populator.load_metadata_buffer(metadata_buffer)
populator.populate()
# Usage
metadata_buffer = create_metadata()
add_metadata_to_tflite("path_to_your_model.tflite", metadata_buffer) This example should help you get started on structuring and applying the correct metadata to your TFLite model. Adjust the details as necessary to fit your specific model's requirements. Let me know if this helps or if you need further assistance! 🚀 |
Hi, I'm sorry for such a late reply. My partners and I are still gradually confirming the problem and correcting it, so we haven't concluded yet. We expect that after fixing all the problems (whether it is an Error on Android or various error messages on TFLite), we will compile a detailed correction process or report on the issues we continue to encounter in the future. Appreciate the assistance, please give us a little more time to confirm. |
Hi there, No worries about the delay! We appreciate your diligence in investigating the issue. It's great to hear that you and your partners are actively working on identifying and resolving the problems. To ensure we can assist you effectively, could you please provide a minimum reproducible code example? This will help us understand the exact context and reproduce the issue on our end. You can find more details on how to create one here: Minimum Reproducible Example. This step is crucial for us to investigate and provide a precise solution. Additionally, please make sure you are using the latest versions of Thank you for your patience and collaboration. We're here to help, so feel free to share any further details or questions you might have. Looking forward to your update! 😊 |
Search before asking
Question
I want to load my TFLite exported Yolov5s model into the official TFLite object detection android app (https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/android). The TFLite Yolov5 model outputs an array of shape
[1, 25200, 17]
.However, all the models (MobileNet SSD, EfficientDet Lite 0, EfficientDet Lite 1, EfficientDet Lite 2) in this app have 4 outputs:
detection_boxes
,detection_classes
,detection_scores
,num_detections
. According to https://www.tensorflow.org/lite/examples/object_detection/overview#output_signature.How should I modify:
yolov5/models/tf.py
Line 421 in 731a2f8
in order to make my Yolov5 model loadable in this app?
Additional
When loading my TFLite Yolov5 model I get:
which clearly states the issue in the output format
The text was updated successfully, but these errors were encountered: