Inference/postprocessing yolov8n on rockchip 3588

[ViktorPavlovA]

Search before asking

• I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

Hello, Yolo community .

I try to used yolov8n model on rockchip, but i miss understand how after inference used output's tensor. Can you answer ? What's order of tensors [x_c,y_c,w,h,confidence, ?, ?, ?] or it's right order.

import cv2
import numpy as np
from rknnlite.api import RKNNLite
import time

RKNN_MODEL = 'yolov8.rknn'
if __name__ == '__main__':
    rknn_lite = RKNNLite()
    
    ret = rknn_lite.load_rknn("yolov8.rknn")

    if ret != 0:
        print('Load RKNN model failed')
        exit(ret)
    
    ret = rknn_lite.init_runtime()
    if ret != 0:
        print('Init runtime environment failed')
        exit(ret)

    img_data = preprocess("test.jpg")
    outputs = rknn_lite.inference(inputs=[img_data])
    for id,i in enumerate(outputs):
        print(id)
        print(i.shape)
        #print(i[0])
        print("\n")
        print("_"*20)

OUTPUT

0
(1, 64, 80, 80)
____________________
1
(1, 80, 80, 80)
____________________
2
(1, 1, 80, 80)
____________________
3
(1, 64, 40, 40)
____________________
4
(1, 80, 40, 40)
____________________
5
(1, 1, 40, 40)
____________________
6
(1, 64, 20, 20)
____________________
7
(1, 80, 20, 20)
____________________
8
(1, 1, 20, 20)

Additional

rknn-lite-1.5.2

[github-actions]

👋 Hello @ViktorPavlovA, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

Hello!

It looks like you're working with the output tensors from the YOLOv8 model on a Rockchip RK3588. The output tensors you're seeing correspond to different scales of detection outputs, each containing bounding box (bbox) information and class predictions.

For YOLO models, the output typically includes:

• bbox attributes: x_center, y_center, width, height
• object confidence: indicating the confidence of detecting an object
• class probabilities: confidence scores for each class

The order in your output tensors likely follows this format: [x_center, y_center, width, height, obj_confidence, class1_prob, class2_prob, ..., classN_prob].

To process these outputs, you'll need to apply post-processing steps such as:

1. Thresholding: Filter out detections based on a confidence threshold.
2. Non-Max Suppression (NMS): Resolve overlapping bboxes that refer to the same object.

Here’s a simplified example of how you might start to interpret these outputs:

# Example to interpret the output tensor
for output in outputs:
    # Assuming the output shape is (1, num_predictions, attributes)
    predictions = output[0]
    for pred in predictions:
        x_center, y_center, width, height, conf, *class_probs = pred
        if conf > CONFIDENCE_THRESHOLD:
            class_id = np.argmax(class_probs)
            class_confidence = class_probs[class_id]
            if class_confidence > CLASS_CONFIDENCE_THRESHOLD:
                # This is a valid detection
                print(f"Detected class: {class_id} with bbox: {x_center}, {y_center}, {width}, {height}")

Adjust CONFIDENCE_THRESHOLD and CLASS_CONFIDENCE_THRESHOLD as needed for your application. This is a basic guideline, and you might need to adapt it based on the specific output format from your RKNN model.

Hope this helps! Let me know if you have more questions. 😊

[ViktorPavlovA]

Thank you for help and expain!

I write guide how to work with post processing on rockchip 3588. Maybe it will be useful for Yolo community.

[glenn-jocher]

Hello Viktor,

Thank you so much for sharing your guide on post-processing with YOLOv8 on the Rockchip 3588! 🌟 It's fantastic to see community members contributing valuable resources. I'm sure many will find it helpful. We appreciate your efforts in enriching the YOLO community. Keep up the great work!

[livelove1987]

Hi, my model has same shape and when using code provided in your comment: #13043 (comment)

Im getting following error:

Could be that reason that im using my custom trained model?

outputs = rknn.inference(inputs=[input_img])

for id,i in enumerate(outputs):
    print(id)
    print(i.shape)
    #print(i[0])
    print("\n")
    print("_"*20)

CONFIDENCE_THRESHOLD =0.5
CLASS_CONFIDENCE_THRESHOLD = 0.5

# Example to interpret the output tensor
for output in outputs:
    # Assuming the output shape is (1, num_predictions, attributes)
    predictions = output[0]
    for pred in predictions:
        x_center, y_center, width, height, conf, *class_probs = pred
        if conf > CONFIDENCE_THRESHOLD:
            class_id = np.argmax(class_probs)
            class_confidence = class_probs[class_id]
            if class_confidence > CLASS_CONFIDENCE_THRESHOLD:
                # This is a valid detection
                print(f"Detected class: {class_id} with bbox: {x_center}, {y_center}, {width}, {height}")
# Extract the output tensor and squeeze extra dimensions```

[ViktorPavlovA]

@livelove1987 , so right now is it works? About custom model: Nope, i also used custom model with script in readme.md and it worked.