Inference Returns No Valid Results

[Tixtor710]

Issue: No class_name or predictions attribute in detections — Inference Returns No Valid Results

Summary

I'm running inference on a video using a Roboflow-trained model via their Python SDK, and although the video opens and processes frame by frame, no detections are returned, and I repeatedly get this warning:

Warning: No class_name or predictions attribute in detections on frame X.

This happens for every frame I've tested so far.

---

Setup

• Python Version: 3.x
• Roboflow SDK: Latest via pip
• Project ID: surgical-instrument-detection-cpjui/2
• Model Invocation: via model.infer(frame)[0]
• Video Input: "Robotic Lung Cancer Surgery Inside the OR (1).mp4"

---

Code Context

I'm using a Typer-based CLI that processes a video frame-by-frame using a Roboflow model and the supervision library:

results = model.infer(frame)[0]
detections = sv.Detections.from_inference(results)

The logic works fine structurally, and model.infer(frame) does return something—but results seems to be either empty or missing the expected keys like predictions or class_name.

---

What I’ve Tried

• Confirmed API Key and Model ID are correct: inference call does not raise an auth error.
• Debugged frame shape: Confirmed frames are being passed correctly from OpenCV in the format (H, W, 3).
• Printed results: results structure for the first frame is either:
• {} (empty dict), or
• Contains no predictions, class_name, or bounding box info.

---

Hypotheses

• The model is not properly trained or published, and Roboflow is serving a placeholder or null output.
• The model expects a specific input resolution, and frames are not resized accordingly.
• The video content is too dark / blurry / unsuitable for detection, though I highly doubt this given the video is a clean surgical OR recording.

---

Ask

Could someone on the Roboflow team or anyone experienced with the SDK confirm:

1. What the expected structure of model.infer(frame)[0] should look like?
2. Does the supervision.Detections.from_inference() method support this model output format?
3. Are there known cases where Roboflow SDK inference returns empty results silently?
4. Do I need to resize the frame to the model’s training resolution before inference?

Any pointers would be appreciated!

---

Reference Documentation for this process: https://inference.roboflow.com/quickstart/explore_models/#run-a-private-fine-tuned-model

My inference Script

import json
import os
from inference import get_model
import supervision as sv
import cv2
import typer

# Define the Typer app
app = typer.Typer()

@app.command()
def main(
    video_file: str = typer.Argument(..., help="Path to the input video file"),
    model_id: str = typer.Argument(..., help="Roboflow model ID (format: project-name/version-number)"),
    api_key: str = typer.Argument(..., help="Roboflow API key"),
    output_json: str = typer.Argument("detections.json", help="Path to the output JSON file"),
):
    """
    Process a video to detect objects and save their appearance times in a JSON file.
    Requires a Roboflow API key and a valid model ID.
    """
    # Check if video file exists
    if not os.path.isfile(video_file):
        typer.echo(f"Error: Video file '{video_file}' does not exist.")
        raise typer.Exit(code=1)

    # Load a pre-trained YOLO model
    try:
        model = get_model(model_id=model_id, api_key=api_key)
    except Exception as e:
        typer.echo(f"Error loading model '{model_id}': {str(e)}")
        typer.echo("Ensure the model ID and API key are correct and the API key has access to the project.")
        typer.echo("Visit https://docs.roboflow.com/api-reference/authentication#retrieve-an-api-key for API key details.")
        raise typer.Exit(code=1)

    # Open the video file
    cap = cv2.VideoCapture(video_file)
    if not cap.isOpened():
        typer.echo(f"Error: Could not open video file '{video_file}'.")
        raise typer.Exit(code=1)

    # Get video properties
    fps = cap.get(cv2.CAP_PROP_FPS)
    if fps == 0:
        cap.release()
        typer.echo("Error: Invalid FPS value. Ensure the video file is valid.")
        raise typer.Exit(code=1)

    # Dictionary to track class appearances
    # Format: {class_name: [{"start_time": start_time, "end_time": end_time or None}]}
    class_tracker = {}

    # Process video frame by frame
    frame_number = 0
    first_frame = True
    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break  # Exit when video ends

        # Calculate current time in seconds
        current_time = frame_number / fps

        # Run inference on the current frame
        try:
            results = model.infer(frame)[0]
        except Exception as e:
            typer.echo(f"Error during inference on frame {frame_number}: {str(e)}")
            cap.release()
            raise typer.Exit(code=1)

        # Debug: Print results structure for the first frame
        if first_frame:
            typer.echo(f"Debug: Results structure for frame 0: {results}")
            first_frame = False

        # Load the results into the supervision Detections API
        try:
            detections = sv.Detections.from_inference(results)
        except Exception as e:
            typer.echo(f"Error processing detections on frame {frame_number}: {str(e)}")
            cap.release()
            raise typer.Exit(code=1)

        # Get detected class names in the current frame
        current_classes = set()
        if hasattr(detections, "class_name"):
            current_classes = set(detections.class_name)
        elif hasattr(detections, "predictions") and detections.predictions:
            # Attempt to extract class names from predictions
            current_classes = set(pred.get("class") for pred in detections.predictions if pred.get("class"))
            if current_classes:
                typer.echo(f"Debug: Extracted class names from predictions: {current_classes}")
            else:
                typer.echo(f"Warning: No class names found in predictions on frame {frame_number}.")
        else:
            typer.echo(f"Warning: No class_name or predictions attribute in detections on frame {frame_number}.")
            # Continue processing even if no class names are found

        # Update class_tracker
        # Step 1: Update existing classes (close end_time if no longer detected)
        for class_name, instances in list(class_tracker.items()):
            for instance in instances:
                if instance["end_time"] is None and class_name not in current_classes:
                    instance["end_time"] = current_time

        # Step 2: Add new classes detected in this frame
        for class_name in current_classes:
            if class_name not in class_tracker:
                class_tracker[class_name] = [{"start_time": current_time, "end_time": None}]
            else:
                # Check if the last instance has end_time set (i.e., it disappeared)
                last_instance = class_tracker[class_name][-1]
                if last_instance["end_time"] is not None:
                    class_tracker[class_name].append({"start_time": current_time, "end_time": None})

        frame_number += 1

    # Close any open end_time values (end of video)
    final_time = frame_number / fps
    for class_name, instances in class_tracker.items():
        for instance in instances:
            if instance["end_time"] is None:
                instance["end_time"] = final_time

    # Release video capture
    cap.release()

    # Prepare JSON output
    json_output = []
    for class_name, instances in class_tracker.items():
        for instance in instances:
            json_output.append({
                "Class_name": class_name,
                "start_time": round(instance["start_time"], 2),
                "end_time": round(instance["end_time"], 2),
            })

    # Save to JSON file
    try:
        with open(output_json, "w") as f:
            json.dump(json_output, f, indent=4)
        typer.echo(f"Detections saved to {output_json}")
    except Exception as e:
        typer.echo(f"Error saving JSON to '{output_json}': {str(e)}")
        raise typer.Exit(code=1)

if __name__ == "__main__":
    app()

[Tixtor710]

I did get this when i tried printing out the results in line number 88:

2f04e96', parent_id=None)]
Detections(xyxy=array([[       1243,          40,        1392,         180],
       [       1812,           0,        1893,          41]]), mask=None, confidence=array([    0.96351,      0.5205]), class_id=array([17, 42]), tracker_id=None, data={'class_name': array(['car', 'person'], dtype='<U6')}, metadata={})
Warning: No class_name or predictions attribute in detections on frame 0.
Detections(xyxy=array([[       1257,          49,        1402,         189],
       [       1818,           0,        1896,          41]]), mask=None, confidence=array([     0.9741,      0.6984]), class_id=array([17, 42]), tracker_id=None, data={'class_name': array(['car', 'person'], dtype='<U6')}, metadata={})
Warning: No class_name or predictions attribute in detections on frame 1.
Detections(xyxy=array([[       1261,          51,        1418,         197],
       [       1816,           0,        1898,          42]]), mask=None, confidence=array([    0.85526,      0.6564]), class_id=array([17, 42]), tracker_id=None, data={'class_name': array(['car', 'person'], dtype='<U6')}, metadata={})
Warning: No class_name or predictions attribute in detections on frame 2.
Detections(xyxy=array([[       1270,          64,        1436,         212],
       [       1820,           0,        1897,          44]]), mask=None, confidence=array([    0.80483,     0.41103]), class_id=array([17, 42]), tracker_id=None, data={'class_name': array(['car', 'person'], dtype='<U6')}, metadata={})
Warning: No class_name or predictions attribute in detections on frame 3.
Detections(xyxy=array([[       1263,          75,        1436,         231],
       [       1826,           0,        1893,          50]]), mask=None, confidence=array([     0.7562,     0.60655]), class_id=array([17, 42]), tracker_id=None, data={'class_name': array(['car', 'person'], dtype='<U6')}, metadata={})
Warning: No class_name or predictions attribute in detections on frame 4.
Detections(xyxy=array([[       1270,          84,        1448,         244],
       [       1825,           0,        1893,          53]]), mask=None, confidence=array([    0.81067,     0.59217]), class_id=array([17, 42]), tracker_id=None, data={'class_name': array(['car', 'person'], dtype='<U6')}, metadata={})
Warning: No class_name or predictions attribute in detections on frame 5.

For this inference i used the COCO model: coco-128-qrc6j/4

[glenn-jocher]

Hi @Tixtor710,

Looking at your output, I can see the issue. Your detections are working correctly, but you're trying to access the class names incorrectly. In your output, the class names are stored in the data attribute of the Detections object:

data={'class_name': array(['car', 'person'], dtype='<U6')}

Instead of checking for hasattr(detections, "class_name"), you should modify your code to access the class names like this:

# Get detected class names in the current frame
current_classes = set()
if hasattr(detections, "data") and "class_name" in detections.data:
    current_classes = set(detections.data["class_name"])
elif hasattr(detections, "predictions") and detections.predictions:
    # Your existing fallback code
    ...

This should resolve your warning messages and properly capture the detected classes. If you'd like a more robust detection framework, you might consider using Ultralytics YOLO directly, which provides a more streamlined API for inference and results handling.