[Detector Support]: RTX 5060 Outdated/Unsupported TensorRT Version

[justbendev]

Describe the problem you are having

Hello 👋

After building Frigate (0.16.0 Beta 2) docker image with TensorRT support for an RTX 5060.
I unfortunately got an error while generating the .trt files (Error in the dedicated section)

After some research, i have deducted that the most probable cause was an unsupported Cuda/TensorRT Versions which i later confirmed with the content of docker/tensorrt/Dockerfile.amd64 that mention the use of TRT Version 8.6.1

From NVIDIA Tensor support matrix and later confirmed on NVIDIA Forums by AakankshaS (mod).

---

I will try on my end to create a compatible Dockerfile.amd64 when i have some free time and will add a pull request if not already done by someone else 👍

Have a great day 👋

Version

0.16.0 Beta 2 tag (TensorRT)

Frigate config file

detectors:
  tensorrt:
    type: tensorrt
    device: 0 #This is the default, select the first GPU

model:
  path: /config/model_cache/tensorrt/yolov7-320.trt
  labelmap_path: /labelmap/coco-80.txt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 320
  height: 320

docker-compose file or Docker CLI command

version: "3.9"
services:
  devfrigate:
    container_name: devfrigate
    privileged: true
    restart: unless-stopped
    image: devtest:0.16.0b2-tensorrt

    shm_size: "2048mb"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
    group_add:
      - video
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config
      - /storage:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "8971:8971"
      # - "5000:5000" # Internal unauthenticated access. Expose carefully.
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "REDACTED"
      NVIDIA_VISIBLE_DEVICES: all
      YOLO_MODELS: yolov7-320,yolov7x-640
      USE_FP16: false

Relevant Frigate log output

Creating yolov7-320.cfg and yolov7-320.weights
Creating yolov7x-640.cfg and yolov7x-640.weights

Done.

Generating yolov7-320.trt. This may take a few minutes.

/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:149: DeprecationWarning: Use network created with NetworkDefinitionCreationFlag::EXPLICIT_BATCH flag instead.
  builder.max_batch_size = MAX_BATCH_SIZE
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:151: DeprecationWarning: Use set_memory_pool_limit instead.
  config.max_workspace_size = 1 << 30
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:174: DeprecationWarning: Use build_serialized_network instead.
  engine = builder.build_engine(network, config)
ERROR: failed to build the TensorRT engine!
[06/06/2025-20:54:08] [TRT] [E] 2: [helpers.h::smVerHex2Dig::694] Error Code 2: Internal Error (Assertion major >= 0 && major < 10 failed. )
Loading the ONNX file...
Adding yolo_layer plugins.
Adding a concatenated output as "detections".
Naming the input tensort as "input".
Building the TensorRT engine.  This would take a while...
(Use "--verbose" or "-v" to enable verbose logging.)

Generating yolov7x-640.trt. This may take a few minutes.

Install method

Docker Compose

Object Detector

TensorRT

Screenshots of the Frigate UI's System metrics pages

N/A

Any other information that may be helpful

No response

[NickM-27]

Hi, 50 series GPUs are already supported with the ONNX detector.

I also would suggest not spending time on a PR as it currently looks like the TensorRT detector will be dropped in favor of only supporting the ONNX detector since it has better performance with less complication

[justbendev]

After seeing this "issue" being created in discussions instead i checked for similar thread about RTX 50XX and found #18169 [HW Accel Support]: Docker Compose and Installation with NVIDIA 5070

@hawkeye217 mentioned that moving to newer TensorRT Version would require alot of refactoring and suggested using ONNX Instead which is a valid solution for me so.

I will keep this open just in case but feel free to close it.