Rtdetr

[Sarath-C-Koppolu]

Hi
I have trained my data with image size 960
And validated with 320 480 640 800 and 960 imgsz

I used v8 v9 v10 and rtdetr

All yolo models shows 320 image mAP around 30 percent

But rtdetr shows very less 2 percent.

What can be the possible explanation I should give when professor ask me

[pderrenger]

@Sarath-C-Koppolu hello,

Thank you for sharing your observations. The significant drop in mAP for RT-DETR at lower image sizes could be due to several factors:

1. Model Architecture Differences: RT-DETR uses a Vision Transformer-based architecture, which might be more sensitive to changes in image resolution compared to the YOLO models. Transformers typically require higher resolution inputs to maintain performance.

2. Training Image Size: Since you trained your model with an image size of 960, the model might not generalize well to significantly smaller image sizes like 320. This is especially true for RT-DETR, which might rely heavily on the detailed features present in higher resolution images.

3. Feature Extraction: RT-DETR's efficient hybrid encoder processes multiscale features. When the input image size is reduced, the model might lose crucial fine-grained details necessary for accurate detection.

4. Inference Speed vs. Accuracy Trade-off: RT-DETR is designed to balance real-time performance and accuracy. Reducing the image size might disproportionately affect its accuracy due to the model's reliance on detailed spatial information.

To address this with your professor, you can explain that RT-DETR's architecture and training specifics make it more sensitive to lower resolution inputs. For optimal performance, it's crucial to use image sizes closer to the training size or to retrain the model with a range of image sizes to improve its robustness across different resolutions.

If you need further assistance, please provide a reproducible example of your training and validation setup. This will help us diagnose the issue more accurately. You can refer to our Minimum Reproducible Example guide for more details.