InferenceSize vs the input size

[Praso28]

Greetings!
I am working with YOLOv8 currently and encountered this particular problem. Our work requires small object detection for animals in wildlife as the surveillance system is placed far away either at cliffs or towers. This makes it really important to preserve the features at their respective pixels rather than downscaling and losing it.
But I was unable to comprehend the logical path flow of data; YOLO versions are trained on 640x640, now if I have exported it in default (i.e 640); it will downscale my given picture's resolution [1056,1600] and still infer on 640x640? Right? [Maintain aspect ratio and pad other area]
Fair enough, now I export it with imgsize:- [1056,1600]; and particularly make my input go through it. The results are a bit unsettling. (PFA)

I want to know whether forcing the inference size to a higher resolution suffers in the long run? I believe custom training on dataset of high resolution would serve our purpose more aptly.
Kindly go through the experiment I ran to try to understand this very thing:

1. took 20 images of high resolution(2 classes : bear and elephant), say (1056x1600) -> cropped 640x640 block which contained animal ; and passed on for training
2. ran for 200 epochs
3. made two models out of it - one at default inference and one at specifically 1056x1600 inference;
4. ran 5 different images of similar resolution , and also 2 images used in training (full picture of the cropped 640x640 image)

Image	Conf (640)	IoU (640)	Conf (1056x1600)	IoU (1056x1600)
test_2k_1.jpg	0.305	0.883	0	0
test_2k_2.jpg	0.426	0.943	0.644	0.624
test_2k_3.jpg	0.973	0.802	0.53	0.748
test_2k_4.jpg	0.953	0.873	0.58	0.576
test_2k_5.jpg	0.435	0.957	0.295	0.008
test_2k_6.jpg	0.988	0.866	0.916	0.887

[UltralyticsAssistant]

👋 Hello @Praso28, thank you for the detailed discussion and for sharing your experiment results and visuals 🚀 This is an automated response to help get things moving quickly, and an Ultralytics engineer will also assist you soon.

We recommend a visit to the Docs for new users, where you can find many Python and CLI usage examples and where many common questions may already be answered.

If this turns out to be a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

Since this appears related to custom training and inference behavior ❓, please provide as much information as possible, including:
-Your exact training command or script
-Your exact inference/export command or script
-Model variant used
-Dataset structure and a few example images
-Training logs and validation metrics
-Your current ultralytics version and environment details
-Any relevant export settings you used
-Confirmation that you are following our Tips for Best Training Results

Join the Ultralytics community where it suits you best. For real-time chat, head to Discord 🎧 Prefer in-depth discussions? Check out Discourse. Or dive into threads on our Subreddit to share knowledge with the community.

Upgrade

Upgrade to the latest ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8 to verify your issue is not already resolved in the latest version:

pip install -U ultralytics

Environments

YOLO may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

-Notebooks with free GPU:
-Google Cloud Deep Learning VM. See GCP Quickstart Guide
-Amazon Deep Learning AMI. See AWS Quickstart Guide
-Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLO Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[raimbekovm]

hi @Praso28, thanks for the detailed write-up.

on your first question - yes. with the default imgsz=640, a 1056x1600 image is letterboxed down to 640x640: aspect ratio is preserved and the rest is padded, so the image is not stretched. a list like imgsz=[1056,1600] (both already multiples of the stride 32) feeds the image at close to native size instead.

a few general points that usually explain this kind of result:

• a model detects objects best at roughly the pixel scale it saw during training. you trained at 640, so inference at 640 stays closest to that, while a much larger inference size shows the object at a very different scale than the model learned. so higher inference resolution is not automatically better - matching training and inference imgsz is usually the safer choice.
• if your real goal is small, far-away animals, the recommended approach is tiled inference with SAHI: it slices the high-res frame, detects on each tile, and merges the results, which preserves small-object detail without changing the scale the model expects.
• if you want to deploy at high resolution, train at that resolution too rather than only raising it at inference.
• it may also be worth trying YOLO26, which brings notable improvements in small-object recognition over earlier versions.
• finally, 20 images is a very small dataset, so individual numbers will be noisy - more varied data will help more than tuning imgsz.

see the predict arguments and training tips for details. results depend on your data and domain, but these are the patterns that usually apply here.

[Praso28]

Alright, I see the clear picture now and that all really makes sense. Yes, we are actually thinking of shifting to yolo26n for our usecase and see how the things unfold. SAHI seems to be a great technique for this, but I am a bit skeptic about its effect of on our edge deployment (jetson nano orin super 8gb) in terms of hardware usage.

Moreover, it would be really helpful if you can suggest some temporal model that can be equipped with our YOLO. The thing is animals move with a lot of stealth and camouflage sometimes. So what we need, is a motion detection to firstly locate that there might be something in the frame, then only either it goes for a super resolution patch and feed to YOLO or uses exact slicing like SAHI in that patch.

Does YOLO have any such extension which covers this application. Or does YOLO intend to do it in the multi-frame/spatial-temporal way in the near future?

Thanks for your guidance and time !!

[raimbekovm]

a few pointers for your setup. slicing with SAHI runs the model once per tile, so latency grows with the tile count. to keep that fast on the orin nano, export to TensorRT (FP16, or INT8 with calibration) instead of running the .pt directly - the jetson guide benchmarks yolo26n on the orin nano super at ~4.6 ms/image and notes the 8gb board is memory-limited, so a small model plus TensorRT plus slicing only when needed is the way to go.

the motion front-end is a good idea, but it sits outside YOLO since ultralytics doesnt ship a motion detector. the common pattern is a cheap stage like background subtraction (OpenCV MOG2/KNN) to flag where something moved, then run YOLO (optionally SAHI) only on that region. this also caps the SAHI cost, since you slice a small patch instead of the whole frame. between super-resolution and slicing for that patch, slicing is usually the safer choice on edge - SR is heavy and can invent detail, while slicing feeds the real pixels at the scale the model expects.

for the temporal part, the tool available today is multi-object tracking (BoT-SORT or ByteTrack) - detection per frame plus association across frames with persist=True. its not a multi-frame model, but it helps here, since ByteTrack can hold a track from low-confidence detections when an animal is briefly hidden. for a full decode to detect to track pipeline running on the jetson, see the deepstream guide. i cant comment on the roadmap, so no promises on multi-frame detection.

for wildlife specifically, the animal monitoring blog and the african wildlife dataset are good starting points.