Skip to content

fix: match Python Ultralytics detection output for YOLO ONNX inference#97

Merged
onuralpszr merged 3 commits into
mainfrom
fix-miss-detection
Apr 10, 2026
Merged

fix: match Python Ultralytics detection output for YOLO ONNX inference#97
onuralpszr merged 3 commits into
mainfrom
fix-miss-detection

Conversation

@onuralpszr

@onuralpszr onuralpszr commented Apr 9, 2026

Copy link
Copy Markdown
Member
  • Use uniform gain for coordinate back-projection in letterbox params, matching Python's scale_boxes() which applies a single gain to both axes instead of per-axis scales that diverge after rounding

  • Fix bilinear interpolation to match OpenCV's INTER_LINEAR exactly: use single-pass fixed-point with rounding bias (ROUND_BIAS = 1 << 21) and OpenCV-matching weight computation (saturate_cast((1-fx)*2048))

  • Replace video-rs Decoder with custom BilinearVideoDecoder using raw ffmpeg-next API with SWS_BILINEAR flag for YUV→RGB conversion, avoiding video-rs's hardcoded SWS_AREA which can drift pixel values

  • Remove unused video_frame_to_image function

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

This PR improves preprocessing and video decoding consistency in ultralytics/inference 🎯, helping Rust inference outputs align more closely with OpenCV/Ultralytics behavior and reducing small prediction differences across images and videos.

📊 Key Changes

  • Updated image resizing and bilinear interpolation logic to better match OpenCV’s INTER_LINEAR behavior 🖼️
  • Added more precise fixed-point rounding during preprocessing, replacing the previous approximation-based path for resized pixels 🔍
  • Removed the SIMD-based 4-pixel resize loop in favor of a simpler single-pass fixed-point implementation ⚙️
  • Changed letterbox coordinate scaling to use one uniform gain for both axes, matching Ultralytics box rescaling behavior 📏
  • Introduced a custom FFmpeg-based video decoder that explicitly uses bilinear scaling for YUV-to-RGB conversion 🎥
  • Replaced the previous video-rs decoder path for videos with the new custom decoder, while preserving frame metadata like FPS and estimated total frames 📦
  • Cleaned up comments and messaging, including a small CLI warning text simplification ✨

🎯 Purpose & Impact

  • Improves output consistency between Rust inference and other Ultralytics/OpenCV-based pipelines, which is especially important for reproducible results ✅
  • Reduces small pixel-level differences that can cause borderline detections, box shifts, or slightly different NMS outcomes 🎯
  • Makes video predictions more stable by standardizing color conversion behavior during decoding 🎞️
  • Helps users get results that better match expectations from Python-based Ultralytics workflows 🤝
  • The preprocessing changes may slightly affect performance due to removal of the SIMD resize path, but they prioritize correctness and cross-platform consistency ⚖️
  • Overall, this should lead to more reliable and predictable inference results for both images and videos 🚀

- Use uniform gain for coordinate back-projection in letterbox params,
matching Python's scale_boxes() which applies a single gain to both axes
instead of per-axis scales that diverge after rounding

- Fix bilinear interpolation to match OpenCV's INTER_LINEAR exactly:
use single-pass fixed-point with rounding bias (ROUND_BIAS = 1 << 21)
and OpenCV-matching weight computation (saturate_cast<short>((1-fx)*2048))

- Replace video-rs Decoder with custom BilinearVideoDecoder using raw
ffmpeg-next API with SWS_BILINEAR flag for YUV→RGB conversion,
avoiding video-rs's hardcoded SWS_AREA which can drift pixel values

- Remove unused video_frame_to_image function

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
@UltralyticsAssistant UltralyticsAssistant added bug Something isn't working fixed Bug has been resolved labels Apr 9, 2026
@UltralyticsAssistant

Copy link
Copy Markdown
Member

👋 Hello @onuralpszr, thank you for submitting a ultralytics/inference 🚀 PR! This is an automated review message to help streamline the process, and an engineer will assist you soon. Your update appears focused on improving inference parity and consistency between Rust and Python behavior. 🛠️✨

To ensure a seamless integration of your work, please review the following checklist:

  • Define a Purpose: Clearly explain the purpose of your fix or feature in your PR description, and link to any relevant issues. Ensure your commit messages are clear, concise, and adhere to the project's conventions.
  • Synchronize with Source: Confirm your PR is synchronized with the ultralytics/inference main branch. If it's behind, update it by clicking the 'Update branch' button or by running git pull and git merge main locally.
  • Ensure CI Checks Pass: Verify all Ultralytics Continuous Integration (CI) checks are passing. If any checks fail, please address the issues.
  • Update Documentation: Update the relevant documentation for any new or modified features.
  • Add Tests: If applicable, include or update tests to cover your changes, and confirm that all tests are passing.
  • Sign the CLA: Please ensure you have signed our Contributor License Agreement if this is your first Ultralytics PR by writing "I have read the CLA Document and I sign the CLA" in a new message.
  • Minimize Changes: Limit your changes to the minimum necessary for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

For more guidance, please refer to our Contributing Guide. Don't hesitate to leave a comment if you have any questions. Thank you for contributing to Ultralytics! 🚀

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>

@UltralyticsAssistant UltralyticsAssistant left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 PR Review

Made with ❤️ by Ultralytics Actions

The preprocessing changes look coherent and the intent to match Python/OpenCV behavior is clear. The main issue is in the custom FFmpeg decoder: packet-send failures are currently swallowed in a way that can drop frames, and the new total_frames assignment regresses infinite stream handling by treating live streams as finite.

💬 Posted 2 inline comments

Comment thread src/source.rs
Comment thread src/source.rs
@codecov

codecov Bot commented Apr 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 17.79661% with 97 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/source.rs 0.00% 97 Missing ⚠️

📢 Thoughts on this report? Let us know!

@onuralpszr onuralpszr merged commit fbb5d2c into main Apr 10, 2026
10 checks passed
@onuralpszr onuralpszr deleted the fix-miss-detection branch April 10, 2026 08:14
@UltralyticsAssistant

Copy link
Copy Markdown
Member

Amazing to see this merged — thank you @onuralpszr, with contributions from @picsalex! 🎉

As Leonardo da Vinci said, “Simplicity is the ultimate sophistication.” This PR reflects that beautifully: by simplifying preprocessing and video decoding paths while improving correctness, you’ve helped ultralytics/inference produce more consistent, reliable results across images and videos.

Your work strengthens alignment with Ultralytics and OpenCV behavior, reduces hard-to-debug prediction differences, and makes the Rust inference experience more predictable for everyone. Really appreciate the care and craftsmanship behind this contribution — it will make a meaningful difference for users. 🙌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working fixed Bug has been resolved

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants