Skip to content

fix(tflite): match PyTorch predict() preprocessing and mask decoder#1131

Merged
Borda merged 9 commits into
roboflow:developfrom
omkar-334:fix-tflite-pytorch-parity
Jun 22, 2026
Merged

fix(tflite): match PyTorch predict() preprocessing and mask decoder#1131
Borda merged 9 commits into
roboflow:developfrom
omkar-334:fix-tflite-pytorch-parity

Conversation

@omkar-334

Copy link
Copy Markdown
Contributor

What does this PR do?

The TFLite inference helper produced inputs the PyTorch graph never saw, and decoded masks with a different half-pixel convention than PostProcess.forward. After this change, FP32 TFLite is bit-exact to PyTorch for both object detection and segmentation. FP16 and dynamic-range int8 are within IoU 0.965 at production thresholds with zero class mismatches.

The conversion graph itself was never broken: a same-input probe shows boxes/logits max|diff| <= 0.0001 between PyTorch and the existing FP32 .tflite. Everything we were seeing came from the inference-side pre/post-processing being slightly off from the training-time pipeline.

Changes

Two helpers added to inference.py to fix where the TFLite predict path diverges from PyTorch's.

_preprocess_image (extracted from _run_inference). The old code called PIL.Image.resize with no filter argument. PIL's default has been BICUBIC since Pillow 9.1, but RFDETR.predict resizes with BILINEAR via torchvision.transforms.functional.resize. Same image, different input tensor, model sees something it wasn't trained on. The new helper uses torchvision.transforms.functional (to_tensor → resize → normalize) when torch is importable for byte-for-byte parity, and falls back to PIL.Image.resize with explicit BILINEAR for torch-free deployments (still close to PyTorch, ~0.016 max residual).

_decode_masks (rewritten) + _bilinear_resize_half_pixel (new fallback). The old _decode_masks did PIL.Image.fromarray(..., "F").resize(..., BILINEAR) per mask. PIL's BILINEAR uses a corner-aligned half-pixel convention; PyTorch's F.interpolate(..., mode="bilinear", align_corners=False) (what PostProcess.forward calls) uses centered half-pixel. The shift is sub-pixel, but on thin or border-hugging masks it flipped enough pixels to drop mask IoU to ~0.5 even when the source logits matched. The rewrite calls torch.nn.functional.interpolate directly when torch is importable, and falls back to _bilinear_resize_half_pixel — a vectorised numpy implementation of centered half-pixel bilinear — when it isn't.

Before vs after (apples-to-apples at threshold 0.3)

Object detection (RFDETRNano on dog/people-walking/bus/zidane, 43 detections total):

Mode Metric Before After Note
FP32 IoU_worst 0.640 1.000 the dog image was the regression source
FP32 conf_max_diff 0.129 0.000
FP32 class_mismatches 1 0
FP16 IoU_worst 0.406 0.996 the IoU_worst FP16 disaster
FP16 conf_max_diff 0.167 0.012
FP16 class_mismatches 2 0
dyn_int8 IoU_mean 0.979 0.99 small lift; int8 was already weight-only

Related to #1024

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Testing

  • I have tested this change locally
  • I have added/updated tests for this change

@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.91525% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 81%. Comparing base (04200cf) to head (91cdb5d).
⚠️ Report is 5 commits behind head on develop.

❌ Your project check has failed because the head coverage (81%) is below the target coverage (95%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #1131   +/-   ##
=======================================
  Coverage       81%     81%           
=======================================
  Files          108     108           
  Lines        11226   11269   +43     
=======================================
+ Hits          9144    9184   +40     
- Misses        2082    2085    +3     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes divergence between the TFLite inference helper and the PyTorch RFDETR.predict() / PostProcess.forward pipeline by aligning image preprocessing (resize + ImageNet normalization) and segmentation mask upsampling (half-pixel bilinear, align_corners=False) so TFLite outputs match PyTorch much more closely.

Changes:

  • Adds _preprocess_image() to standardize TFLite input preprocessing and avoid Pillow’s default resize filter mismatch.
  • Rewrites _decode_masks() to use a centered half-pixel bilinear convention via a new _bilinear_resize_half_pixel() helper.
  • Updates _run_inference() to use the new preprocessing path and pass NHWC batched input directly to the interpreter.

Comment thread src/rfdetr/export/_tflite/inference.py
Comment thread src/rfdetr/export/_tflite/inference.py
omkar-334 and others added 7 commits June 19, 2026 19:05
_decode_masks now calls torch.nn.functional.interpolate(..., mode="bilinear",
align_corners=False) directly when torch is importable, falling back to the
numpy half-pixel implementation for torch-free deployments. The torch path
makes parity with PostProcess.forward tautological rather than a claim that
has to be tested. Empty (K=0) input is guarded before the torch call so
F.interpolate's non-empty-tensor assertion never trips.

Also adds tests/export/test_tflite_inference_parity.py (eight regression
tests for _preprocess_image): bit-exact assertion against torchvision in
the standard environment, fallback-stays-close when torch is masked, a
BICUBIC sentry test, plus unit-level checks for grayscale, NHWC, and
ImageNet normalisation.
Adds two new tests in TestBilinearResizeHalfPixelParity exercising the
torch-free fallback path against torch.nn.functional.interpolate with
align_corners=False:

- test_matches_torch_interpolate_on_random_logits: parametrised over
  upsample/downsample/identity/non-square shape transitions. Bound is
  1e-4 to absorb float32 op-order noise on large ratios; half-pixel
  convention drift would push diffs several orders of magnitude higher.
- test_sharp_edge_mask_matches_torch: the regression case. A mask with
  a sharp vertical zero-crossing is exactly the input shape that
  previously dropped mask IoU below 0.6 with PIL.BILINEAR. Asserts both
  numeric and post-threshold (boolean) equivalence to F.interpolate.
Wrap the broadcast result in np.asarray(dtype=np.float32) so mypy can
see the declared NDArray[np.float32] return type instead of inferring Any
from the chained float arithmetic.
Same pattern as the bilinear resize return: wrap the transpose result in
np.asarray(dtype=np.float32) so mypy can see the declared return type.
- Fix no-any-return in _bilinear_resize_half_pixel and _preprocess_image: use
  np.asarray(..., dtype=np.float32) instead of .astype() on Any-typed expressions
- Add torch-first path in _decode_masks via F.interpolate(align_corners=False);
  keep _bilinear_resize_half_pixel as ImportError fallback; guard K=0 tensors
- Hoist _IMAGENET_MEAN/_IMAGENET_STD to module level; remove two inline duplicates
- Add TestBilinearResizeHalfPixel: parity vs F.interpolate (4 parametrized cases),
  shape, dtype, identity tests
- Add TestPreprocessImage: shape (RGB/grayscale), normalization, PIL fallback tests
- Add Note docstrings to _bilinear_resize_half_pixel, _preprocess_image, _decode_masks

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
@Borda Borda added the bug Something isn't working label Jun 22, 2026
@Borda
Borda merged commit da3e31e into roboflow:develop Jun 22, 2026
26 checks passed
@Borda Borda mentioned this pull request Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants