fix(tflite): match PyTorch predict() preprocessing and mask decoder#1131
Merged
Conversation
omkar-334
requested review from
Borda,
SkalskiP,
isaacrob and
probicheaux
as code owners
June 18, 2026 23:39
Codecov Report❌ Patch coverage is ❌ Your project check has failed because the head coverage (81%) is below the target coverage (95%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #1131 +/- ##
=======================================
Coverage 81% 81%
=======================================
Files 108 108
Lines 11226 11269 +43
=======================================
+ Hits 9144 9184 +40
- Misses 2082 2085 +3 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes divergence between the TFLite inference helper and the PyTorch RFDETR.predict() / PostProcess.forward pipeline by aligning image preprocessing (resize + ImageNet normalization) and segmentation mask upsampling (half-pixel bilinear, align_corners=False) so TFLite outputs match PyTorch much more closely.
Changes:
- Adds
_preprocess_image()to standardize TFLite input preprocessing and avoid Pillow’s default resize filter mismatch. - Rewrites
_decode_masks()to use a centered half-pixel bilinear convention via a new_bilinear_resize_half_pixel()helper. - Updates
_run_inference()to use the new preprocessing path and pass NHWC batched input directly to the interpreter.
_decode_masks now calls torch.nn.functional.interpolate(..., mode="bilinear", align_corners=False) directly when torch is importable, falling back to the numpy half-pixel implementation for torch-free deployments. The torch path makes parity with PostProcess.forward tautological rather than a claim that has to be tested. Empty (K=0) input is guarded before the torch call so F.interpolate's non-empty-tensor assertion never trips. Also adds tests/export/test_tflite_inference_parity.py (eight regression tests for _preprocess_image): bit-exact assertion against torchvision in the standard environment, fallback-stays-close when torch is masked, a BICUBIC sentry test, plus unit-level checks for grayscale, NHWC, and ImageNet normalisation.
Adds two new tests in TestBilinearResizeHalfPixelParity exercising the torch-free fallback path against torch.nn.functional.interpolate with align_corners=False: - test_matches_torch_interpolate_on_random_logits: parametrised over upsample/downsample/identity/non-square shape transitions. Bound is 1e-4 to absorb float32 op-order noise on large ratios; half-pixel convention drift would push diffs several orders of magnitude higher. - test_sharp_edge_mask_matches_torch: the regression case. A mask with a sharp vertical zero-crossing is exactly the input shape that previously dropped mask IoU below 0.6 with PIL.BILINEAR. Asserts both numeric and post-threshold (boolean) equivalence to F.interpolate.
Wrap the broadcast result in np.asarray(dtype=np.float32) so mypy can see the declared NDArray[np.float32] return type instead of inferring Any from the chained float arithmetic.
Same pattern as the bilinear resize return: wrap the transpose result in np.asarray(dtype=np.float32) so mypy can see the declared return type.
- Fix no-any-return in _bilinear_resize_half_pixel and _preprocess_image: use np.asarray(..., dtype=np.float32) instead of .astype() on Any-typed expressions - Add torch-first path in _decode_masks via F.interpolate(align_corners=False); keep _bilinear_resize_half_pixel as ImportError fallback; guard K=0 tensors - Hoist _IMAGENET_MEAN/_IMAGENET_STD to module level; remove two inline duplicates - Add TestBilinearResizeHalfPixel: parity vs F.interpolate (4 parametrized cases), shape, dtype, identity tests - Add TestPreprocessImage: shape (RGB/grayscale), normalization, PIL fallback tests - Add Note docstrings to _bilinear_resize_half_pixel, _preprocess_image, _decode_masks --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
…334/rf-detr into fix-tflite-pytorch-parity
Borda
approved these changes
Jun 22, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
The TFLite inference helper produced inputs the PyTorch graph never saw, and decoded masks with a different half-pixel convention than
PostProcess.forward. After this change, FP32 TFLite is bit-exact to PyTorch for both object detection and segmentation. FP16 and dynamic-range int8 are within IoU 0.965 at production thresholds with zero class mismatches.The conversion graph itself was never broken: a same-input probe shows boxes/logits
max|diff| <= 0.0001between PyTorch and the existing FP32.tflite. Everything we were seeing came from the inference-side pre/post-processing being slightly off from the training-time pipeline.Changes
Two helpers added to
inference.pyto fix where the TFLite predict path diverges from PyTorch's._preprocess_image(extracted from_run_inference). The old code calledPIL.Image.resizewith no filter argument. PIL's default has been BICUBIC since Pillow 9.1, butRFDETR.predictresizes with BILINEAR viatorchvision.transforms.functional.resize. Same image, different input tensor, model sees something it wasn't trained on. The new helper usestorchvision.transforms.functional(to_tensor → resize → normalize) when torch is importable for byte-for-byte parity, and falls back toPIL.Image.resizewith explicit BILINEAR for torch-free deployments (still close to PyTorch, ~0.016 max residual)._decode_masks(rewritten) +_bilinear_resize_half_pixel(new fallback). The old_decode_masksdidPIL.Image.fromarray(..., "F").resize(..., BILINEAR)per mask. PIL's BILINEAR uses a corner-aligned half-pixel convention; PyTorch'sF.interpolate(..., mode="bilinear", align_corners=False)(whatPostProcess.forwardcalls) uses centered half-pixel. The shift is sub-pixel, but on thin or border-hugging masks it flipped enough pixels to drop mask IoU to ~0.5 even when the source logits matched. The rewrite callstorch.nn.functional.interpolatedirectly when torch is importable, and falls back to_bilinear_resize_half_pixel— a vectorised numpy implementation of centered half-pixel bilinear — when it isn't.Before vs after (apples-to-apples at threshold 0.3)
Object detection (RFDETRNano on dog/people-walking/bus/zidane, 43 detections total):
Related to #1024
Type of Change
Testing