Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TFLite] Fix resize bilinear tests by raising a too low error_threshold when align_corners is true #47009

Merged
merged 1 commit into from Feb 11, 2021

Conversation

Tessil
Copy link
Contributor

@Tessil Tessil commented Feb 8, 2021

Hi,

The '//tensorflow/lite/kernels/internal:resize_bilinear_test' is failing on our side with the following errors since commit 297ffd8:

tensorflow/lite/kernels/internal/resize_bilinear_test.cc:73: Failure
Expected: (relative_error) < (error_threshold), actual: 0.000531915 vs 0.0003
tensorflow/lite/kernels/internal/resize_bilinear_test.cc:73: Failure
Expected: (relative_error) < (error_threshold), actual: 0.000400641 vs 0.0003
tensorflow/lite/kernels/internal/resize_bilinear_test.cc:73: Failure
Expected: (relative_error) < (error_threshold), actual: 0.000674764 vs 0.0003
tensorflow/lite/kernels/internal/resize_bilinear_test.cc:73: Failure
 Expected: (relative_error) < (error_threshold), actual: 0.000558036 vs 0.0003
tensorflow/lite/kernels/internal/resize_bilinear_test.cc:73: Failure
Expected: (relative_error) < (error_threshold), actual: 0.000300481 vs 0.0003

The threshold when comparing the reference and optimized resize bilinear kernel with align_corners is too low. This PR raises it to 1e-3 as it seems the kernel is correct and the problem comes from the too tight threshold.

Thibaut

@google-ml-butler google-ml-butler bot added the size:XS CL Change Size: Extra Small label Feb 8, 2021
@google-cla google-cla bot added the cla: yes label Feb 8, 2021
@Tessil Tessil changed the title [TFLote] Fix resize bilinear tests by raising a too low error_threshold when align_corners is true [TFLite] Fix resize bilinear tests by raising a too low error_threshold when align_corners is true Feb 8, 2021
@abattery
Copy link
Contributor

abattery commented Feb 8, 2021

@Tessil Is this PR required because the current optimized kernels in TFLite have not succeeded with your system and configuration only? Or are you optimizing the kernels further?

If this failure happens at your environment only, could you share your testing environment's details to understand the problem better?

@abattery abattery self-requested a review February 8, 2021 20:14
@gbaned gbaned self-assigned this Feb 9, 2021
@gbaned gbaned added the comp:lite TF Lite related issues label Feb 9, 2021
@gbaned gbaned added this to Assigned Reviewer in PR Queue via automation Feb 9, 2021
@Tessil
Copy link
Contributor Author

Tessil commented Feb 10, 2021

It fails on our CI with the latest commit in the TF master branch without any further change.

It also fails locally on my desktop though the error is slightly different:

tensorflow/lite/kernels/internal/resize_bilinear_test.cc:73: Failure
Expected: (relative_error) < (error_threshold), actual: 0.000365497 vs 0.0003

The CI uses the tensorflow/tensorflow:custom-op-gpu-ubuntu16 docker image with USE_NEON enabled. My desktop uses g++ 7.5.0 without USE_NEON (Ubuntu 18.04).

Raising the kTestsToRun generates more similar errors on our side (can be useful for a quick test).

Copy link
Contributor

@abattery abattery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sharing the details. :)

@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Feb 10, 2021
PR Queue automation moved this from Assigned Reviewer to Approved by Reviewer Feb 10, 2021
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Feb 10, 2021
@gbaned gbaned added ready to pull PR ready for merge process and removed ready to pull PR ready for merge process labels Feb 11, 2021
@copybara-service copybara-service bot merged commit 30090de into tensorflow:master Feb 11, 2021
PR Queue automation moved this from Approved by Reviewer to Merged Feb 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes comp:lite TF Lite related issues ready to pull PR ready for merge process size:XS CL Change Size: Extra Small
Projects
PR Queue
  
Merged
Development

Successfully merging this pull request may close these issues.

None yet

4 participants