Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bindings float precision #81

Open
PPakalns opened this issue May 30, 2021 · 1 comment
Open

Bindings float precision #81

PPakalns opened this issue May 30, 2021 · 1 comment

Comments

@PPakalns
Copy link

PPakalns commented May 30, 2021

It looks like somewhere these bindings are adding error to float output.

When using native C++ onnxruntime API in the output for yolov3 onnx model I receive

Using python and native C++ onnxruntime API:

[[[     4.5295      4.3708      8.5376       11.65  2.2829e-05     0.98627]
  [     11.613      3.9123      22.815      9.5377  2.1607e-05     0.98834]
  [     19.874      4.0934      34.348      8.3545  1.0371e-05     0.98514]
  ...

Using onnxruntime-rs API:

[[[4.4851866, 4.155593, 8.946828, 12.582257, 0.0000130140525, 0.986418],
  [11.476209, 3.759369, 23.092682, 9.8777075, 0.000009529847, 0.9887239],
  [19.693684, 3.971115, 34.107414, 8.499779, 0.0000042610786, 0.9855219],
...

These results are approximately the same, but you can see the errors, which are pretty huge!

I even tried to compile onnxruntime with different backends and enable them with modifying this crate.

Tried download and system onnxruntimes, with and without GPU, with custom backend, etc.

@haixuanTao
Copy link

haixuanTao commented Jul 29, 2021

Hey, just for info, I've runned a custom Bert Model on onnxruntime using HF tokenizer on Python and Rust and the result are identical. I've runned it on a personal onnxruntime v1.8 with GPU which could be the reason of the fix.

My code if it can help: https://github.com/haixuanTao/bert-onnx-rs-pipeline

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants