-
Notifications
You must be signed in to change notification settings - Fork 223
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adopt changes from JNI for casting from float to decimal #10917
base: branch-24.08
Are you sure you want to change the base?
Conversation
The results look really close. The
Related Spark UT still fails, also close:
Performance test to cast 5000000 floats to 10 kinds of decimal types (in ms):
|
Should that case be acceptable as the relative error is just |
I've just updated my JNI code in NVIDIA/spark-rapids-jni#2078, adding some utilities functions by @pmattione-nvidia and it seems to fix all of our failed tests. @thirtiseven please run tests again on your machine. |
It passed Spark UT, personally I think it is good enough for this issue. If set eps = 0 in
Only diffs in last few digits. Some cases in double test:
There some cases like New performance:
|
And again some thoughts about the |
My solution in NVIDIA/spark-rapids-jni#2078 is adopted from the ongoing work of @pmattione-nvidia. Paul is working on even a better solution, so hopefully it will be able to fix the failures above. Let's wait for it. |
TBA
Depends on:
Closes #9682, and closes #10809.