New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix internal precision loss #10513
fix internal precision loss #10513
Conversation
Given the localized and private nature of those changes, I suppose this is a fine workaround, but in the long term we need to use the right word size for the data, even if it means that we will need to use synthetic coordinates and/or use more memory in webgl backend. |
@mattpap this has nothing to do with webgl, as we discussed previously, we are doing in all the mapping in non-webgl now, which means that webgl only sees things that fit in float32 (e.g. screen coords). More pointedly, this problem surfaced on non-webgl backends. The issue is specifically wrt to any 64-bit input data e.g timestamps, and their conversion to screen coordinates, which needs to be handled carefully enough to not lose precision internally along the way. I am happy to consider any general breaking changes for this for 3.0 (that is what #10379 is for hashing out). |
226976b
to
64a4ca4
Compare
@mattpap It think the BokehJS-CI tests just need update, the actual code from the issue in question looks correct: i.e. probably a bunch of fractional pixel diffs. |
Marked as ready. Happy to add a test but would need some guidance. |
* fix internal precision loss * use Float64Array internally * update integration tests images * regression test * lint * remaining baselines
* fix internal precision loss * use Float64Array internally * update integration tests images * regression test * lint * remaining baselines
* Backport: fix internal precision loss (#10513) * fix internal precision loss * use Float64Array internally * update integration tests images * regression test * lint * remaining baselines * Backport: Pin mypy (part of #10568) * Add 2.2.3 release notes * Add 2.2.3 to versions.json Co-authored-by: Bryan Van de Ven <bryan@bokeh.org>
I could not find any other CenterRotatable that needed similar treatment, and I verified that hover works with just these changes.
@mattpap any thoughts welcome:
NumberArray
when possible instead ofArray
for everything in these places?Rect._map_data
unit test that worked but did not succeed