implement Float32 without losing Float64 precision (rebased) #3681
+2,474
−1,109
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Compile Times benchmarkNote, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running: using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(fig)
|
SimonDanisch
added a commit
that referenced
this pull request
Mar 7, 2024
I noticed in #3681, that we dont actually have the exact statement from ttf precompile in here. This should be in here, since it should test if everything is cached and precompiled correctly. If we want to benchmark compile speed in general without caching, we would need to add some other benchmark, since without this statement, this is neither benchmarking pure precompiled performance, nor pure compilation time.
SimonDanisch
added a commit
that referenced
this pull request
Mar 7, 2024
* Add precompile used in ttf benchmark I noticed in #3681, that we dont actually have the exact statement from ttf precompile in here. This should be in here, since it should test if everything is cached and precompiled correctly. If we want to benchmark compile speed in general without caching, we would need to add some other benchmark, since without this statement, this is neither benchmarking pure precompiled performance, nor pure compilation time. * dont nightly
ffreyer
reviewed
Mar 14, 2024
Comment on lines
88
to
91
| function data_limits(p::HLines) | ||
| scene = parent_scene(p) | ||
| limits = projview_to_2d_limits(scene.camera.projectionview[]) | ||
| itf = inverse_transform(p.transformation.transform_func[]) | ||
| xmin, xmax = apply_transform.(itf[1], first.(extrema(limits))) | ||
| ymin, ymax = extrema(p[1][]) | ||
| return Rect3f(Point3f(xmin, ymin, 0), Vec3f(xmax - xmin, ymax - ymin, 0)) | ||
| return Rect3d(Point3d(NaN, ymin, 0), Vec3d(NaN, ymax - ymin, 0)) | ||
| end |
There was a problem hiding this comment.
Just to note - you can now ignore a dimension in limits by setting it to NaN
|
I noticed that scatter plots with large markersizes no longer set limits to the full marker. Is this intentional, and should we reverse it? MWE is: scatter(Point2f(0); markersize = 5, markerspace = :data)
|
This doesn't work on breaking-0.21 or master either, but I was thinking about adding it since it's the case for meshscatter |
ffreyer
reviewed
Mar 15, 2024
Comment on lines
+108
to
+142
| ################################################################################ | ||
|
|
||
| # TODO: remove after GeometryBasics#214 | ||
| const Point2d = Point2{Float64} | ||
| const Point3d = Point3{Float64} | ||
| const Point4d = Point4{Float64} | ||
| const Vec2d = Vec2{Float64} | ||
| const Vec3d = Vec3{Float64} | ||
| const Vec4d = Vec4{Float64} | ||
| const Rect2d = Rect2{Float64} | ||
| const Rect3d = Rect3{Float64} | ||
| const Rectd = Rect{N, Float64} where N | ||
| const Mat2d = Mat2{Float64} | ||
| const Mat3d = Mat3{Float64} | ||
| const Mat4d = Mat4{Float64} | ||
| export Point2d, Point3d, Point4d, Vec2d, Vec3d, Vec4d, Rect2d, Rect3d | ||
|
|
||
| # TODO: move to GeometryBasics? | ||
| function GeometryBasics.Rect3{T}(r::Rect2) where {T} # used in text boundingbox | ||
| return Rect3{T}(Vec3{T}(origin(r)..., zero(T)), Vec3{T}(widths(r)..., zero(T))) | ||
| end | ||
|
|
||
| function Base.convert(::Type{Rect{N, T}}, r::Rect{N}) where {N, T} | ||
| return Rect{N, T}(r) | ||
| end | ||
|
|
||
| # TODO: patch GridLayoutBase, probably to use Float64 consistently? | ||
| function GridLayoutBase.BBox(left::T1, right::T2, bottom::T3, top::T4) where {T1 <: Real, T2 <: Real, T3 <: Real, T4 <: Real} | ||
| mini = (left, bottom) | ||
| maxi = (right, top) | ||
| T = promote_type(T1, T2, T3, T4, Float32) # Float32 to skip Int outputs | ||
| return Rect2{T}(mini, maxi .- mini) | ||
| end | ||
|
|
||
| ################################################################################ |
There was a problem hiding this comment.
TODO: clean up before merge
There was a problem hiding this comment.
just registering gridlayoutbase 0.10.1 with your bbox change
|
I don't know why the benchmarks are suddenly so noisy... Locally, its quite significantly in favor of this PR for all backends :) Local results:
|
3 tasks
SimonDanisch
added a commit
that referenced
this pull request
May 8, 2024
* run CI against this PR * bump version * fix circular reference * remove GeoInterfaceMakie tests for good to avoid circular dependencies * add poly1 * Rework line shaders for WGLMakie compatability (#3558) * prototyping * fix joint with non-uniform linewidths * fix transform * add patterns * fix pattern sampling * fix truncated join cutoff + some cleanup * improve pattern overwrite threshhold * reorganize code * cleanup * add function for line vec -> line normal * don't try to cleanup joints if linewidth difference is critical * rename some variables * cleanup comments, restructure linewidth sdf generation * minor cleanup + notes * fix orientation problems with truncated joints * minor performance tweaks + cleanup * handle line joints in fragment shader & ditch variable linewidths * improve truncation overlap/gap * improve pattern adjustments for truncated joints * cleanup * reorganize code to reduce memory usage * mostly fix gap in sharp joints * explain pattern overwrite a bit more * use new fragment shader for linesegments * disable debug rendering * fix incorrect line placement * adjust line segments to pattern * use const over define & fix dots * tweak debug rendering * make truncation overlap a bit nicer * minor cleanup * fix linestyle in linesegments * skip rendering at 0 linewidth * add slight bias to avoid missing pixels * differentiate different segments in debug render * make color interpolation continuous at joint * fix dense line color artifacts * remove unused * allocate space for joints * use sdf for AA + debug render * add miter joints * handle colors * some cleanup + linesegments prep * fix some errors [skip ci] * add patterns * clean up linesegments * add normal rendering mode * minor cleanup * experiment with geometry adjustments * use rounding to solve joint over/underdraw * WebGL doesn't like scaling * smooth out inner edge of truncated join * remove derivative to reduce float accuracy issues * improve thin lines * fix lastlen error * match buffer sizes to number of drawn segments * avoid dissipation of lines at high point densities * calculate uv from quad_sdf1 * reduce number of interleaved buffers * fix disappearing grid lines * fix missing preprocessors * fix pattern artifacts in dense line sections * cleanup some variables * allow patterns to adjust to linewidth * fix AA at line end * fix nan handling * discard fully transparent pixels * fix patterns with nan, reset pattern on nan * fix size issues * extrude lines slightly to cleanly close them * fix pixel skipping? * cleanup patterns * remove linewidth from pattern based line adjustments * consider AA in shape factor * reset shape_factor if pattern changes segment shape * minor cleanup * apply changes to WGLMakie + cleanup * cleanup pattern overwrite * use the same AA_RADIUS in fragment shader * fix linelen transform * disable debug * fix line start/end AA * fix px_per_unit? * fix px_per_unit? * reduce line start/end extrusion * simplify extrusion and shape_factor * summarize breaking changes * move miter joint test * update docstring * minor cleanup * enable more tests * move gappy & friends to Makie * note change to linestyle scaling * cleanup * fix line start/end AA * remove global var * disable GeoMakie tests for now * improve transparency for WGLMakie * fix problems with 180° change in line direction * fix directionality * interpolate colormap in fragment shader * remove util.vert from line/segments --------- Co-authored-by: SimonDanisch <sdanisch@protonmail.com> * fix line inversion (#3651) * fix lines shader * cleanup debug code * fix rebase error * fix pattern connectivity * some cleanup * apply changes to WGLMakie * add test * fix CairoMakie too * update changelog * fix missing start/end segment * Update CHANGELOG.md * update comments --------- Co-authored-by: Simon <sdanisch@protonmail.com> * Documented / validated plot attributes (#3626) * add second `@recipe` method * rename function * implement recipe and keyword validation * fix some bugs * rename for clarity * change Scatter recipe * use argument symbols correctly * change Lines * change linesegments * do a bunch of plots more * fix text bugs * splice in internal attribute after mesh creation * add allowlist * fix text * add attribute deprecation mechanism * add explanation * remove `text` for GLMakie * fix contour volume attributes * add rasterize to allowlist * add `enable_depth` to volume * add `matcap` to mesh * set `fxaa = false` on scatter, lines, linesegments, text * move fxaa after mixins * fix keywords * remove markersize attribute * remove invalid markersize * add depthsorting to attributes for scatter * fix forwarded attributes * add absorption to volume * don't use `used_attributes` for datashader canvas convert * fix positional args * image * add better printing and rudimentary tests * make allowlist a tuple for fewer allocations * attribute names tuple instead of set * make deprecations also a tuple * avoid intermediate vector to speed up attribute creation * apply same optimization to blocks * implement mechanism to augment user docstring * delete manually written out attribute docs * use funcsym in docstring * incorporate attribute docs into docstring again * use equal * add changelog * introduce `DocumentedAttributes` with macro * add function to access `DocumentedAttributes` of a recipe * fix module mutation problem for now * move definition behind PlotType * return Attributes instead of Dict * convert wireframe recipe * close over reference * don't include prereleases in benchmark, failed with 1.11 alpha * stringify default expr * convert arrows * add docs * directly convert default exprs to strings * convert ablines * print wrong attribute in red * convert annotations * convert arc * remove attributes in arrows docstring * convert band * fix escaping issue * convert barplot * convert bracket * fix barplot usage in hist * convert contourf * typo * convert contour * fix waterfall * comment out arrows test * convert datashader * convert errorbar and rangebar * convert hvlines * convert hvspan * convert pie * convert rainclouds * convert scatterlines * convert series * convert spy * convert stairs * convert stem * convert streamplot * convert timeseries * convert tooltip * convert tricontourf * add transformation to allowlist * convert triplot * convert volumeslices * convert voronoiplot * convert waterfall * convert boxplot * convert crossbar * convert density * convert qqplot and qqnorm * convert ecdfplot * convert hexbin * convert hist * convert violin * improve error message printing * add docstrings for mutating functions and plot types * add missing docstrings * reconnect lines docstring * move functions out of macro * fix arrows example and adjust docs * put necessary functions back into macro --------- Co-authored-by: Simon <sdanisch@protonmail.com> * Add voxel plot type (#3527) * create voxel rendering prototype * enable lighting * prototype voxel id generation & color handling * add is_air attribute * prototype texture mapping * fix shader reloading * fix texture mapping * implement local updates * optimize render order (depthsorting = false) * add depthsorting = true * render z planes first * add lowclip and highclip * add refimg tests + some fixes * fix colorrange * fix local chunk update * handle colorrange more efficiently * handle voxel id data more efficiently * docstring & formatting * switch back to lrbt order for uvmap * add docs * try fix tests * fix show * fix test? * add missing dimensions * add arguments for placement and scale * allow Colon * add Colon() to local_update * minor cleanup * prototype WGLMakie version * add fallback in CairoMakie * add RPRMakie fallback * skip invisible voxels * fix typo * rename voxel -> voxels * update docs, fix placement * update news * fix Colorbar for voxels * enable tests * fix texture rotation * cleanup print * cleanup comment * generalize array access * debug WGLMakie * get voxels rendering in WGLMakie * fix texture mapping * activate tests * fix moving planes, cleanup prints * add unit tests * add gap attribute * tests & docs * mention potential issues with picking * fix WGLMakie picking * fix depthsorting/gap handling * switch to integer mod * fix render order * use RNG * fix 1.6 3d array syntax * fix refimage * Update CHANGELOG.md * fix julia 1.6 --------- Co-authored-by: Simon <sdanisch@protonmail.com> * implement Float32 without losing Float64 precision (rebased) (#3681) * implement float64 precision * remove merge conflicts * update CairoMakie * move some code around * simplify syntax * add non-Observable apply_transform_and_f32_conversion * clean up convert_arguments * update WGLMakie * fix tests * patch model to act after f32convert * fix function name * update project and plot_to_screen * fix patch_model * add first test * rename file * fix type after patch_model * fix ticks beyond limits * fix image and heatmap conversion * convert to float types * update test * Fix CairoMakie not precompiling missed a `Makie.` in the image recipe * update mesh converts * fix poly convert type * fix dict access * fix first test in WGLMakie * don't use triangle_mesh * fix incorrect convert_arguments * fix test * fix DataInspector error * add meshscatter + surface to tests, fix convert_arguments for poly * Fix docs + Makie tests * Fix 1.6!? * fix picking test * update DataInspector * fix indicator rotation for meshscatter in DataInspector * fix Vector{<: Integer}, Vector{Float32} -> Vector{Float32} [skip ci] * fix stackoverflow on mixed tuple types [skip ci] * fix tuples correctly [skip ci] * fix geom -> points eltype [skip ci] * fix geom -> PointBased output type * fix Rect2 -> points output type [skip ci] * fix PointBased mesh conversion * fix PointBased multi-linestrings type [skip ci] * update bezierpath * split up conversions * fix missing Points convert [skip ci] * fix CellGridBased types [skip ci] * always convert volumes to float32 * start adding type tests for every convert_arguments * fix tests * undo splitting conversions.jl * test and cleanup mesh conversions * fix docs * update & test Annotations * update & test arrows * update and test band and bracket * update & test errorbars and rangebars conversions * test & update series converts * make type stable on 1.6 * update & test remaining convert_arguments * fix & test model application * add tests for float32convert * at floatmin/max refimg test * fix typo * update text bbox test & add backtrace * avoid Float32 in data_limits * fix Polygon -> Bezierpath conversion * fix WGLMakie meshscatter model patching * fix Float64 normals * fix Float64 matrices in Voxels * test normal and uv types * fix for 1.6 * make new project method more complete * clean up some TODOs * restore text tests * restore Polar transform tests * avoid BBox for Axis limits * fix rectangle zoom, deprecate to_world with matrix inputs * fix 1.6 * fix Float64 latexstrings * fix hvlines, hvspan, errorbars, rangebars * fix & test ablines * fix other usage of projview_to_2d_limits * remove dublicated line * update changelog + cleanup [skip ci] * fix missing to_value * consider markersize and offset for scatter data_limits * fix tests * revert BBox -> Rect2d changes * get hist and barplot working --------- Co-authored-by: ffreyer <frederic481994@hotmail.de> Co-authored-by: Anshul Singhvi <anshulsinghvi@gmail.com> * Document conversion pipeline (#3719) * document conversion pipeline * fix doc build * fix block --------- Co-authored-by: SimonDanisch <sdanisch@protonmail.com> * Deprecate `rotations` for `rotation` for Scatter and MeshScatter (#3724) * deprecate `rotations` for `rotation` for Scatter and MeshScatter * one more rename * remove double conversion * fix cairomakie meshscatter * fix usage in docs * remove mat4 code again * fix CairoMakie rror * change one more rotations * fix WGLMakie * add changelog entry --------- Co-authored-by: ffreyer <frederic481994@hotmail.de> * fix merge * update scatter boundingbox (#3716) * implement scatter boundingbox with marker metrics * add tests * always consider marker rotations * update changelog [skip ci] * fix data_limits * fix some more issues * rotations -> rotation * Update boundingboxes.jl --------- Co-authored-by: Simon <sdanisch@protonmail.com> * Change how boundingbox(::Text) is deprecated (#3723) * change how boundingbox is deprecated * update changelog * fix test errors * fix docs errors and cleanup error message * fix type instability * fix function name * fix function name * fix boundingbox overwrites * update changelog * add types to function args maybe this allows boundingbox(::MyPlot, space) to work without ::Symbol... * fix typing * fully remove :world space * fix test * Various fixes for next release (#3731) * fix error for 2d meshscatter data_limits * avoid Rect3(::Rect2) constructor * allow :inspector_label * allow inspector_clear and inspector_hover too * fix rect zoom & cleanup to_world * move inspector attributes to default attributes * fix NaN handling in limits & deprecate _update_rect * Update CHANGELOG.md * Fix some issues with voxels (#3748) * fix single color * update to new attribute docs and hide internal attributes * fix missing voxel planes in Axis3 * use transformationmatrix * should be normalized * make view_direction optimization less error prone * Fix rare missing/duplicate pixels in truncated joint (#3794) * rework truncated joint discard * update WGLMakie * fix non-solid linestyle joints * Add linecaps and jointstyles (#3771) * add attributes * prototype linecap & linestyle in GLMakie * move code around, add comments * add capstyle for linesegments * update WGLMakie * revert change in padding of uncapped lines * make sure truncation can't trigger * update CairoMakie * add :bevel * make miter_limit adjustable * capstyle -> linecap * update changelog * add refimg tests * add example * fix rendering issue with bevel for continued lines * use named constants * add more space to test * consider miter_limit in CairoMakie as well * also enable refimg test * switch to angle based miter_limit * fix default * tweak tests * note change in default miter_limit * add new attributes to recipes * rename jointstyle -> joinstyle * update a few more jointstyles * Fix rare missing/duplicate pixels in truncated joint (#3794) * rework truncated joint discard * update WGLMakie * fix non-solid linestyle joints * improve truncated linecap a bit * regenerate wglmakie bundled * tweak shape_factor * restore file * try fix connected sphere * add debug refimgs * more testing * more testing * revert debugging * fix test? * Cleanup for breaking 0.21 (#3765) * cleanup temp defitions * finalize data_limits cleanup * GeometryBasics should probably be breaking * adjust other versions * use newly tagged GeometryBasics * use breaking gridlayoutbase * try updating registry * somehow this is needed?! * add to missing CI --------- Co-authored-by: SimonDanisch <sdanisch@protonmail.com> * Unit support for Axes & Recipes, a.k.a axis converts (#3226) * take over most of the work from #1347 * add typed argument conversion (#3565) * add typed argument conversion * fix volume * add function to get available conversions * make conversion apply more narrowly * more cleanly separate recursion in convert_arguments * clean up * allow to get axis before creating a plot * clean up * fix tests * bring back dim converts (axis_convert) * update tests * fix tests and work around conversion problems * fix WGLMakie * fix errors * clean up conversion pipeline * fix tests * add changelog entry * disable project run * improve performance slightly * might as well use array * tmp * wip * implement axis convert recursion * fix tests * fix datashader * fix datashader * move unitful integration * fix performance regression!? * fix merge & new date time improvements * fix scaling test * remove test false * clean up * converts shouldnt be here * move axis converts to scene * further refactor [skip ci] * finish refactor for AxisConversion type * allow limit setting and ticks * make tests less noisy * cleanup * clean up and fix unitful/date conversion * make sure all tests work correctly * remove rand * rename, clean up and make axis spec work * clean up and test new conversion pipeline * undo feature deletion, don't reintroduce Rect2f * be explicit about Volume Interval types * minor docstring cleanup * try to clarify new conversion docstrings * remove convert_arguments_typed in favor of types_for_plot_arguments * fix remaining bugs for conversion simplification * fix ticks not updating * fix specapi * fix qqnorm * clean up types_for_plot_arguments * fix tuple conversion * try to fix compile time regression * try to fix compile time regression * clean up and introduce expand_dimensions * fix #3655 and clean up convert_arguments + add tests * fix #3509 and add tests for * clean up observables and more docs * final rename * fix docs * cleanup * small clean up * small doc improvements * improve docs * fix docs * try relative link * try without .md * take out link * try fix --------- Co-authored-by: ffreyer <frederic481994@hotmail.de> * fix comment * Add wrap recipe tutorial (#3816) * [Docs] add wrap/redirect recipe tutorial * fix sentence * Update docs/tutorials/wrap-existing-recipe.md Co-authored-by: Anshul Singhvi <anshulsinghvi@gmail.com> * update * fix doc build * add changelog * fix preview --------- Co-authored-by: Moelf <proton@jling.dev> Co-authored-by: Anshul Singhvi <anshulsinghvi@gmail.com> * only forward attributes usable by recipe * make sharing of attributes explicit * ci * fix RPRMakie material * small fixes * fix RPRMakie * fix series with BezierPath * fix series conversions --------- Co-authored-by: Frederic Freyer <frederic481994@hotmail.de> Co-authored-by: Julius Krumbiegel <22495855+jkrumbiegel@users.noreply.github.com> Co-authored-by: Anshul Singhvi <anshulsinghvi@gmail.com> Co-authored-by: Moelf <proton@jling.dev>
This was referenced May 24, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cleaned up rebase of #3680, based against breaking-0.21 directly, instead of sd/axis-converts
TODO from #3663 & #3671
convert_argumentspipeline to either let numeric types pass through OR always convert them to Float32/Float64 as appropriateprojectfunctionsdata_limits,boundingboxto handle Real inputs, make outputs Float64or input based(mostly Streamline data_limits and boundingbox #3671)implement matrix adjustments for scalingimplement Float32 conversions in RPRMakiebase + offsettick formatting (e.g.1e9 + 1 .. 1e9 + 10)consider changingmodelback to a Float32 matrixlims[2] - lims[1] < min(eps(lims[1]), eps(lims[2]))test every plot that projects internally (errorbars, h/vspan/lines, bracket, ...)convert_argumentsexhaustively (i.e.@test convert_arguments(a, b) == c)reconsidermerge Generalize BBox jkrumbiegel/GridLayoutBase.jl#52BBox()changes (Do we need them?)Rect{N, Float32} <-> Rect{N, Float64}conversions (at the end as this is currently a nice way to catch issues)position_om_plotandDataInspectorboundingbox(::Scatter)TODO from testing
Breaking Changes:
data_limitsandboundingboxnow return Rect3d instead of Rect3fdata_limitsno longer considersscale!androtate!/ parts of the model matrixboundingboxnow considers the full model matrix and transform_functransform_funcis now expected to keep the input type or convert to Float64to_world(point, matrix, resolution)deprecated forto_world(scene, point)which handles float32convertprojview_to_2d_limitsto take a plot and output an observable (i.e. apply float32convert as part of the function)Related Issues
yscale=log10not working withlinesand small values #1670Unrelated fixes:
meshscatterobjects #3512, DataInspector bounding boxes inmeshscatterdo not transform correctly withrotations+markersize#3670