Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breaking 0.21 release #3650

Merged
merged 47 commits into from
May 8, 2024
Merged

Breaking 0.21 release #3650

merged 47 commits into from
May 8, 2024

Conversation

@ffreyer
Copy link
Collaborator

ffreyer commented Feb 23, 2024

Also #3527

@MakieBot
Copy link
Collaborator

MakieBot commented Feb 23, 2024

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(fig)
using create display create display
GLMakie 4.25s (4.19, 4.44) 0.09+- 108.71ms (104.75, 118.18) 4.82+- 477.05ms (469.67, 495.07) 8.43+- 7.67ms (7.42, 8.13) 0.24+- 25.51ms (25.34, 26.15) 0.29+-
master 3.89s (3.83, 4.07) 0.08+- 163.50ms (160.72, 167.53) 2.71+- 378.10ms (374.50, 386.98) 5.58+- 7.24ms (7.14, 7.31) 0.08+- 25.40ms (25.24, 25.56) 0.11+-
evaluation 0.91x slower❌, 0.36s (4.17d, 0.00p, 0.09std) 1.50x faster✅, -54.79ms (-14.01d, 0.00p, 3.77std) 0.79x slower❌, 98.95ms (13.84d, 0.00p, 7.01std) 0.94x slower❌, 0.44ms (2.48d, 0.00p, 0.16std) 1.00x invariant, 0.11ms (0.51d, 0.37p, 0.20std)
CairoMakie 3.69s (3.58, 3.85) 0.10+- 109.49ms (105.51, 115.83) 3.36+- 132.20ms (127.13, 141.14) 5.02+- 7.89ms (7.61, 8.11) 0.18+- 978.97μs (970.70, 989.30) 6.33+-
master 3.33s (3.23, 3.41) 0.07+- 170.51ms (163.37, 182.92) 6.57+- 104.19ms (98.77, 112.14) 4.48+- 7.45ms (7.20, 7.69) 0.18+- 895.75μs (889.89, 903.05) 5.22+-
evaluation 0.90x slower❌, 0.36s (4.31d, 0.00p, 0.08std) 1.56x faster✅, -61.02ms (-11.70d, 0.00p, 4.96std) 0.79x slower❌, 28.02ms (5.89d, 0.00p, 4.75std) 0.94x slower❌, 0.44ms (2.41d, 0.00p, 0.18std) 0.91x slower❌, 83.22μs (14.35d, 0.00p, 5.77std)
WGLMakie 4.46s (4.43, 4.53) 0.03+- 108.87ms (107.64, 111.57) 1.35+- 8.86s (8.78, 8.92) 0.05+- 10.08ms (9.88, 10.20) 0.11+- 71.04ms (70.41, 72.16) 0.62+-
master 3.99s (3.85, 4.14) 0.13+- 167.40ms (164.60, 172.85) 3.02+- 8.65s (8.52, 8.80) 0.11+- 9.40ms (9.33, 9.45) 0.04+- 69.05ms (67.76, 73.59) 2.08+-
evaluation 0.89x slower❌, 0.48s (4.97d, 0.00p, 0.08std) 1.54x faster✅, -58.54ms (-25.04d, 0.00p, 2.18std) 0.98x slower X, 0.21s (2.41d, 0.00p, 0.08std) 0.93x slower❌, 0.68ms (8.36d, 0.00p, 0.07std) 0.97x slower X, 1.99ms (1.30d, 0.05p, 1.35std)

SimonDanisch and others added 8 commits February 27, 2024 10:45
* prototyping

* fix joint with non-uniform linewidths

* fix transform

* add patterns

* fix pattern sampling

* fix truncated join cutoff + some cleanup

* improve pattern overwrite threshhold

* reorganize code

* cleanup

* add function for line vec -> line normal

* don't try to cleanup joints if linewidth difference is critical

* rename some variables

* cleanup comments, restructure linewidth sdf generation

* minor cleanup + notes

* fix orientation problems with truncated joints

* minor performance tweaks + cleanup

* handle line joints in fragment shader & ditch variable linewidths

* improve truncation overlap/gap

* improve pattern adjustments for truncated joints

* cleanup

* reorganize code to reduce memory usage

* mostly fix gap in sharp joints

* explain pattern overwrite a bit more

* use new fragment shader for linesegments

* disable debug rendering

* fix incorrect line placement

* adjust line segments to pattern

* use const over define & fix dots

* tweak debug rendering

* make truncation overlap a bit nicer

* minor cleanup

* fix linestyle in linesegments

* skip rendering at 0 linewidth

* add slight bias to avoid missing pixels

* differentiate different segments in debug render

* make color interpolation continuous at joint

* fix dense line color artifacts

* remove unused

* allocate space for joints

* use sdf for AA + debug render

* add miter joints

* handle colors

* some cleanup + linesegments prep

* fix some errors [skip ci]

* add patterns

* clean up linesegments

* add normal rendering mode

* minor cleanup

* experiment with geometry adjustments

* use rounding to solve joint over/underdraw

* WebGL doesn't like scaling

* smooth out inner edge of truncated join

* remove derivative to reduce float accuracy issues

* improve thin lines

* fix lastlen error

* match buffer sizes to number of drawn segments

* avoid dissipation of lines at high point densities

* calculate uv from quad_sdf1

* reduce number of interleaved buffers

* fix disappearing grid lines

* fix missing preprocessors

* fix pattern artifacts in dense line sections

* cleanup some variables

* allow patterns to adjust to linewidth

* fix AA at line end

* fix nan handling

* discard fully transparent pixels

* fix patterns with nan, reset pattern on nan

* fix size issues

* extrude lines slightly to cleanly close them

* fix pixel skipping?

* cleanup patterns

* remove linewidth from pattern based line adjustments

* consider AA in shape factor

* reset shape_factor if pattern changes segment shape

* minor cleanup

* apply changes to WGLMakie + cleanup

* cleanup pattern overwrite

* use the same AA_RADIUS in fragment shader

* fix linelen transform

* disable debug

* fix line start/end AA

* fix px_per_unit?

* fix px_per_unit?

* reduce line start/end extrusion

* simplify extrusion and shape_factor

* summarize breaking changes

* move miter joint test

* update docstring

* minor cleanup

* enable more tests

* move gappy & friends to Makie

* note change to linestyle scaling

* cleanup

* fix line start/end AA

* remove global var

* disable GeoMakie tests for now

* improve transparency for WGLMakie

* fix problems with 180° change in line direction

* fix directionality

* interpolate colormap in fragment shader

* remove util.vert from line/segments

---------

Co-authored-by: SimonDanisch <sdanisch@protonmail.com>
* fix lines shader

* cleanup debug code

* fix rebase error

* fix pattern connectivity

* some cleanup

* apply changes to WGLMakie

* add test

* fix CairoMakie too

* update changelog

* fix missing start/end segment

* Update CHANGELOG.md

* update comments

---------

Co-authored-by: Simon <sdanisch@protonmail.com>
* add second `@recipe` method

* rename function

* implement recipe and keyword validation

* fix some bugs

* rename for clarity

* change Scatter recipe

* use argument symbols correctly

* change Lines

* change linesegments

* do a bunch of plots more

* fix text bugs

* splice in internal attribute after mesh creation

* add allowlist

* fix text

* add attribute deprecation mechanism

* add explanation

* remove `text` for GLMakie

* fix contour volume attributes

* add rasterize to allowlist

* add `enable_depth` to volume

* add `matcap` to mesh

* set `fxaa = false` on scatter, lines, linesegments, text

* move fxaa after mixins

* fix keywords

* remove markersize attribute

* remove invalid markersize

* add depthsorting to attributes for scatter

* fix forwarded attributes

* add absorption to volume

* don't use `used_attributes` for datashader canvas convert

* fix positional args

* image

* add better printing and rudimentary tests

* make allowlist a tuple for fewer allocations

* attribute names tuple instead of set

* make deprecations also a tuple

* avoid intermediate vector to speed up attribute creation

* apply same optimization to blocks

* implement mechanism to augment user docstring

* delete manually written out attribute docs

* use funcsym in docstring

* incorporate attribute docs into docstring again

* use equal

* add changelog

* introduce `DocumentedAttributes` with macro

* add function to access `DocumentedAttributes` of a recipe

* fix module mutation problem for now

* move definition behind PlotType

* return Attributes instead of Dict

* convert wireframe recipe

* close over reference

* don't include prereleases in benchmark, failed with 1.11 alpha

* stringify default expr

* convert arrows

* add docs

* directly convert default exprs to strings

* convert ablines

* print wrong attribute in red

* convert annotations

* convert arc

* remove attributes in arrows docstring

* convert band

* fix escaping issue

* convert barplot

* convert bracket

* fix barplot usage in hist

* convert contourf

* typo

* convert contour

* fix waterfall

* comment out arrows test

* convert datashader

* convert errorbar and rangebar

* convert hvlines

* convert hvspan

* convert pie

* convert rainclouds

* convert scatterlines

* convert series

* convert spy

* convert stairs

* convert stem

* convert streamplot

* convert timeseries

* convert tooltip

* convert tricontourf

* add transformation to allowlist

* convert triplot

* convert volumeslices

* convert voronoiplot

* convert waterfall

* convert boxplot

* convert crossbar

* convert density

* convert qqplot and qqnorm

* convert ecdfplot

* convert hexbin

* convert hist

* convert violin

* improve error message printing

* add docstrings for mutating functions and plot types

* add missing docstrings

* reconnect lines docstring

* move functions out of macro

* fix arrows example and adjust docs

* put necessary functions back into macro

---------

Co-authored-by: Simon <sdanisch@protonmail.com>
* create voxel rendering prototype

* enable lighting

* prototype voxel id generation & color handling

* add is_air attribute

* prototype texture mapping

* fix shader reloading

* fix texture mapping

* implement local updates

* optimize render order (depthsorting = false)

* add depthsorting = true

* render z planes first

* add lowclip and highclip

* add refimg tests + some fixes

* fix colorrange

* fix local chunk update

* handle colorrange more efficiently

* handle voxel id data more efficiently

* docstring & formatting

* switch back to lrbt order for uvmap

* add docs

* try fix tests

* fix show

* fix test?

* add missing dimensions

* add arguments for placement and scale

* allow Colon

* add Colon() to local_update

* minor cleanup

* prototype WGLMakie version

* add fallback in CairoMakie

* add RPRMakie fallback

* skip invisible voxels

* fix typo

* rename voxel -> voxels

* update docs, fix placement

* update news

* fix Colorbar for voxels

* enable tests

* fix texture rotation

* cleanup print

* cleanup comment

* generalize array access

* debug WGLMakie

* get voxels rendering in WGLMakie

* fix texture mapping

* activate tests

* fix moving planes, cleanup prints

* add unit tests

* add gap attribute

* tests & docs

* mention potential issues with picking

* fix WGLMakie picking

* fix depthsorting/gap handling

* switch to integer mod

* fix render order

* use RNG

* fix 1.6 3d array syntax

* fix refimage

* Update CHANGELOG.md

* fix julia 1.6

---------

Co-authored-by: Simon <sdanisch@protonmail.com>
Comment on lines +43 to +45
function mixin_generic_plot_attributes()
@DocumentedAttributes begin
transformation = automatic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add material here for RPRMakie? It's not quite a shading attribute, IMO, since it can also control the height through subsurface materials.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember Simon saying he was not super happy with material because it has no way of being supported in the other backends. But I'm not sure if it can all be handled by color plus types?

Copy link
Member

@asinghvi17 asinghvi17 Mar 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be a bit difficult to handle colors passed as values while also setting the material properties (metallicity, etc) if they were all packed in color to my understanding, so it's probably better to provide it IMO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could always be color = RPRColor(values, metallicity = ...) or whatever. @SimonDanisch what's your opinion?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the best would be to move the attributes from shading_attributes to something like material, and then have it support different types... But that would be a bigger refactor, so to allow RPRMakie to work with minimal effort, I'd just whitelist the material attribute for now

@asinghvi17
Copy link
Member

This causes GeoMakie to fail when assigning source and dest projections to separate plots, since it's not in the allow list. I added it for now but we probably should have an interface for any Makie package to add an extension to the allowlist for this reason.

For example, a ternary plot package may want to assign a plane, or some other nonlinear package may have some other global argument which they want to allow at the top level.

My current thoughts are that we could have global list of allowed attributes (we can route consumers through a registration function and don't have to allow them to access this directly) which is picked up by the function attribute_name_allowlist.

SimonDanisch and others added 2 commits March 18, 2024 11:32
* implement float64 precision

* remove merge conflicts

* update CairoMakie

* move some code around

* simplify syntax

* add non-Observable apply_transform_and_f32_conversion

* clean up convert_arguments

* update WGLMakie

* fix tests

* patch model to act after f32convert

* fix function name

* update project and plot_to_screen

* fix patch_model

* add first test

* rename file

* fix type after patch_model

* fix ticks beyond limits

* fix image and heatmap conversion

* convert to float types

* update test

* Fix CairoMakie not precompiling

missed a `Makie.` in the image recipe

* update mesh converts

* fix poly convert type

* fix dict access

* fix first test in WGLMakie

* don't use triangle_mesh

* fix incorrect convert_arguments

* fix test

* fix DataInspector error

* add meshscatter + surface to tests, fix convert_arguments for poly

* Fix docs + Makie tests

* Fix 1.6!?

* fix picking test

* update DataInspector

* fix indicator rotation for meshscatter in DataInspector

* fix Vector{<: Integer}, Vector{Float32} -> Vector{Float32} [skip ci]

* fix stackoverflow on mixed tuple types [skip ci]

* fix tuples correctly [skip ci]

* fix geom -> points eltype [skip ci]

* fix geom -> PointBased output type

* fix Rect2 -> points output type [skip ci]

* fix PointBased mesh conversion

* fix PointBased multi-linestrings type [skip ci]

* update bezierpath

* split up conversions

* fix missing Points convert [skip ci]

* fix CellGridBased types [skip ci]

* always convert volumes to float32

* start adding type tests for every convert_arguments

* fix tests

* undo splitting conversions.jl

* test and cleanup mesh conversions

* fix docs

* update & test Annotations

* update & test arrows

* update and test band and bracket

* update & test errorbars and rangebars conversions

* test & update series converts

* make type stable on 1.6

* update & test remaining convert_arguments

* fix & test model application

* add tests for float32convert

* at floatmin/max refimg test

* fix typo

* update text bbox test & add backtrace

* avoid Float32 in data_limits

* fix Polygon -> Bezierpath conversion

* fix WGLMakie meshscatter model patching

* fix Float64 normals

* fix Float64 matrices in Voxels

* test normal and uv types

* fix for 1.6

* make new project method more complete

* clean up some TODOs

* restore text tests

* restore Polar transform tests

* avoid BBox for Axis limits

* fix rectangle zoom, deprecate to_world with matrix inputs

* fix 1.6

* fix Float64 latexstrings

* fix hvlines, hvspan, errorbars, rangebars

* fix & test ablines

* fix other usage of projview_to_2d_limits

* remove dublicated line

* update changelog + cleanup [skip ci]

* fix missing to_value

* consider markersize and offset for scatter data_limits

* fix tests

* revert BBox -> Rect2d changes

* get hist and barplot working

---------

Co-authored-by: ffreyer <frederic481994@hotmail.de>
Co-authored-by: Anshul Singhvi <anshulsinghvi@gmail.com>
* document conversion pipeline

* fix doc build

* fix block

---------

Co-authored-by: SimonDanisch <sdanisch@protonmail.com>
@asinghvi17
Copy link
Member

asinghvi17 commented Mar 23, 2024

https://github.com/JuliaLang/Pkg.jl/blob/9c6356fa9d838388cc308cfedaec6cd38fdf00ba/src/Types.jl#L1013C20-L1013C85

We could integrate a fuzzy suggestor into Makie for misspelled attributes, since we know all available attribute names (but this requires a dependency on Pkg). It could also be implemented as an extension, or we could directly copy the source.

jkrumbiegel and others added 3 commits March 25, 2024 14:04
* deprecate `rotations` for `rotation` for Scatter and MeshScatter

* one more rename

* remove double conversion

* fix cairomakie meshscatter

* fix usage in docs

* remove mat4 code again

* fix CairoMakie rror

* change one more rotations

* fix WGLMakie

* add changelog entry

---------

Co-authored-by: ffreyer <frederic481994@hotmail.de>
@ffreyer ffreyer removed the skip-changelog Skips changelog enforcer label Mar 25, 2024
ffreyer and others added 2 commits March 26, 2024 14:46
* implement scatter boundingbox with marker metrics

* add tests

* always consider marker rotations

* update changelog [skip ci]

* fix data_limits

* fix some more issues

* rotations -> rotation

* Update boundingboxes.jl

---------

Co-authored-by: Simon <sdanisch@protonmail.com>
* change how boundingbox is deprecated

* update changelog

* fix test errors

* fix docs errors and cleanup error message

* fix type instability

* fix function name

* fix function name

* fix boundingbox overwrites

* update changelog

* add types to function args

maybe this allows boundingbox(::MyPlot, space) to work without ::Symbol...

* fix typing

* fully remove :world space

* fix test
ffreyer and others added 7 commits April 30, 2024 15:37
* cleanup temp defitions

* finalize data_limits cleanup

* GeometryBasics should probably be breaking

* adjust other versions

* use newly tagged GeometryBasics

* use breaking gridlayoutbase

* try updating registry

* somehow this is needed?!

* add to missing CI

---------

Co-authored-by: SimonDanisch <sdanisch@protonmail.com>
* take over most of the work from #1347

* add typed argument conversion (#3565)

* add typed argument conversion

* fix volume

* add function to get available conversions

* make conversion apply more narrowly

* more cleanly separate recursion in convert_arguments

* clean up

* allow to get axis before creating a plot

* clean up

* fix tests

* bring back dim converts (axis_convert)

* update tests

* fix tests and work around conversion problems

* fix WGLMakie

* fix errors

* clean up conversion pipeline

* fix tests

* add changelog entry

* disable project run

* improve performance slightly

* might as well use array

* tmp

* wip

* implement axis convert recursion

* fix tests

* fix datashader

* fix datashader

* move unitful integration

* fix performance regression!?

* fix merge & new date time improvements

* fix scaling test

* remove test false

* clean up

* converts shouldnt be here

* move axis converts to scene

* further refactor [skip ci]

* finish refactor for AxisConversion type

* allow limit setting and ticks

* make tests less noisy

* cleanup

* clean up and fix unitful/date conversion

* make sure all tests work correctly

* remove rand

* rename, clean up and make axis spec work

* clean up and test new conversion pipeline

* undo feature deletion, don't reintroduce Rect2f

* be explicit about Volume Interval types

* minor docstring cleanup

* try to clarify new conversion docstrings

* remove convert_arguments_typed in favor of types_for_plot_arguments

* fix remaining bugs for conversion simplification

* fix ticks not updating

* fix specapi

* fix qqnorm

* clean up types_for_plot_arguments

* fix tuple conversion

* try to fix compile time regression

* try to fix compile time regression

* clean up and introduce expand_dimensions

* fix #3655 and clean up convert_arguments + add tests

* fix #3509 and add tests for

* clean up observables and more docs

* final rename

* fix docs

* cleanup

* small clean up

* small doc improvements

* improve docs

* fix docs

* try relative link

* try without .md

* take out link

* try fix

---------

Co-authored-by: ffreyer <frederic481994@hotmail.de>
* [Docs] add wrap/redirect recipe tutorial

* fix sentence

* Update docs/tutorials/wrap-existing-recipe.md

Co-authored-by: Anshul Singhvi <anshulsinghvi@gmail.com>

* update

* fix doc build

* add changelog

* fix preview

---------

Co-authored-by: Moelf <proton@jling.dev>
Co-authored-by: Anshul Singhvi <anshulsinghvi@gmail.com>
# Only forward attributes that are usable by Plot{Func}
# TODO, there are two problems with this:
# 1) It's a bit magical which attributes are forwarded, when writing a recipe
# 2) We can't warn for wrong attributes, since if we do `plot(p, some_other_plot.attributes, ...)`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wasn't the warning for wrong attributes the reason why we did this in the first place? I wouldn't like reintroducing that behavior again in a different place..

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, but this is only for forwarding all recipe attributes, which seems like a fair use to only forward valid attributes?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what's the scenario where someone does that? Is it likely that there are many other invalid attributes in the collection that are annoying to take out manually? One could also filter them by the keys of the recipe to forward to?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One could also filter them by the keys of the recipe to forward to?

That's what we do here, or do you mean something else?

@SimonDanisch SimonDanisch reopened this May 7, 2024
@SimonDanisch
Copy link
Member Author

Seems like we have some regressions in performance, some are expected (e.g. using time), while others are not (since this came with changes that should not have changed performance).
I dont have time to debug this, so I guess we need to merge & tag this with those regressions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants