Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

early attribute conversion + no proxy scene anymore -> much improved compile times #3082

Closed
wants to merge 26 commits into from

Conversation

SimonDanisch
Copy link
Member

@SimonDanisch SimonDanisch commented Jul 22, 2023

This extracts useful bits from #2220, which aren't as breaking as #2220:

  • convert plot attributes as early as possible, to not pass namedtuples through lots of functions, therefore reducing compilation costs
  • uses plot_preferred_axis to figure out the default axis, instead of using boundingbox(plot!(Scene(), args...)), which is expensive and kind of hacky

closes #2820

src/basic_recipes/axis.jl Outdated Show resolved Hide resolved
@MakieBot
Copy link
Collaborator

MakieBot commented Jul 22, 2023

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(display(fig))
using create display create display
GLMakie 11.66s (11.47, 11.84) 0.12+- 861.41ms (823.92, 892.75) 23.25+- 731.21ms (703.99, 798.63) 31.06+- 9.99ms (9.85, 10.07) 0.08+- 142.78ms (140.50, 144.96) 1.57+-
master 12.56s (12.37, 12.72) 0.14+- 1.10s (1.08, 1.12) 0.02+- 766.09ms (753.94, 794.09) 14.42+- 11.16ms (11.08, 11.49) 0.15+- 143.96ms (141.74, 146.38) 1.83+-
evaluation -7.70%, -0.9s faster✅ (-7.15d, 0.00p, 0.13std) -27.31%, -235.27ms faster✅ (-11.76d, 0.00p, 19.68std) -4.77%, -34.88ms faster ✓ (-1.44d, 0.03p, 22.74std) -11.74%, -1.17ms faster✅ (-9.89d, 0.00p, 0.11std) -0.83%, -1.18ms invariant (-0.69d, 0.22p, 1.70std)
CairoMakie 9.54s (9.46, 9.65) 0.07+- 797.21ms (755.70, 818.91) 19.91+- 215.46ms (212.43, 217.14) 1.75+- 9.19ms (9.09, 9.24) 0.06+- 5.75ms (5.71, 5.88) 0.06+-
master 10.42s (10.35, 10.51) 0.05+- 1.03s (1.02, 1.05) 0.01+- 214.98ms (211.98, 218.53) 2.34+- 10.37ms (10.25, 10.67) 0.14+- 6.13ms (6.06, 6.23) 0.06+-
evaluation -9.20%, -0.88s faster✅ (-15.38d, 0.00p, 0.06std) -29.27%, -233.38ms faster✅ (-14.53d, 0.00p, 15.43std) +0.22%, 0.48ms invariant (0.23d, 0.67p, 2.04std) -12.90%, -1.19ms faster✅ (-10.66d, 0.00p, 0.10std) -6.58%, -0.38ms faster✅ (-6.70d, 0.00p, 0.06std)
WGLMakie 15.28s (14.93, 15.47) 0.19+- 1.37s (1.27, 1.47) 0.07+- 16.29s (16.06, 16.69) 0.26+- 19.83ms (19.07, 21.02) 0.73+- 1.54s (1.48, 1.67) 0.07+-
master 16.71s (15.95, 17.97) 0.71+- 1.63s (1.47, 1.74) 0.10+- 16.23s (15.64, 16.78) 0.40+- 21.78ms (20.51, 23.50) 0.98+- 1.49s (1.42, 1.57) 0.06+-
evaluation -9.39%, -1.44s faster✅ (-2.76d, 0.00p, 0.45std) -18.61%, -0.26s faster✅ (-2.98d, 0.00p, 0.08std) +0.37%, 0.06s invariant (0.18d, 0.74p, 0.33std) -9.80%, -1.94ms faster✅ (-2.26d, 0.00p, 0.85std) +3.17%, 0.05s invariant (0.76d, 0.18p, 0.06std)

@github-actions
Copy link
Contributor

Missing reference images

Found 1 new images without existing references.
Upload new reference images before merging this PR.

SimonDanisch added a commit that referenced this pull request Aug 1, 2023
commit 67e436d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 31 18:00:55 2023 +0200

    small cleanups

commit a38091f
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 31 15:52:39 2023 +0200

    small clean ups

commit 97cfc01
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 16:27:04 2023 +0200

    fix theming

commit f31b4ab
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 14:14:59 2023 +0200

    fix WGLMakie

commit 2067612
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 13:16:45 2023 +0200

    fix WGLMakie

commit 3172c6a
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 12:48:12 2023 +0200

    fix last tests

commit 786c55c
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Wed Jul 26 17:49:39 2023 +0200

    try to fix cycling

commit 3f2aee8
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Wed Jul 26 15:28:06 2023 +0200

    bring back error messages

commit d352d9f
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Wed Jul 26 11:57:36 2023 +0200

    fix Makie tests

commit b95b72b
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Tue Jul 25 15:40:36 2023 +0200

    address frederics code review

commit ee1728d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Tue Jul 25 15:38:51 2023 +0200

    fix performance?!?!

commit 22488bf
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 18:15:39 2023 +0200

    add back precompiles

commit ee5e570
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 18:14:55 2023 +0200

    fix PlotSpec

commit 64576ce
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 17:24:07 2023 +0200

    fix GLMakie tests

commit 0196af5
Merge: bb2f665 04f6704
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 11:51:25 2023 +0200

    Merge branch 'master' into sd/early-attribute-conversion

commit bb2f665
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 11:51:10 2023 +0200

    dont use plotspec

commit dc205a1
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 14:48:19 2023 +0200

    fix precompilation

commit 017372f
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 13:35:21 2023 +0200

    remove erroring precompiles

commit 79c5294
Merge: cfeb4e6 2bdd105
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 12:06:49 2023 +0200

    Merge branch 'sd/early-attribute-conversion' of https://github.com/MakieOrg/Makie.jl into sd/early-attribute-conversion

commit cfeb4e6
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 12:04:47 2023 +0200

    fix CairoMakie

commit 2bdd105
Merge: 8d87ca7 aab593f
Author: Simon <sdanisch@protonmail.com>
Date:   Sat Jul 22 11:38:56 2023 +0200

    Merge branch 'master' into sd/early-attribute-conversion

commit 8d87ca7
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 11:30:09 2023 +0200

    fix most issues

commit eb4a42d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 10:12:50 2023 +0200

    first plot showing

commit 76a3f3d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 09:26:14 2023 +0200

    try to get things running

commit d5d457c
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 20 15:42:26 2023 +0200

    try to get things working

commit 8674156
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 20 14:03:41 2023 +0200

    get code from breaking branch
SimonDanisch added a commit that referenced this pull request Aug 8, 2023
commit 67e436d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 31 18:00:55 2023 +0200

    small cleanups

commit a38091f
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 31 15:52:39 2023 +0200

    small clean ups

commit 97cfc01
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 16:27:04 2023 +0200

    fix theming

commit f31b4ab
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 14:14:59 2023 +0200

    fix WGLMakie

commit 2067612
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 13:16:45 2023 +0200

    fix WGLMakie

commit 3172c6a
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 12:48:12 2023 +0200

    fix last tests

commit 786c55c
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Wed Jul 26 17:49:39 2023 +0200

    try to fix cycling

commit 3f2aee8
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Wed Jul 26 15:28:06 2023 +0200

    bring back error messages

commit d352d9f
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Wed Jul 26 11:57:36 2023 +0200

    fix Makie tests

commit b95b72b
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Tue Jul 25 15:40:36 2023 +0200

    address frederics code review

commit ee1728d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Tue Jul 25 15:38:51 2023 +0200

    fix performance?!?!

commit 22488bf
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 18:15:39 2023 +0200

    add back precompiles

commit ee5e570
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 18:14:55 2023 +0200

    fix PlotSpec

commit 64576ce
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 17:24:07 2023 +0200

    fix GLMakie tests

commit 0196af5
Merge: bb2f665 04f6704
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 11:51:25 2023 +0200

    Merge branch 'master' into sd/early-attribute-conversion

commit bb2f665
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 11:51:10 2023 +0200

    dont use plotspec

commit dc205a1
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 14:48:19 2023 +0200

    fix precompilation

commit 017372f
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 13:35:21 2023 +0200

    remove erroring precompiles

commit 79c5294
Merge: cfeb4e6 2bdd105
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 12:06:49 2023 +0200

    Merge branch 'sd/early-attribute-conversion' of https://github.com/MakieOrg/Makie.jl into sd/early-attribute-conversion

commit cfeb4e6
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 12:04:47 2023 +0200

    fix CairoMakie

commit 2bdd105
Merge: 8d87ca7 aab593f
Author: Simon <sdanisch@protonmail.com>
Date:   Sat Jul 22 11:38:56 2023 +0200

    Merge branch 'master' into sd/early-attribute-conversion

commit 8d87ca7
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 11:30:09 2023 +0200

    fix most issues

commit eb4a42d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 10:12:50 2023 +0200

    first plot showing

commit 76a3f3d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 09:26:14 2023 +0200

    try to get things running

commit d5d457c
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 20 15:42:26 2023 +0200

    try to get things working

commit 8674156
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 20 14:03:41 2023 +0200

    get code from breaking branch
SimonDanisch added a commit that referenced this pull request Aug 16, 2023
commit 67e436d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 31 18:00:55 2023 +0200

    small cleanups

commit a38091f
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 31 15:52:39 2023 +0200

    small clean ups

commit 97cfc01
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 16:27:04 2023 +0200

    fix theming

commit f31b4ab
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 14:14:59 2023 +0200

    fix WGLMakie

commit 2067612
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 13:16:45 2023 +0200

    fix WGLMakie

commit 3172c6a
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 27 12:48:12 2023 +0200

    fix last tests

commit 786c55c
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Wed Jul 26 17:49:39 2023 +0200

    try to fix cycling

commit 3f2aee8
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Wed Jul 26 15:28:06 2023 +0200

    bring back error messages

commit d352d9f
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Wed Jul 26 11:57:36 2023 +0200

    fix Makie tests

commit b95b72b
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Tue Jul 25 15:40:36 2023 +0200

    address frederics code review

commit ee1728d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Tue Jul 25 15:38:51 2023 +0200

    fix performance?!?!

commit 22488bf
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 18:15:39 2023 +0200

    add back precompiles

commit ee5e570
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 18:14:55 2023 +0200

    fix PlotSpec

commit 64576ce
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 17:24:07 2023 +0200

    fix GLMakie tests

commit 0196af5
Merge: bb2f665 04f6704
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 11:51:25 2023 +0200

    Merge branch 'master' into sd/early-attribute-conversion

commit bb2f665
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Mon Jul 24 11:51:10 2023 +0200

    dont use plotspec

commit dc205a1
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 14:48:19 2023 +0200

    fix precompilation

commit 017372f
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 13:35:21 2023 +0200

    remove erroring precompiles

commit 79c5294
Merge: cfeb4e6 2bdd105
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 12:06:49 2023 +0200

    Merge branch 'sd/early-attribute-conversion' of https://github.com/MakieOrg/Makie.jl into sd/early-attribute-conversion

commit cfeb4e6
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 12:04:47 2023 +0200

    fix CairoMakie

commit 2bdd105
Merge: 8d87ca7 aab593f
Author: Simon <sdanisch@protonmail.com>
Date:   Sat Jul 22 11:38:56 2023 +0200

    Merge branch 'master' into sd/early-attribute-conversion

commit 8d87ca7
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 11:30:09 2023 +0200

    fix most issues

commit eb4a42d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 10:12:50 2023 +0200

    first plot showing

commit 76a3f3d
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Sat Jul 22 09:26:14 2023 +0200

    try to get things running

commit d5d457c
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 20 15:42:26 2023 +0200

    try to get things working

commit 8674156
Author: SimonDanisch <sdanisch@protonmail.com>
Date:   Thu Jul 20 14:03:41 2023 +0200

    get code from breaking branch
@SimonDanisch
Copy link
Member Author

Merged in #3113

@SimonDanisch SimonDanisch mentioned this pull request Sep 11, 2023
16 tasks
@SimonDanisch SimonDanisch changed the title early attribute conversion + no proxy scene anymore early attribute conversion + no proxy scene anymore -> much improved compile times Sep 11, 2023
SimonDanisch added a commit that referenced this pull request Nov 17, 2023
Continues #2831 !
Still needs to check, if I rebased correctly and didn't incorrectly
apply some of the changes!

## Merged PRs
- #2598
- #2746
- #2346
- #2544
- #3082
- #2868
- #3062
- #3106
- #3281
- #3246

## TODOS

- [x] fix flaky test `@test GLMakie.window_size(screen.glscreen) ==
scaled(screen, (W, H))`
- [x] Merge axis type inferences from #2220 
- [x] Test on different resolution screens, IJulia, Pluto, VSCode,
Windowed
- [x] rebase to only have merge commits from the PRs 
- [x] investigate unexpected speed ups
- [x] reset camera settings from tests
- [ ] check doc image generation
- [x] rethink default near/far in Camera3D (compatability with OIT)
- [x] merge #3246
- [x] fix WGLMakie issues/tests:
- [x] fix line depth issues (see tests: ~~hexbin colorrange~~ (not new),
LaTeXStrings in Axis3, Axis3 axis reversal)
  - [x] fix lighting of surface with nan points (fixed in #3246)
- ~~volume/3D contour artifacts (see 3D Contour with 2D contour
slices)~~ not new
  - ~~artifacting in "colorscale (lines)"~~ not new
- [x] GLMakie:
  - [x] slight outline in "scatter image markers" test
  - ~~clipping/z-fighting in "volume translated"~~ not new
- [x] CairoMakie:
  -  ~~Artfiacting in `colorscale (lines)"~~ not new
  - ~~markersize in "scatter rotations" changed?~~ not new
  - ~~color change in "colorscale (poly)"~~ not new
  - ~~transparency/render order of "OldAxis + Surface"~~ not new
  - ~~render order in "Merged color mesh"~~ not new
  - ~~render order of "Surface + wireframe + contour"~~ not new
- [x] Check "SpecApi in convert_arguments" (colors swapped?)


## Fixes the following errors

- fixes #2721 via #2746
- fixes #1600 via #2746
- fixes #1236 via #2746
- fixes MakieOrg/GeoMakie.jl#133 via #2598
- closes #2522
- closes #3239 via #3246
- fixes #3238 via #3246
- fixes #2985 via #3246
- fixes #3307 via #3281
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Empty vectors of 2d Polygon plot 3d
3 participants