Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

better performance for Menus and fix clicks on items #2299

Merged
merged 8 commits into from
Sep 27, 2022
Merged

Conversation

SimonDanisch
Copy link
Member

addmouseevents! seems to ignore clicks and I couldn't really get to the bottom of the problem :(
I started using scene.events directly to debug things, and since it fixed the issue, should be faster, uses less listeners and does all the work in one function, i sticked with that solution ;)

I'm not 100% sure though, if it's correct in regards to Consume behaviour, but it seems to work pretty well.
We should also think about adding a z-index value, since in some cases the z-value seems too low.
Maybe, we should also think about having overdraw=true for scenes, so that we can better create pop-up scenes without worrying about any z-index.

This also defaults to the first item in the option list instead of nothing, which at least to me seems more practical. Willing to remove that change, though ;)

@MakieBot
Copy link
Collaborator

MakieBot commented Sep 24, 2022

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(display(fig))
using create display create display
GLMakie 17.14s (17.03, 17.29) 0.09+- 18.44s (18.32, 18.81) 0.17+- 17.48s (17.31, 17.73) 0.13+- 16.73ms (16.38, 17.05) 0.22+- 99.94ms (99.06, 100.89) 0.62+-
master 16.91s (16.74, 17.27) 0.19+- 18.44s (18.14, 18.77) 0.20+- 17.42s (17.21, 17.67) 0.16+- 16.31ms (16.03, 16.54) 0.20+- 99.20ms (98.51, 99.91) 0.50+-
evaluation +1.31%, 0.22s slower X (1.51d, 0.02p, 0.14std) +0.00%, 0.0s invariant (0.00d, 1.00p, 0.19std) +0.36%, 0.06s invariant (0.43d, 0.43p, 0.15std) +2.46%, 0.41ms slower X (1.96d, 0.00p, 0.21std) +0.74%, 0.74ms slower X (1.32d, 0.03p, 0.56std)
CairoMakie 16.50s (16.26, 16.83) 0.22+- 23.41s (22.63, 24.06) 0.56+- 3.72s (3.57, 3.87) 0.10+- 23.90ms (23.58, 24.40) 0.27+- 29.69ms (29.04, 30.20) 0.40+-
master 16.62s (16.21, 16.88) 0.24+- 23.06s (22.54, 23.66) 0.38+- 3.78s (3.68, 3.89) 0.08+- 23.98ms (23.57, 24.53) 0.36+- 29.57ms (28.90, 30.54) 0.53+-
evaluation -0.68%, -0.11s invariant (-0.50d, 0.37p, 0.23std) +1.52%, 0.36s invariant (0.74d, 0.19p, 0.47std) -1.61%, -0.06s invariant (-0.68d, 0.23p, 0.09std) -0.36%, -0.09ms invariant (-0.27d, 0.63p, 0.32std) +0.43%, 0.13ms invariant (0.27d, 0.62p, 0.47std)
WGLMakie 22.34s (21.86, 22.93) 0.37+- 31.12s (30.19, 32.43) 0.74+- 52.70s (50.84, 55.89) 1.87+- 32.33ms (31.64, 33.11) 0.51+- 1.91s (1.80, 1.99) 0.06+-
master 22.75s (22.08, 24.18) 0.77+- 31.78s (31.13, 33.21) 0.86+- 51.92s (50.60, 54.09) 1.37+- 32.68ms (31.64, 33.34) 0.64+- 1.89s (1.84, 1.95) 0.04+-
evaluation -1.86%, -0.42s invariant (-0.69d, 0.23p, 0.57std) -2.12%, -0.66s invariant (-0.82d, 0.15p, 0.80std) +1.48%, 0.78s invariant (0.48d, 0.39p, 1.62std) -1.09%, -0.35ms invariant (-0.61d, 0.28p, 0.58std) +1.09%, 0.02s invariant (0.39d, 0.48p, 0.05std)

@ffreyer
Copy link
Collaborator

ffreyer commented Sep 24, 2022

Some more interaction issues/differences:

  • When moving outside the expanded selection menu, the last hovered element remains highlighted.
  • After selecting a new element and opening the menu again, the selected element is highlighted as hovered and the previous element is highlighted as selected. This updates when the mouse hovers one of the options
  • Clicking outside the expanded selection menu does not close the menu anymore.

@SimonDanisch
Copy link
Member Author

SimonDanisch commented Sep 26, 2022

Thanks for the thorough review :) I should have fixed all problems now - and found a new appreciation for mousestatemachine :D
But, maybe we could refactor it to be without observables and just use a stateful struct? Then we can do everything in one function as well:

mousestate1 = mousestatemachine(scene, plots...)
mousestate2 = mousestatemachine(scene, other_plots...)
on(events.mouse) do mouse 
   if ishover(mousestate1, mouse) 
       .... 
    end
    if ishover(mousestate2, mouse)  
        ....
    end
end

Well, that's for another PR and I also still need to take a closer look at mousestatemachine. Maybe it's not as bad to have one observable and multiple functions registered...But then one needs to fight for the event via consume, which seems easier if the whole state of something complex as a menu is handled in one function 🤷

@jkrumbiegel
Copy link
Member

If mouseevents ignores clicks it's probably because they are registered on multiple objects or scenes at once I think. Unless the mouse down and mouse up events are already not received. But if they are, then some other condition must not be met, which could be a bug in the implementation but also in our event system.

@github-actions
Copy link
Contributor

Missing reference images

Found 1 new images without existing references.
Upload new reference images before merging this PR.

@ffreyer
Copy link
Collaborator

ffreyer commented Sep 26, 2022

Back when I reworked the event system (i.e. introduced priority) I also noticed some differences between how mouse state machine works and how my OS reacts. I reworked it back then but didn't push it because I didn't want to add more complexity to the pr. I noted some of the oddities in the tests but I never got back to making a pr for it...

Makie.jl/test/events.jl

Lines 361 to 435 in 087dfbd

# drag
e.mousebutton[] = MouseButtonEvent(getfield(Mouse, button), Mouse.press)
e.mouseposition[] = (500, 300)
e.mouseposition[] = (700, 200)
e.mousebutton[] = MouseButtonEvent(getfield(Mouse, button), Mouse.release)
@test length(eventlog) == 6
prev_px = Point2f[(300, 300), (300, 300), (300, 300), (500, 300), (700, 200), (700, 200)]
px = Point2f[(300, 300), (500, 300), (500, 300), (700, 200), (700, 200), (700, 200)]
for (i, t) in enumerate((
getfield(MouseEventTypes, Symbol(button, :down)),
getfield(MouseEventTypes, Symbol(button, :dragstart)),
getfield(MouseEventTypes, Symbol(button, :drag)),
getfield(MouseEventTypes, Symbol(button, :drag)),
getfield(MouseEventTypes, Symbol(button, :dragstop)),
getfield(MouseEventTypes, :out),
# TODO this is kinda missing an "up outside"
))
@test eventlog[i].type == t
@test eventlog[i].px == px[i]
@test eventlog[i].prev_px == prev_px[i]
end
e.mouseposition[] = (300, 300)
empty!(eventlog)
end
# TODO: This should probably produce:
# left down > right down > right click > right up > left up
e.mousebutton[] = MouseButtonEvent(Mouse.left, Mouse.press)
e.mousebutton[] = MouseButtonEvent(Mouse.right, Mouse.press)
e.mousebutton[] = MouseButtonEvent(Mouse.right, Mouse.release)
e.mousebutton[] = MouseButtonEvent(Mouse.left, Mouse.release)
@test length(eventlog) == 3
@test eventlog[1].type == MouseEventTypes.leftdown
@test eventlog[2].type == MouseEventTypes.leftclick
@test eventlog[3].type == MouseEventTypes.leftup
empty!(eventlog)
# double left up? :(
e.mousebutton[] = MouseButtonEvent(Mouse.left, Mouse.press)
e.mousebutton[] = MouseButtonEvent(Mouse.right, Mouse.press)
e.mousebutton[] = MouseButtonEvent(Mouse.left, Mouse.release)
e.mousebutton[] = MouseButtonEvent(Mouse.right, Mouse.release)
@test length(eventlog) == 4
@test eventlog[1].type == MouseEventTypes.leftdown
@test eventlog[2].type == MouseEventTypes.leftdoubleclick
@test eventlog[3].type == MouseEventTypes.leftup
@test eventlog[4].type == MouseEventTypes.leftup
empty!(eventlog)
# This should probably produce a leftdragstop on right down instead of left up
e.mouseposition[] = (300, 300)
empty!(eventlog)
e.mousebutton[] = MouseButtonEvent(Mouse.left, Mouse.press)
e.mouseposition[] = (350, 350)
e.mousebutton[] = MouseButtonEvent(Mouse.right, Mouse.press)
e.mouseposition[] = (350, 400)
e.mousebutton[] = MouseButtonEvent(Mouse.right, Mouse.release)
e.mouseposition[] = (400, 400)
e.mousebutton[] = MouseButtonEvent(Mouse.left, Mouse.release)
@test length(eventlog) == 7
@test eventlog[1].type == MouseEventTypes.leftdown
@test eventlog[2].type == MouseEventTypes.leftdragstart
@test eventlog[3].type == MouseEventTypes.leftdrag
@test eventlog[4].type == MouseEventTypes.leftdrag
@test eventlog[5].type == MouseEventTypes.over
@test eventlog[6].type == MouseEventTypes.leftdragstop
@test eventlog[7].type == MouseEventTypes.leftup
@test eventlog[1].px == Point2f(300, 300)
@test eventlog[2].px == Point2f(350, 350)
@test eventlog[3].px == Point2f(350, 350)
@test eventlog[4].px == Point2f(350, 400)
@test eventlog[5].px == Point2f(400, 400)
@test eventlog[6].px == Point2f(400, 400)
@test eventlog[7].px == Point2f(400, 400)
empty!(eventlog)

@jkrumbiegel
Copy link
Member

ah good that you collected some of those, it's much better to test them in this way, although I only found some of the edge cases by trying to break the code and acting weird with the gui. Some of these cases are not well defined but I think if the big OSes differ in behavior we should use that as reference. Like with the mentioned case where left and right click happen in a nested fashion

@SimonDanisch
Copy link
Member Author

I say we merge this and leave further refactors & tests of the event system / mousestate machine for another day?

@ffreyer
Copy link
Collaborator

ffreyer commented Sep 26, 2022

There are still some issues:

  • left click outside opened menu doesn't close it
  • when opening a menu the button, the last hovered element and the selected element are highlighted. Should only be the selected and hovered element/button.
  • when moving the mouse outside the menu area the button hover color clears, but the element hover color doesn't.
  • After selecting a new element, closing and reopening the menu the selected entry is still highlighted as hovered. That updates to selected once the mouse hovers any element in the dropdown menu

@SimonDanisch
Copy link
Member Author

Are you sure you have all the new changes? That sounds exactly like all the issues I fixed...

@ffreyer
Copy link
Collaborator

ffreyer commented Sep 26, 2022

Oh nvm, I didn't pull correctly. Looks good to me 😆

@SimonDanisch SimonDanisch merged commit 38296f5 into master Sep 27, 2022
@SimonDanisch SimonDanisch deleted the sd/fix-menu branch September 27, 2022 12:46
@jkrumbiegel jkrumbiegel mentioned this pull request Oct 8, 2022
5 tasks
t-bltg pushed a commit to t-bltg/Makie.jl that referenced this pull request Dec 31, 2022
* better perf + fix clicks on items

* fix menu behaviour

* clean up and comment

* clean up and reference test

* fix tests

* don't make background black
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants