Improve skeleton proofreading and fix some shortcuts on MacOS #7678

philippotto · 2024-03-07T16:28:12Z

Enables the existing (legacy) shortcuts for deleting/creating edges for orthogonal and arbitrary mode. Now, they are not restricted to legacy mode anymore.
Improves the shortcut indicator in the status bar.
Allows creating trees and deleting nodes in arbitrary mode with the keyboard.
Treats the Cmd (meta) key in MacOS as a synonym to ctrl everywhere.
Shows the ⌥ symbol instead of "alt" and Cmd instead of Ctrl in the status bar and context menu on MacOS.

URL of deployed dev instance (used for testing):

https://improveskeletonproofreading.webknossos.xyz/

Steps to test:

ortho mode:
- create a tree with some nodes
- delete an edge by holding ctrl+shift while clicking on the neighbor of the active node (the statusbar should indicate that action while pressing the modifiers)
- merge the edge again by holding ctrl+alt (on some linux setups, shift also needs to be pressed to avoid that the default handler for moving a window is triggered)
- the above should also work in legacy mode
- test that node selection still works correctly (legacy mode should require shift for that)
ortho mode
- c should create a new tree
- delete should delete the active node
- clicking on a node should select it (sometimes the node is then not precisely visible, I don't really know how we could circumvent this)
- as above, try out creating and deleting edges
on mac:
- test that Zoom hotkey hint could be improved on macOS #7652 is fixed
- test that Ctrl+Click proofreading shortcut does not work on MacBook #7584 is fixed

Issues:

(Please delete unneeded items, merge only when none are left open)

Updated changelog
Updated migration guide if applicable
Updated documentation if applicable
Adapted wk-libs python client if relevant API parts change
Removed dev-only changes like prints and application.conf edits
Considered common edge cases
Needs datastore update after deployment

…; adapt status bar accordingly, too

…ary mode

…rove-skeleton-proofreading

… MacOS

daniel-wer

Nice improvements! Code looks great 👍

Notes from testing:

Ctrl + Leftclick to delete meshes doesn't work (I suggested the fix during my review, I just want to remember to retest this)
From the internal feature request that spawned this PR, number (1) refers to easy skeleton merging and splitting using keyboard shortcuts similar to proofreading. Would it be possible to reuse the shortcuts used in proofreading mode or would this be a breaking change?
The keyboard shortcut overview is no longer accurate. For example, the arbitrary mode section doesn't show that nodes can be deleted or edges can be removed/created.
In flight/oblique mode, the active node is not highlighted making it hard to know what's going to happen if delete is pressed or an edge is created.
Someone with a Mac should test the Mac-related fixes in this PR

frontend/javascripts/oxalis/controller/combinations/skeleton_handlers.ts

frontend/javascripts/oxalis/controller/td_controller.tsx

frontend/javascripts/oxalis/controller/viewmodes/arbitrary_controller.tsx

daniel-wer · 2024-03-12T15:59:33Z

frontend/javascripts/oxalis/view/rendering_utils.ts

+        // camera.far = state.userConfiguration.clippingDistanceArbitrary;
+        camera.near = ARBITRARY_CAM_DISTANCE - state.userConfiguration.clippingDistanceArbitrary;
+      } else {
+        camera.far = state.userConfiguration.clippingDistance;


Was this a bug before? If so, the comment can be removed

I dug into this for a bit, but I'm still not really sure what is going on. The far/near clipping still doesn't seem to work as expected when selecting nodes. This is also a problem in orthogonal mode (not new). It seemed like regardless of what I did, I could always select nodes even when they weren't visible and I did stuff like setting near=far=0.
I noticed that frustumCulled is set to false for the skeleton geometry (this is relatively new), which seemed like a likely culprit. However, even setting that to true, didn't make a difference.

Unless you have another idea, I'd ignore this for now :S

Thank you for your further investigations!

It seemed like regardless of what I did, I could always select nodes even when they weren't visible and I did stuff like setting near=far=0.

One more thing that just came to my mind is that I think we enlarge the nodes in the picking shader a bit to make it easier to click them - maybe this is done in z as well 🤔

However, we don't need to investigate this further as part of this PR. Did you find an existing issue or could you create one so we remember to have another look at this in the future.

One more thing that just came to my mind is that I think we enlarge the nodes in the picking shader a bit to make it easier to click them - maybe this is done in z as well 🤔

Interesting idea! However, here it is written that gl_PointSize "contains size of rasterized points, in pixels". So, I would assume that it only impacts the size in 2D as rasterization is explicitly mentioned.

Did you find an existing issue or could you create one so we remember to have another look at this in the future.

I just created #7725 :)

frontend/javascripts/oxalis/view/statusbar.tsx

Co-authored-by: Daniel <daniel.werner@scalableminds.com>

philippotto · 2024-03-19T13:03:06Z

From the internal feature request that spawned this PR, number (1) refers to easy skeleton merging and splitting using keyboard shortcuts similar to proofreading. Would it be possible to reuse the shortcuts used in proofreading mode or would this be a breaking change?

I think, this won't be possible without breaking changes. CTRL + Leftclick is used for moving nodes in classic mode and Shift + leftclick is used for selecting nodes. Regardless of classic mode, there is ctrl + leftclick to create a node without focusing it. in that case, one would click somewhere where no existing node is, but I think, it's still a collision with the split operation.
I'd defer this change to another point when we decide to improve the shortcuts. For now, this PR should empower users to do the skeleton editing without the context menu.

philippotto · 2024-03-19T14:20:56Z

@valentin-pinkau could you test the two mac-related items from the test plan? :)

on mac:
test that #7652 is fixed
test that #7584 is fixed

valentin-pinkau · 2024-03-21T10:02:47Z

@valentin-pinkau could you test the two mac-related items from the test plan? :)

Yes, both issues are fixed!

test that https://github.com/scalableminds/webknossos/issues/7652 is fixed
test that https://github.com/scalableminds/webknossos/issues/7584 is fixed

However, it would be great to show the mac user to use CMD click to split two segments.

… anyway

philippotto · 2024-03-25T16:39:06Z

However, it would be great to show the mac user to use CMD click to split two segments.

Good point, done!

daniel-wer

Mostly LGTM, but two things to consider :)

The current active tool seems to remain active when switching to flight or oblique mode. So when the brush is active before switching, the mouse pointer is wrong or when the proofreading tool is active before, skeletons cannot be modified. This is, although the tools are not visible and cannot be switched, there. This doesn't necessarily need to be fixed in this PR as the bug likely existed before, but users might hit it more frequently now that skeleton modifications are allowed in flight mode.
The casing of Shift and Ctrl in the context menu shortcuts is mixed now. Would be nice to align that.

frontend/javascripts/oxalis/geometries/cube.ts

frontend/javascripts/oxalis/geometries/materials/node_shader.ts

…rove-skeleton-proofreading

daniel-wer

LGTM, so many bug fixes and improvements in one PR - great work 🌈

…rove-skeleton-proofreading

philippotto added 5 commits March 7, 2024 14:19

allow complex shortcuts for merging/deleting trees in non-legacy mode…

74f4272

…; adapt status bar accordingly, too

improve typing

d727e4f

make node picking compatible with arbitary viewport

015e56f

use ortho-mode leftclick handler in arbitrary mode, too

3ad32b7

update status bar for skeleton related shortcuts in arbitrary view

04c8cba

philippotto self-assigned this Mar 7, 2024

philippotto added 2 commits March 7, 2024 18:03

fix cyclic dependency

0bf8a2a

allow deleting a node and creating a tree with the keyboard in arbitr…

008234d

…ary mode

philippotto requested a review from daniel-wer March 8, 2024 08:54

philippotto marked this pull request as ready for review March 8, 2024 08:54

philippotto added 5 commits March 8, 2024 09:54

Merge branch 'master' of github.com:scalableminds/webknossos into imp…

d3d33d0

…rove-skeleton-proofreading

update changelog

bdca4e7

handle metaKey as an alternative to ctrlKey to better support mac os

e323d88

show option instead of alt key in status bar and context menu when on…

2770c10

… MacOS

update changelog

df82be2

philippotto changed the title ~~Improve skeleton proofreading~~ Improve skeleton proofreading and fix some shortcuts on MacOS Mar 8, 2024

philippotto mentioned this pull request Mar 8, 2024

Zoom hotkey hint could be improved on macOS #7652

Closed

2 tasks

daniel-wer reviewed Mar 12, 2024

View reviewed changes

philippotto and others added 3 commits March 19, 2024 13:43

Apply suggestions from code review

26c4a5e

Co-authored-by: Daniel <daniel.werner@scalableminds.com>

incorporate PR feedback

92034f1

always show cursor for move tool in 3d viewport

6315448

philippotto added 2 commits March 19, 2024 15:00

bring docs for keyboard shortcuts up to date

3839b41

change color of active node a bit when not in ortho mode

a37923b

Merge branch 'master' into improve-skeleton-proofreading

6680a97

philippotto added 3 commits March 21, 2024 16:56

restore old camera.far code

f551d8b

avoid attaching store listener when cross sections are never rendered…

0f52d6c

… anyway

rename ctrl to ctrlOrCmd event in input.ts

f86fdbb

philippotto added 2 commits March 25, 2024 17:37

show Cmd instead of Ctrl in context menu when on OSX

99f0ae9

write ctrl/cmd everywhere in keyboard docs

c2feb20

remove unused import

3277bc1

daniel-wer approved these changes Mar 26, 2024

View reviewed changes

frontend/javascripts/oxalis/geometries/cube.ts Outdated Show resolved Hide resolved

frontend/javascripts/oxalis/geometries/materials/node_shader.ts Show resolved Hide resolved

philippotto added 5 commits March 26, 2024 17:45

remove accidental visibility code in cube.ts

aeea5d0

add comment about shifted hue for node in shader

2766297

SHIFT -> Shift

9923759

fix bg color of navbar buttons on hover in light mode

56c6c42

always show crosshair as cursor in arbitrary mode

ddee300

philippotto mentioned this pull request Mar 27, 2024

Clipping distance changes with zoom in arbitrary mode #7725

Open

3 tasks

Merge branch 'master' of github.com:scalableminds/webknossos into imp…

1c52802

…rove-skeleton-proofreading

daniel-wer approved these changes Mar 27, 2024

View reviewed changes

Merge branch 'master' of github.com:scalableminds/webknossos into imp…

a26dc62

…rove-skeleton-proofreading

philippotto enabled auto-merge (squash) March 27, 2024 10:16

philippotto merged commit 7f75576 into master Mar 27, 2024
2 checks passed

philippotto deleted the improve-skeleton-proofreading branch March 27, 2024 10:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve skeleton proofreading and fix some shortcuts on MacOS #7678

Improve skeleton proofreading and fix some shortcuts on MacOS #7678

philippotto commented Mar 7, 2024 •

edited

daniel-wer left a comment •

edited by philippotto

daniel-wer Mar 12, 2024

philippotto Mar 21, 2024

daniel-wer Mar 26, 2024

philippotto Mar 27, 2024

philippotto commented Mar 19, 2024

philippotto commented Mar 19, 2024

valentin-pinkau commented Mar 21, 2024

philippotto commented Mar 25, 2024

daniel-wer left a comment •

edited by philippotto

daniel-wer left a comment

Improve skeleton proofreading and fix some shortcuts on MacOS #7678

Improve skeleton proofreading and fix some shortcuts on MacOS #7678

Conversation

philippotto commented Mar 7, 2024 • edited

URL of deployed dev instance (used for testing):

Steps to test:

Issues:

daniel-wer left a comment • edited by philippotto

Choose a reason for hiding this comment

daniel-wer Mar 12, 2024

Choose a reason for hiding this comment

philippotto Mar 21, 2024

Choose a reason for hiding this comment

daniel-wer Mar 26, 2024

Choose a reason for hiding this comment

philippotto Mar 27, 2024

Choose a reason for hiding this comment

philippotto commented Mar 19, 2024

philippotto commented Mar 19, 2024

valentin-pinkau commented Mar 21, 2024

philippotto commented Mar 25, 2024

daniel-wer left a comment • edited by philippotto

Choose a reason for hiding this comment

daniel-wer left a comment

Choose a reason for hiding this comment

philippotto commented Mar 7, 2024 •

edited

daniel-wer left a comment •

edited by philippotto

daniel-wer left a comment •

edited by philippotto