Skip to content

Commit

Permalink
New Toolbar (#4875)
Browse files Browse the repository at this point in the history
* implement some small performance improvements

* implement further small perf improvements

* fix linting

* avoid Three.Vector3 creation, vector unpacking etc. for position updates

* avoid some THREE.Vector3 instantiations and redundant set of scale

* fix linting

* only set position in updateCameraForScene

* use nanoEvents for buckets instead of backbone events

* avoid array allocation in TextureBucketManager

* fix test

* rework toolbar: use icons for tools; show current cell id in navbar
etc.

* reflect modifier change directly in toolbar

* temporarily disable most CI checks

* work on dynamic overwrite-mode in UI

* refactor overwrite-cell mechanism

* fix tests

* fix test and restore CI

* fix broken merge and some padding

* fix useEffect toggle bug; also adapt active tool when pressing Alt

* improve tooltips for tools and fix CSS issues with borders on hover

* move hooks into own module

* also show tooltips when pressing modifiers in move-tool; fix missing keyRelease-bug when leaving browser tab

* fix that ctrl+right-click always re-centered the current node

* remove unused idle enum

* swap brush and trace-tool; make lasso.svg blue via css-filtering when being active

* support dragging the plane via middle-click

* update changelog

* only re-render SaveButton if necessary (by explicitly passing progressFraction and by accessing the oldest unsaved timestamp ad-hoc)

* fix cycling order of tools; update changelog

* Apply suggestions from code review

Co-authored-by: Daniel <daniel.werner@scalableminds.com>

* remove duplicate from changelog

* allow middle-drag-movement in all tracing modes; add more comments

* fix border on hovering disabled radio button

* respect mapping when rendering the current cell color in the new-cell-color-button

* workaround modifier-release-bug and adapt merge-tree-message

* satisfy linter

Co-authored-by: Daniel <daniel.werner@scalableminds.com>
  • Loading branch information
philippotto and daniel-wer committed Oct 22, 2020
1 parent 0553f8e commit aa959cd
Show file tree
Hide file tree
Showing 27 changed files with 661 additions and 192 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.unreleased.md
Expand Up @@ -14,6 +14,7 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
- Hybrid tracings can now be imported directly in the tracing view via drag'n'drop. [#4837](https://github.com/scalableminds/webknossos/pull/4837)
- The find data function now works for volume tracings, too. [#4847](https://github.com/scalableminds/webknossos/pull/4847)
- Added admins and dataset managers to dataset access list, as they can access all datasets of the organization. [#4862](https://github.com/scalableminds/webknossos/pull/4862)
- Added the possibility to move the current position by dragging with the middle-mouse-button (regardless of the active tool). [#4875](https://github.com/scalableminds/webknossos/pull/4875)
- Sped up the NML parsing via dashboard import. [#4872](https://github.com/scalableminds/webknossos/pull/4872)
- Movements in the 3D viewport are now time-tracked. [#4876](https://github.com/scalableminds/webknossos/pull/4876)

Expand All @@ -23,6 +24,8 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
- webknossos.org only: Accounts associated with new organizations can now be created even when a datastore is unreachable. The necessary folders are created lazily when needed. [#4846](https://github.com/scalableminds/webknossos/pull/4846)
- When downloading a volume tracing, buckets containing a single 0 byte, that were created to restore older versions, are now skipped. [#4851](https://github.com/scalableminds/webknossos/pull/4851)
- Task information CSV now contains additional column `creationInfo`, containing the original NML filename for tasks based on existing NMLs. [#4866](https://github.com/scalableminds/webknossos/pull/4866)
- Improved the toolbar to make the different webKnossos tools easier to use. For example, the fill-tool and the cell-picker have a dedicated button in volume annotations now. [#4875](https://github.com/scalableminds/webknossos/pull/4875)
- The default overwrite-behavior in volume annotating changed so that erasing with the brush- or trace-tool always erases all voxels (regardless of their segment id). Before that, only the current segment id was overwritten by default. As before, this behavior can be toggled by pressing CTRL. Alternatively, one can now also switch the mode in the toolbar. [#4875](https://github.com/scalableminds/webknossos/pull/4875)
- The dataset upload is now more robust and can recover from a failed upload of a data chunk. [#4860](https://github.com/scalableminds/webknossos/pull/4860)

### Fixed
Expand Down
1 change: 1 addition & 0 deletions docs/volume_annotation.md
Expand Up @@ -11,6 +11,7 @@ Select one of the drawing tools from the toolbar or toggle through with the keyb
- `Move`: Navigate around the dataset.
- `Trace`: Draw outlines around the voxel you would like to label.
- `Brush`: Draw over the voxels you would like to label. Adjust the brush size with *SHIFT + Mousewheel*.
- `Fill Tool`: Flood-fill the clicked region. All adjacent voxels with the same voxel id as the clicked voxel will be changed to the active cell id.

Add labels with *Left Mouse Drag*.
Remove labels with *Right Mouse Drag*.
Expand Down
9 changes: 7 additions & 2 deletions frontend/javascripts/libs/input.js
Expand Up @@ -38,8 +38,8 @@ type KeyboardLoopHandler = {
type KeyboardBindingPress = [KeyboardKey, KeyboardHandler, KeyboardHandler];
type KeyboardBindingDownUp = [KeyboardKey, KeyboardHandler, KeyboardHandler];
type BindingMap<T: Function> = { [key: KeyboardKey]: T };
type MouseButtonWhich = 1 | 3;
type MouseButtonString = "left" | "right";
type MouseButtonWhich = 1 | 2 | 3;
type MouseButtonString = "left" | "middle" | "right";
type MouseHandler =
| ((deltaY: number, modifier: ?ModifierKeys) => void)
| ((position: Point2, id: ?string, event: MouseEvent) => void)
Expand Down Expand Up @@ -308,6 +308,7 @@ export class InputMouse {
id: ?string;

leftMouseButton: InputMouseButton;
middleMouseButton: InputMouseButton;
rightMouseButton: InputMouseButton;
isMouseOver: boolean = false;
lastPosition: ?Point2 = null;
Expand Down Expand Up @@ -338,6 +339,7 @@ export class InputMouse {
this.id = id;

this.leftMouseButton = new InputMouseButton("left", 1, this, this.id);
this.middleMouseButton = new InputMouseButton("middle", 2, this, this.id);
this.rightMouseButton = new InputMouseButton("right", 3, this, this.id);
this.lastPosition = null;
this.delegatedEvents = {};
Expand Down Expand Up @@ -416,12 +418,14 @@ export class InputMouse {
this.lastPosition = this.getRelativeMousePosition(event);

this.leftMouseButton.handleMouseDown(event);
this.middleMouseButton.handleMouseDown(event);
this.rightMouseButton.handleMouseDown(event);
};

mouseUp = (event: MouseEvent): void => {
isDragging = false;
this.leftMouseButton.handleMouseUp(event, this.triggeredByTouch);
this.middleMouseButton.handleMouseUp(event, this.triggeredByTouch);
this.rightMouseButton.handleMouseUp(event, this.triggeredByTouch);

this.triggeredByTouch = false;
Expand Down Expand Up @@ -458,6 +462,7 @@ export class InputMouse {

if (delta != null && (delta.x !== 0 || delta.y !== 0)) {
this.leftMouseButton.handleMouseMove(event, delta);
this.middleMouseButton.handleMouseMove(event, delta);
this.rightMouseButton.handleMouseMove(event, delta);
if (this.isHit(event)) {
this.trigger("mouseMove", delta, this.position, this.id, event);
Expand Down
87 changes: 87 additions & 0 deletions frontend/javascripts/libs/react_hooks.js
@@ -0,0 +1,87 @@
// @flow
import { useState, useEffect, useRef } from "react";

// Adapted from: https://usehooks.com/usePrevious/
export function usePrevious<T>(value: T): ?T {
// The ref object is a generic container whose current property is mutable ...
// ... and can hold any value, similar to an instance property on a class
const ref = useRef<?T>(null);

// Store current value in ref
useEffect(() => {
ref.current = value;
}, [value]); // Only re-run if value changes

// Return previous value (happens before update in useEffect above)
return ref.current;
}

const extractModifierState = event => ({
Shift: event.shiftKey,
Alt: event.altKey,
Control: event.ctrlKey,
});

// Adapted from: https://gist.github.com/gragland/b61b8f46114edbcf2a9e4bd5eb9f47f5
export function useKeyPress(targetKey: string) {
// State for keeping track of whether key is pressed
const [keyPressed, setKeyPressed] = useState(false);

// If pressed key is our target key then set to true
function downHandler(event) {
const modifierState = extractModifierState(event);

if (modifierState[targetKey] === undefined) {
// The targetKey is not a modifier. Compare to the pressed key.
if (event.key === targetKey) {
pressKey();
}
} else if (modifierState[targetKey]) {
// Use the modifierState as this seems to be more robust. See
// the other comment below which describes some edge cases
// regarding modifiers.
pressKey();
}
}

function pressKey() {
setKeyPressed(true);
window.addEventListener("blur", releaseKey);
}

function releaseKey() {
setKeyPressed(false);
window.removeEventListener("blur", releaseKey);
}

// If released key is our target key then set to false
const upHandler = event => {
const modifierState = extractModifierState(event);

if (modifierState[targetKey] === undefined) {
// The targetKey is not a modifier. Compare to the pressed key.
if (event.key === targetKey) {
releaseKey();
}
} else if (!modifierState[targetKey]) {
// The targetKey is a modifier. Use the modifierState as this
// is more robust against pressing multiple modifiers. For example,
// on Linux, pressing Shift and then toggling Alt, will send a release
// of the Meta key even though that key was never touched.
releaseKey();
}
};

// Add event listeners
useEffect(() => {
window.addEventListener("keydown", downHandler);
window.addEventListener("keyup", upHandler);
// Remove event listeners on cleanup
return () => {
window.removeEventListener("keydown", downHandler);
window.removeEventListener("keyup", upHandler);
};
}, []); // Empty array ensures that effect is only run on mount and unmount

return keyPressed;
}
17 changes: 12 additions & 5 deletions frontend/javascripts/oxalis/constants.js
Expand Up @@ -94,24 +94,31 @@ export type ControlMode = $Keys<typeof ControlModeEnum>;

export const VolumeToolEnum = {
MOVE: "MOVE",
TRACE: "TRACE",
BRUSH: "BRUSH",
TRACE: "TRACE",
FILL_CELL: "FILL_CELL",
PICK_CELL: "PICK_CELL",
};
export const ToolsWithOverwriteCapabilities = [VolumeToolEnum.TRACE, VolumeToolEnum.BRUSH];
export type VolumeTool = $Keys<typeof VolumeToolEnum>;

export function volumeToolEnumToIndex(volumeTool: ?VolumeTool): number {
return Object.keys(VolumeToolEnum).indexOf(volumeTool);
}

export const ContourModeEnum = {
IDLE: "IDLE",
DRAW: "DRAW",
DRAW_OVERWRITE: "DRAW_OVERWRITE",
DELETE_FROM_ACTIVE_CELL: "DELETE_FROM_ACTIVE_CELL",
DELETE_FROM_ANY_CELL: "DELETE_FROM_ANY_CELL",
DELETE: "DELETE",
};
export type ContourMode = $Keys<typeof ContourModeEnum>;

export const OverwriteModeEnum = {
OVERWRITE_ALL: "OVERWRITE_ALL",
OVERWRITE_EMPTY: "OVERWRITE_EMPTY", // In case of deleting, empty === current cell id
};

export type OverwriteMode = $Keys<typeof OverwriteModeEnum>;

export const NODE_ID_REF_REGEX = /#([0-9]+)/g;
export const POSITION_REF_REGEX = /#\(([0-9]+,[0-9]+,[0-9]+)\)/g;

Expand Down
Expand Up @@ -12,7 +12,7 @@ import {
VolumeToolEnum,
} from "oxalis/constants";
import { V3 } from "libs/mjs";
import { calculateGlobalPos } from "oxalis/controller/viewmodes/plane_controller";
import { movePlane, calculateGlobalPos } from "oxalis/controller/viewmodes/plane_controller";
import { enforce } from "libs/utils";
import {
enforceSkeletonTracing,
Expand Down Expand Up @@ -41,10 +41,7 @@ import {
setNodePositionAction,
updateNavigationListAction,
} from "oxalis/model/actions/skeletontracing_actions";
import {
setDirectionAction,
movePlaneFlycamOrthoAction,
} from "oxalis/model/actions/flycam_actions";
import { setDirectionAction } from "oxalis/model/actions/flycam_actions";
import type PlaneView from "oxalis/view/plane_view";
import Store from "oxalis/store";
import type { Edge, Tree, Node } from "oxalis/store";
Expand Down Expand Up @@ -79,20 +76,17 @@ export function getPlaneMouseControls(planeView: PlaneView) {
return {
leftDownMove: (delta: Point2, pos: Point2, _id: ?string, event: MouseEvent) => {
const { tracing } = Store.getState();
const state = Store.getState();
if (tracing.skeleton != null && event.ctrlKey) {
moveNode(delta.x, delta.y);
} else {
const { activeViewport } = state.viewModeData.plane;
const v = [-delta.x, -delta.y, 0];
Store.dispatch(movePlaneFlycamOrthoAction(v, activeViewport, true));
movePlane([-delta.x, -delta.y, 0]);
}
},
leftClick: (pos: Point2, plane: OrthoView, event: MouseEvent, isTouch: boolean) =>
onClick(planeView, pos, event.shiftKey, event.altKey, event.ctrlKey, plane, isTouch, event),
rightClick: (pos: Point2, plane: OrthoView, event: MouseEvent) => {
const { volume } = Store.getState().tracing;
if (!volume || volume.activeTool !== VolumeToolEnum.BRUSH) {
if (!volume || volume.activeTool === VolumeToolEnum.MOVE) {
// We avoid creating nodes when in brushing mode.
setWaypoint(calculateGlobalPos(pos), event.ctrlKey);
}
Expand Down Expand Up @@ -354,13 +348,10 @@ function setWaypoint(position: Vector3, ctrlPressed: boolean): void {
const rotation = getRotationOrtho(activeViewport);
addNode(position, rotation, !ctrlPressed);

// Strg + Rightclick to set new not active branchpoint
// Ctrl + right click to set new not active branchpoint
const { newNodeNewTree } = Store.getState().userConfiguration;
if (ctrlPressed && !newNodeNewTree) {
Store.dispatch(createBranchPointAction());
activeNodeMaybe.map(activeNode => {
Store.dispatch(setActiveNodeAction(activeNode.id));
});
}
}

Expand All @@ -384,6 +375,8 @@ function addNode(position: Vector3, rotation: Vector3, centered: boolean): void
rotation,
OrthoViewToNumber[Store.getState().viewModeData.plane.activeViewport],
getRequestLogZoomStep(state),
null,
!centered,
),
);

Expand Down
Expand Up @@ -86,8 +86,7 @@ export function getPlaneMouseControls(_planeId: OrthoView): * {

if (
(tool === VolumeToolEnum.TRACE || tool === VolumeToolEnum.BRUSH) &&
(contourTracingMode === ContourModeEnum.DRAW ||
contourTracingMode === ContourModeEnum.DRAW_OVERWRITE)
contourTracingMode === ContourModeEnum.DRAW
) {
Store.dispatch(addToLayerAction(calculateGlobalPos(pos)));
}
Expand All @@ -97,23 +96,16 @@ export function getPlaneMouseControls(_planeId: OrthoView): * {
const tool = Utils.enforce(getVolumeTool)(Store.getState().tracing.volume);

if (!event.shiftKey && (tool === VolumeToolEnum.TRACE || tool === VolumeToolEnum.BRUSH)) {
if (event.ctrlKey) {
if (isAutomaticBrushEnabled()) {
return;
}
Store.dispatch(setContourTracingModeAction(ContourModeEnum.DRAW));
} else {
Store.dispatch(setContourTracingModeAction(ContourModeEnum.DRAW_OVERWRITE));
if (event.ctrlKey && isAutomaticBrushEnabled()) {
return;
}
Store.dispatch(setContourTracingModeAction(ContourModeEnum.DRAW));
Store.dispatch(startEditingAction(calculateGlobalPos(pos), plane));
}
},

leftMouseUp: () => {
const tool = Utils.enforce(getVolumeTool)(Store.getState().tracing.volume);

Store.dispatch(setContourTracingModeAction(ContourModeEnum.IDLE));

if (tool === VolumeToolEnum.TRACE || tool === VolumeToolEnum.BRUSH) {
Store.dispatch(finishEditingAction());
}
Expand All @@ -127,8 +119,7 @@ export function getPlaneMouseControls(_planeId: OrthoView): * {

if (
(tool === VolumeToolEnum.TRACE || tool === VolumeToolEnum.BRUSH) &&
(contourTracingMode === ContourModeEnum.DELETE_FROM_ACTIVE_CELL ||
contourTracingMode === ContourModeEnum.DELETE_FROM_ANY_CELL)
contourTracingMode === ContourModeEnum.DELETE
) {
Store.dispatch(addToLayerAction(calculateGlobalPos(pos)));
}
Expand All @@ -138,28 +129,28 @@ export function getPlaneMouseControls(_planeId: OrthoView): * {
const tool = Utils.enforce(getVolumeTool)(Store.getState().tracing.volume);

if (!event.shiftKey && (tool === VolumeToolEnum.TRACE || tool === VolumeToolEnum.BRUSH)) {
if (event.ctrlKey) {
Store.dispatch(setContourTracingModeAction(ContourModeEnum.DELETE_FROM_ANY_CELL));
} else {
Store.dispatch(setContourTracingModeAction(ContourModeEnum.DELETE_FROM_ACTIVE_CELL));
}
Store.dispatch(setContourTracingModeAction(ContourModeEnum.DELETE));
Store.dispatch(startEditingAction(calculateGlobalPos(pos), plane));
}
},

rightMouseUp: () => {
const tool = Utils.enforce(getVolumeTool)(Store.getState().tracing.volume);

Store.dispatch(setContourTracingModeAction(ContourModeEnum.IDLE));

if (tool === VolumeToolEnum.TRACE || tool === VolumeToolEnum.BRUSH) {
Store.dispatch(finishEditingAction());
Store.dispatch(setContourTracingModeAction(ContourModeEnum.IDLE));
}
},

leftClick: (pos: Point2, plane: OrthoView, event: MouseEvent) => {
if (event.shiftKey && !event.ctrlKey) {
const tool = Utils.enforce(getVolumeTool)(Store.getState().tracing.volume);

const shouldPickCell =
tool === VolumeToolEnum.PICK_CELL || (event.shiftKey && !event.ctrlKey);

const shouldFillCell = tool === VolumeToolEnum.FILL_CELL || (event.shiftKey && event.ctrlKey);

if (shouldPickCell) {
const segmentation = Model.getSegmentationLayer();
if (!segmentation) {
return;
Expand All @@ -172,7 +163,7 @@ export function getPlaneMouseControls(_planeId: OrthoView): * {
Store.dispatch(setActiveCellAction(cellId));
isosurfaceLeftClick(pos, plane, event);
}
} else if (event.shiftKey && event.ctrlKey) {
} else if (shouldFillCell) {
Store.dispatch(floodFillAction(calculateGlobalPos(pos), plane));
} else if (event.metaKey) {
if (isAutomaticBrushEnabled()) {
Expand Down

0 comments on commit aa959cd

Please sign in to comment.