Mouse camera controls can get temporarily 'stuck on'

[wallw-teal]

This is best demonstrated by the following in Sandcastle:

var viewer = new Cesium.Viewer('cesiumContainer');
var controller = viewer.scene.screenSpaceCameraController;
controller.enableLook = false;

To produce the issue, do the following in order:

• Mouse down
• Shift down
• Mouse up
• Shift up

Cesium now rotates the globe with mouse move even though both mouse buttons are up. Clicking the globe gets the camera controller back in the correct state.

Obviously this demo isn't actually doing anything with shift+drag, but my application is using it to start drawing regions. controller.enableInputs is false during that interaction and changed back to true when it finishes, but this issue still occurs.

Any suggestions for solutions at the application level are also welcome. I'm aware that I could re-implement all of the camera movement myself, but that seems a bit ridiculous.

[emackey]

I think this bug is in CameraEventAggregator, which does its own button up/down tracking differently from how ScreenSpaceEventHandler tracks up/down state. The former is too particular about tracking multiple buttons at once, and thinks that shift-mouse-up is a different button being released than the non-shift button-down. ScreenSpaceEventHandler takes a simpler approach, where any mouse up event is a release of any/all buttons, but that knowledge doesn't propagate down into CameraEventAggregator, hence the bug.

At the very least, both classes should use the same button-tracking strategy. But ideally, we wouldn't have multiple independent button tracking systems in Cesium.

[WarpDrive]

Is it intentional that camera look is left out of the Navigation Instructions button on the top right?
(edited the rest of this out since it was simply a specific case, my next post has a more useful generalization)

[WarpDrive]

If you press/hold any mouse button first:
-ctrl and shift presses go unrecorded (drag just moves pointer)
-however if you let go of the mouse button while ctrl or shift is down mouse up will go unrecorded (as you'll notice once you do let go of the modifier key)

Cesium has a recording problem:
-If any mouse button is down it won't record modifier key down.
-If any modifier key is down it won't record mouse buttons release.(if mouse button pushed first)

Button/key press/release recordings shouldn't depend on conditions, they should simply be recorded.

[emackey]

@WarpDrive what's happening is that Cesium tracks buttons individually by both button and modifier, so it thinks that "LEFT" is a separate button from "SHIFT+LEFT", which is a separate button from "CTRL+LEFT". So, a mouse-down on LEFT, followed by a mouse-up on SHIFT-LEFT, releases a "different" button than the one that was down. You can do this the other way around too, push down on SHIFT-LEFT and release LEFT to get the other one stuck down, and then you can use the SHIFT key by itself (with no mouse button down) to look around.

This is a bug, but it's fairly deep-rooted. The CameraEventAggregator does this complex per-button-per-modifier tracking, based on upstream events coming from ScreenSpaceEventHandler. But the upstream one does its own button-down tracking, much more simply with a single boolean flag indicating that any button is down, and uses that to "capture" the mouse if it drifts off the canvas but stays in the document. So you can get into situations where you push down both LEFT and RIGHT, then move the mouse off the canvas but still over the document, then release the two buttons, and CameraEventAggregator will only get one of the two mouse-up messages because ScreenSpaceEventHandler has given up playing mouse capture. It takes some doing to get your mouse unstuck from that one. The whole system could use an overhaul.

[WarpDrive]

It would be nice if browsers would provide a list of pushed keys and buttons (like GamePad API does), in addition to providing changes to key/button states. If you hold a key/button down then the document loses focus (like clicking on the URL bar), then you let go of the key/button then go back to the document how the heck is the document going to know that the key/button was let go? When losing focus you could assume all keys are let go, but that might not be the case either.

For example http://cesiumjs.org/Cesium/Apps/Sandcastle/index.html?src=Camera%20Tutorial.html&label=Showcases press and hold w, then click on the URL bar, then let go of w, it doesn't fire a keyup event.

[daguej]

This sadly remains a bug eight years later in v1.102.

I was able to monkey patch the bug by monitoring for mousedown/mouseup pairs with mismatched modifier keys, and then forcibly calling the Cesium camera's mouseup handler function for the same modifier key that was used when the mouse went down (if any). This gets Cesium's internal state to correctly reflect the fact that the mouse button is no longer down, and the camera controls no longer get 'stuck'.

const canvasEl = viewer.canvas;
const cameraEventAgg = viewer.scene.screenSpaceCameraController._aggregator;

let downModifiers = null;

function onpointerdown(e) {
    if (e.pointerType == 'mouse') { 
        downModifiers = new CesiumModifiers(e.shiftKey, e.ctrlKey, e.altKey);
    }
}
function onpointerup(e) {
    if (e.pointerType == 'mouse') {
        if (downModifiers && (
            downModifiers.shift != e.shiftKey ||
            downModifiers.ctrl != e.ctrlKey ||
            downModifiers.alt != e.altKey)) {
                const modifier = downModifiers?.toCesiumType();
                downModifiers = null;
                setTimeout(() => { // to ensure we don't cause further weird behavior, call the event handler in the next tick
                    const handler = cameraEventAgg._eventHandler;
                    if (handler) {
                        handler.getInputAction(domMouseButtonToCesiumEventType(e), modifier)();
                        downModifiers = null;   
                    }
                }, 0);
            }
    }
}

canvasEl.addEventListener('pointerdown', onpointerdown);
canvasEl.addEventListener('pointerup', onpointerup);

class CesiumModifiers {
    constructor(shift, ctrl, alt) {
        this.shift = shift;
        this.ctrl = ctrl;
        this.alt = alt;
    }

    toCesiumType() {
        if (this.shift) return 0;
        if (this.ctrl) return 1;
        if (this.alt) return 2;
        // none = undefined
    }
}

function domMouseButtonToCesiumEventType(button) {
    switch (button) {
        case 0: return ScreenSpaceEventType.LEFT_UP;
        case 1: return ScreenSpaceEventType.MIDDLE_UP;
        case 2: return ScreenSpaceEventType.RIGHT_UP;
    }
}

[ggetz]

@daguej Would you be able to open a PR with your fix so the monkey patch isn't necessary? 😀