Add guiEvent handling for web backends

[blink1073]

Fixes #4098.
Pass the jQuery event to the Python layer as the guiEvent.

[tacaswell]

Interesting. Did not know about guiEvent on Event objects. It looks like we don't use it anywhere internally.

👍 from me

attn @pelson

[blink1073]

Not currently working yet, but getting there...

[blink1073]

Okay, it works now. I tested with the UAT and this script:

import matplotlib
matplotlib.use('webagg')
import numpy as np
import matplotlib.pyplot as plt

fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(np.random.rand(10))


def onkeyevent(event):
    fig.suptitle('%s: %s' % (event.name, event.key))
    print('%s\n%s\n' % (event.name, event.guiEvent))
    fig.canvas.draw()

cid = fig.canvas.mpl_connect('key_press_event', onkeyevent)
cid = fig.canvas.mpl_connect('button_press_event', onkeyevent)
plt.show()

Output:

key_press_event
{'srcElement': '{ ... }', 'keyCode': 68, 'timeStamp': 1424014005997, 'cancelable': True, 'target': '{ ... }', 'metaKey': False, 'charCode': 0, 'eventPhase': 3, 'type': 'keydown', 'shiftKey': True, 'bubbles': True, 'currentTarget': '{ ... }', 'which': 68, 'ctrlKey': False, 'handleObj': '{ ... }', 'jQuery17104813391100615263': True, 'view': '{ ... }', 'altKey': False, 'delegateTarget': '{ ... }', 'originalEvent': '{ ... }', 'data': 'key_press'}

button_press_event
{'srcElement': '{ ... }', 'timeStamp': 1424014007711, 'cancelable': True, 'clientY': 310, 'clientX': 250, 'delegateTarget': '{ ... }', 'type': 'mousedown', 'pageY': 310, 'button': 0, 'which': 1, 'bubbles': True, 'jQuery17104813391100615263': True, 'target': '{ ... }', 'pageX': 250, 'originalEvent': '{ ... }', 'screenY': 389, 'screenX': 255, 'relatedTarget': '{ ... }', 'ctrlKey': False, 'altKey': False, 'eventPhase': 2, 'metaKey': False, 'shiftKey': False, 'currentTarget': '{ ... }', 'offsetY': 266, 'handleObj': '{ ... }', 'offsetX': 230, 'view': '{ ... }', 'fromElement': '{ ... }', 'toElement': '{ ... }', 'data': 'button_press'}

[blink1073]

Added support for originalEvent:

key_press_event
{'altKey': False, 'shiftKey': True, 'originalEvent': {'bubbles': True, 'repeat': False, 'path': '{ ... }', 'metaKey': False, 'srcElement': '{ ... }', 'returnValue': True, 'keyCode': 68, 'keyLocation': 0, 'keyIdentifier': 'U+0044', 'timeStamp': 1424014154131, 'layerY': 0, 'altKey': False, 'shiftKey': True, 'pageY': 0, 'cancelBubble': False, 'layerX': 0, 'cancelable': True, 'target': '{ ... }', 'type': 'keydown', 'defaultPrevented': False, 'eventPhase': 3, 'ctrlKey': False, 'detail': 0, 'currentTarget': '{ ... }', 'location': 0, 'charCode': 0, 'which': 68, 'pageX': 0, 'view': '{ ... }'}, 'bubbles': True, 'eventPhase': 3, 'type': 'keydown', 'metaKey': False, 'target': '{ ... }', 'srcElement': '{ ... }', 'handleObj': '{ ... }', 'keyCode': 68, 'ctrlKey': False, 'data': 'key_press', 'charCode': 0, 'currentTarget': '{ ... }', 'cancelable': True, 'timeStamp': 1424014154131, 'which': 68, 'delegateTarget': '{ ... }', 'jQuery17100706983779091388': True, 'view': '{ ... }'}

button_press_event
{'bubbles': True, 'currentTarget': '{ ... }', 'toElement': '{ ... }', 'metaKey': False, 'srcElement': '{ ... }', 'button': 0, 'relatedTarget': '{ ... }', 'offsetY': 300, 'timeStamp': 1424014155365, 'data': 'button_press', 'fromElement': '{ ... }', 'altKey': False, 'shiftKey': False, 'originalEvent': {'bubbles': True, 'currentTarget': '{ ... }', 'toElement': '{ ... }', 'path': '{ ... }', 'target': '{ ... }', 'metaKey': False, 'webkitMovementX': 0, 'srcElement': '{ ... }', 'returnValue': True, 'button': 0, 'keyCode': 0, 'movementY': 0, 'relatedTarget': '{ ... }', 'offsetY': 300, 'altKey': False, 'timeStamp': 1424014155365, 'layerY': 300, 'fromElement': '{ ... }', 'dataTransfer': '{ ... }', 'movementX': 0, 'shiftKey': False, 'screenX': 339, 'pageY': 344, 'eventPhase': 2, 'cancelBubble': False, 'layerX': 314, 'cancelable': True, 'defaultPrevented': False, 'type': 'mousedown', 'screenY': 423, 'pageX': 334, 'x': 334, 'ctrlKey': False, 'y': 344, 'charCode': 0, 'webkitMovementY': 0, 'clientX': 334, 'offsetX': 314, 'detail': 1, 'which': 1, 'clientY': 344, 'view': '{ ... }'}, 'screenX': 339, 'pageY': 344, 'delegateTarget': '{ ... }', 'type': 'mousedown', 'cancelable': True, 'target': '{ ... }', 'screenY': 423, 'pageX': 334, 'eventPhase': 2, 'ctrlKey': False, 'jQuery17100706983779091388': True, 'clientX': 334, 'offsetX': 314, 'handleObj': '{ ... }', 'which': 1, 'clientY': 344, 'view': '{ ... }'}

[blink1073]

@tacaswell, it would be nice to add these to the Qt backend, don't you think?

[blink1073]

I know I've queried the guiEvent before and was frustrated that it was None.

[tacaswell]

Yes it would. It seems tk, gtk, gtk3, and wx already do this...

[blink1073]

PR forthcoming 😄

[tacaswell]

@pelson this one too?

[pelson]

@pelson this one too?

Good question. I'm trying to get my head around that JS fn - it is pretty opaque 😄

[pelson]

What does the payload typically look like. Is its size a concern?

[pelson]

@pelson this one too?

I'm 👍 in favour of the change. I'd just like to get an idea of payload size, and would like to try to document the obscure JS fn a bit more.

[blink1073]

@pelson, I dumped a couple outputs a few comments up.

[blink1073]

We could grab a subset and avoid the issue altogether.

[pelson]

@pelson, I dumped a couple outputs a few comments up.

I'm clearly doing too many things at once... sorry about that

We could grab a subset and avoid the issue altogether.

I think that might make sense. That event payload is pretty busy! It might be nice to be able to control what is being sent over - but for now, let's hard-code it.

[blink1073]

@pelson, I decided to completely remove all nested objects. This results in 21 keys for a mouse event and 14 keys for a key event. I timed it and it takes about a millisec on average to simplify and send the message (on the JS side). The JS method is clearer now without the ternary operator, and I think the comment now speaks for itself.