Webview: traps keyboard events once focused

[bpasero]

• Electron Version: 3.0.0-beta.6
• Operating System (Platform and Version): macOS
• Last known working Electron version: 3.0.0-beta.5

Expected Behavior
The onkeydown event is fired when clicking into a webview contents and typing using the keyboard.

Actual behavior
The onkeydown event is not fired.

To Reproduce

• run this fiddle: https://gist.github.com/bpasero/52358eaaf2f45f4d7e5e5e3d94a56128
• click into the input box of the GitHub landing page
• type
• observe that the onkeydown event is not emitted

This breaks VSCode's keyboard handling once focus is within a webview.

[ckerr]

Confirmed exactly as described as @bpasero -- this broke sometime between 3.0.0-beta.5 and 3.0.0-beta.6

[MarshallOfSound]

@ckerr That commit range includes the OOPIF thing --> 44b0245

[zcbenz]

With webview being iframe, this issue essentially means how to trap keyboard events in iframe, and don't think there is any solution to it. It is actually surprising to me that keyboard events could be trapped in the old implementation, I think this is more a hack than expected behavior.

I'm afraid we can not fix this issue, VS Code probably has to work around it by manually installing event hooks into the webview.

I'll write down this behavior change in the docs.

[bpasero]

I am capturing the keyboard events from a webview element with the following code:

webview.getWebContents().on('before-input-event', (event, input) => {
	if (input.type !== 'keyDown') {
		return;
	}

	// Create a fake KeyboardEvent from the data provided
	const emulatedKeyboardEvent = new KeyboardEvent('keydown', {
		code: input.code,
		key: input.key,
		shiftKey: input.shift,
		altKey: input.alt,
		ctrlKey: input.control,
		metaKey: input.meta,
		repeat: input.isAutoRepeat
	});

	// do something with the event as before
});

[bpasero]

@zcbenz there is still an issue even with my workaround from #14258 (comment). I am not able to prevent a keyboard event from going to the main menu potentially triggering a menu item if it has that keybinding. This results in certain actions to fire twice when the keyboard shortcut is triggered while the webview has focus.

I created #14514 to follow up.

[pnegahdar]

@bpasero did you find that pages changes don't work when you add this listener? Very tough bug to nail down. but loadURL or changes to the src property of a webview tag no longer update the page with that simple input listener. Maybe its blocking some event propagation?

[ChocoPieLotte]

@bpasero did you find that pages changes don't work when you add this listener? Very tough bug to nail down. but loadURL or changes to the src property of a webview tag no longer update the page with that simple input listener. Maybe its blocking some event propagation?

I have the same problem

[vikt20]

Working example (catch keypress on div):

<div id="container" tabindex="0">Catch Event Here</div>

<script>
let id="container"
document.getElementById(id)?.addEventListener("mouseenter", function (event) {
    console.log(`mouse enter the div`);
    document.getElementById(id)?.focus()
});
document.getElementById(id)?.addEventListener("keydown", function (event) {
    console.log(event.key);
});
</script>

[KiruPoruno]

While trying to accomplish something similar to this, I tried various things, and came up with my own (arguably more modern) solution:

(async () => {
	while (true) {
		let key_event = webview.executeJavaScript(`
			new Promise((resolve) => {
				let event_handler = (e) => {
					e.target.removeEventListener("keyup", event_handler);

					resolve({
						key: e.key,
						code: e.code,
						repeat: e.repeat,
						altKey: e.altKey,
						ctrlKey: e.ctrlKey,
						metaKey: e.metaKey,
						shiftKey: e.shiftKey
					})
				}

				window.addEventListener("keyup", event_handler);
			})
		`);

		if (document.activeElement == webview) {
			console.log("Pressed inside webview:", key_event.key);
			window.dispatchEvent(new KeyboardEvent("keyup", key_event));
		}
	}
})()

This doesn't use any modules or anything, so it's safe to use on webviews that load arbitrary pages or similar. This works by simply running some JavaScript code with .executeJavaScript() which will return a Promise that'll eventually resolve to the latest key that's been pressed, it'll then use .dispatchEvent() to simulate a key press. And that'll then run in a loop, and due to .executeJavaScript() returning a Promise, we don't have to do any polling or similar.

In this example the variable: webview, is set to your <webview> tag. You can also adapt this to other events, i.e keydown, mousemove, mousedown, mouseup, and any event really.

You should keep in mind, that you could still be inside a text field typing with this, and it'd work the exact same way, so you may still have to do a few extra checks, as an example you could with .executeJavaScript() also return back the type of element which the webview considers in focus, and then do things from there.

Before doing this, I also had a far far more wonky way of doing it, but I suppose for the sake of completeness, I'll also note it down here...

This method requires you put something like pointer-events: none; on the webview, stopping the webview from ever getting focused. And then the below code (assuming the webview variable is set to the webview element), will simulate the keypresses from the renderer and into the webview.

This mostly works just fine, scrolling, clicking, :active, :hover, and even dragging all work. The only minor gripe is that :focus does not function. As it can never be focused. This also means all text fields are inherently non-functioning, if you don't however need this, then it's not a big issue.

// converts the renderer's mouse coordinates into ones that are
// equivalent in the webview itself, this means subtracting and adding
// pixels depends on where the webview is on screen and so forth, and if
// the position isn't even in view on the webview, then we return `false`
function window_to_tab_pos(x, y) {
	// contains all the needed properties for determining the webview's
	// position in the renderer
	let webview_rect = webview.getBoundingClientRect();

	// if `y` is above or below the webview, then we return `false`
	if (y < webview_rect.top
		|| y > webview_rect.bottom) {

		return false;
	}

	// if `x` is to the side of the webview, then we return `false`
	if (x < webview_rect.left
		|| x > webview_rect.right) {

		return false;
	}

	// return the subtracted coordinates
	return {
		x: x - webview_rect.x,
		y: y - webview_rect.y
	}
}

// sends a click event to a webview, using a click event (`e`)
function mouse_click(e) {
	// convert `e`'s coordinates to the webview equivalent
	let positions = window_to_tab_pos(e.x, e.y);

	// make sure they're in frame
	if (! positions) {return}

	// this may be mouseup or mousedown, or contextmenu
	let type = e.type;

	// make the 5th character uppercase, i.e "mouseDown"
	type[5] = type[5].toUpperCase();

	// if the type is actually "contextmenu" then set it correctly
	if (e.type == "contextmenu") {
		type = "contextMenu";
	}

	// set the button correctly, according to `e.button`
	let button = "left";
	switch(e.button) {
		case 1:
			button = "right";
			break;
		case 2:
			button = "middle";
	}

	// send the actual event
	webview.sendInputEvent({
		type: type,
		clickCount: 1,
		x: positions.x,
		y: positions.y,
		button: button
	})
}

// send a mouse move event to the webview using a `mousemove` event
function mouse_move(e) {
	// convert `e`'s coordinates to the webview equivalent
	let positions = window_to_tab_pos(e.x, e.y);

	// make sure they're in frame
	if (! positions) {return}

	// this may be mouseup or mousedown, or contextmenu
	let type = e.type;

	// make the 5th character uppercase, i.e "mouseDown"
	type[5] = type[5].toUpperCase();

	// send the actual event
	webview.sendInputEvent({
		type: type,
		x: positions.x,
		y: positions.y
	})
}

window.addEventListener("mouseup", mouse_click);
window.addEventListener("mousedown", mouse_click);

window.addEventListener("mousemove", mouse_move);
window.addEventListener("mouseenter", mouse_move);
window.addEventListener("mouseleave", mouse_move);
window.addEventListener("contextmenu", mouse_move);

// send the `mouseWheel` event to the webview
window.addEventListener("wheel", (e) => {
	// convert `e`'s coordinates to the webview equivalent
	let positions = window_to_tab_pos(e.x, e.y);

	// make sure they're in frame
	if (! positions) {return}

	// send the actual event
	webview.sendInputEvent({
		type: "mouseWheel",
		x: positions.x,
		y: positions.y,
		deltaX: e.deltaX * -1,
		deltaY: e.deltaY * -1,
	})
})

// send the `keyUp` event to the webview
window.addEventListener("keyup", (e) => {
	webview.sendInputEvent({
		type: "keyUp",
		keyCode: e.key
	})
})

// send the `keyDown` event to the webview
window.addEventListener("keydown", (e) => {
	webview.sendInputEvent({
		type: "keyDown",
		keyCode: e.key
	})
})

Hopefully this is all useful to at least one person, if anything, I at least got what I needed out of this.