Would it be possible to add Native Event Handlers

[shawty]

At the request of @liminzhu on issue #253 I'm opening this as a new issue so it can be more easily tracked.

Following the recent discussion on the issue mentioned above, a number of users (Myself included) have come to the conclusion that adding a native pass through to the C# side of things for common JavaScript events would be a good idea.

Primarily the request was asked about passing through either the "DOMContentReady" or "load" event handlers so that there was a notification on the C# side when either of these 2 JavaScript events where raised.

As the discussion progressed however, it became apparent that a universal event system would be more beneficial.

There are some common events such as the load events, and possibly click, blur and focus events that should have ready made interception points, but in general something like a 'RaiseEvent("eventName", new { ... event data ... })' in C#, along with a corresponding "onEventRaised(sender, string eventName, new { ... event data ...})" would help with linking up C# code to JS code.

In my Custom browser (https://github.com/shawty/hbbtvbrowserEXPERIMENTAL) I've more or less done that myself using the existing message passing features, so I don't think it would be too difficult for a permanent JavaScript stub to be added to the existing code in the control that performed a similar task and surfaced these events in an easier to digest way.

AB#27010334

[liminzhu]

Thanks for opening the issue. I generally agree having some form of native sugar is more ergonomic than calling ExecuteScript/AddScriptToExecuteOnDocumentCreated.

I do want to avoid blanket-wrapping all DOM events into native events (i.e. support something like wv.CoreWebView.<any DOM event> += handler). There are a lot of them for us to wrap, and web is moving standard, which means we constantly have to look out for new web events and if web standards decide to delete/break, we are either forced to break native developers or keep on supporting behavior that the web platform does not. This is precisely why we do not want to make native APIs to support DOM manipulation and advise developers to call scripts instead. We can brainstorm how to make the experience better without literally wrapping DOM events.

[shawty]

@liminzhu Agreed with everything you stated about blanket wrapping, which is why I'm suggesting, wrapping just a core set of the most commonly used ones, so that those that are just looking for a quick hook to do a simple job will find them easily, and then a more generic solution to allow others to more easily implement thier own specific hooks.

Let's consider what I've been doing as an example.

If I want to raise an event from JS to C# currently I might:

1. Wrap a webview post message in a JS event hook

2. create an event handler on the message recived event in C#

3. when a JS event occurs, create a custom JSON structure, add to that a string with the event name in, and also add to that any pertinent data from the event.

4. Receive the message in C#, then first check my name field for an event name, if there's no name field then it must be something else, or what if I receive a message that is something else but has a name filed, ok that's fine, we'll also add a type or tag field of some kind that says "Event"... so then we have to look for that, and if we have that then look for the name and then unpack the data...

Step 4 is a silly amount of work for a simple task... we could easily do the following

1. Wrap a webview.postEvent call in a JS message handler

2. add a C# handler for "EventRecieved"

3. when we call post event, we supply a string, eventName and a generic object of data

4. when we recive the event, we already know it's a JS event, and the name is in a seperate string outside the data, so we can easily use that with something like the new switch statement syntax, and pattern matching against the data object.

Job done.

By doing this, we make it easy to get the most common events, but still maintain an easy route for the developer to extend as they see fit.

Common ones would never be expected to change, and even more so, because we don't expect the common set to change, we can also provide strongly typed event models in the handler, meaning for simple cases we don't have to start matching data models either, we just hook em up and go....

[liminzhu]

Yeah that's fair and I get the ergonomic argument. If there are events that a significant amount of hosts would want to handle, it is very justifiable that we expose a native event for it. The next thing is to decide what constitutes the core set of events, and the priority in which we'd expose them, which I think would base on how much feedback we get for each event.

[ukandrewc]

There is an experimental inherited control to do this, can be extended to raise custom events: https://github.com/ukandrewc/Webview2.Winforms.DOM

[shawty]

@liminzhu There are a lot of places we could look for information on events to expose, but I don't think any of them have any usage stats.

The current DOM Level 3 events as designated by the W3C can be found here:
https://www.w3.org/TR/DOM-Level-3-Events/

IMHO, we definitely need the DOMReady and Load events, simply because that's the only way we have of knowing when the browser instance has told us "Hey your document is fully loaded and ready for you to do stuff with", and given that the primary use case is for apps that manipulate the document, then 99% of cases are going to want to know when this happens.

Likewise I think "unload" is also a pertinent one to include, I know from experience that in a regular browser many of them don't fire the unload event, and in many cases one of the reasons for using a custom browser instance is so that the "container" can detect if a document is unloaded.

Again drawing on an example of what I've been working on, part of the OIPF specification (The Application Manager) actually needs to be able to tell when an application exits, so it can free up resources it may have used.

Abort & Error may also be good to surface too, again these are not commonly seen in the main browsers, but in a custom browser generally It will be being used to load only specific documents, and so knowing if something failed to load would be an advantage.

I don't think focus events are a particularly good idea, largely because there can be so many different levels of these. We know we get a focus when the document is clicked in, but we also get them for ANY element that is focusable, and that doesn't just mean buttons an input boxes. If you take a div element for example and add a tab index attribute EG: '

Blah

' that entire div will then fire "onFocus" events when the control is focused.

Now don't get me wrong, IF your building a browser for special needs for example, EG: a person with vision problems, then having tab order allowing you to "spatially navigate" the document while a screen reader reads things aloud, is a great idea, and most mainstream browsers will handle that just fine.

In a custom app using webview however, depending on the speed the person navigates the keyboard and the number of focusable elements there are in the page it may a) NOT be possible to hook them all (Especially if they are dynamically inserted in JS code) and b)Might cause to many performance issues IE: rapidly calling the C# event handler.

Mouse events I personally think would be expected by users of the control, I think we already have mouse in/mouse out (Can't remember :-) ) not sure if we want to add "click" and "dblclick" though.

The click events have many of the same problems that the focus events do, you would need to be monitoring the DOM constantly, and for ANY mouse clickable element added to the DOM you would need to attach an interception for the click, You could attach a click handler only to the document, that would catch clicks on the document itself and any that are bubbled (You might wanna have a chat with Steve Sanderson on the Blazor team about that one as he did a lot of analysis on how best to deal with this while developing Blazor), trying to handle and surface every click would I think be far too difficult.

One exception where you might want to handle click specifically however, would be an HTML canvas, that would allow those folks who specifically want to draw custom UI's directly at a pixel level, or at a control level, to also manage their own UI surface. Bare minimum would be click co-ordinates as an offset from the 0,0 origin of the drawing surface, and potentially at a much later date the ability to "pick" a shape that might have been drawn under the mouse pointer. This is currently something that all the browsers struggle to do accurately, but would be an absolute boon, for those writing HTML based games, and then wanting to use something like webview to make that game a desktop application.

I don't know if tapping keyup/down is a good idea, again namely for the same reasons as mouse, but possibly one handler at document level might be worth doing. Once place where a key event is a great idea however is from C# into JS. Being able to make the application send a virtual key code to webview is going to be massive for those folks writing HTML test systems.

If you could take a winforms program, using webview. Point it at a newly deployed azure app, record a series of key presses, up, down, left, right, type, enter etc that recorded a testable set of actions, then play those key codes back you could very easily build a UI test tool that could compete with things like PhantomJS but would be visible (Imagine if you will visible web UI testing integrated into visual studio), if you look at my browser example, you'll see that I've done exactly that in order to simulate using a remote control, in my private build I also have an infra-red receiver attached to my PC, that uses winLirc to generate key codes using an actual TV set remote control.

Document Keyup/Keydown i think would suffice and be expected by most, and fire only on bubbled events, and mouse in/out along with a "GetMouseXY()" rather than events to realtime track the mouse, would in my opinion be the minimum for the input side of things.

Aside from what I've documented above, I personally think DOM Change events are going to be the most commonly requested by most users, so they can track changes to the loaded document. In many cases the webview control is going to be used for screen scraping, so being able to know that an element has been added to or removed from the dom will probably be something highly requested.

I think painting various areas too, allowing a canvas tag to be painted from C# code will open up a huge amount of opportunity's, as I mentioned in my issue on using object tags, knowing when to paint the rectangle in those would be useful.

Events raised on the DOM Validation API's I think would also be a great candidate for adding native handlers, that would allow someone to handle form validation errors in a web UI using C# code. To give you a bit of an idea what I'm thinking with that, you might want to read this blog post I did a while back:

https://shawtyds.wordpress.com/2019/04/05/pure-html-validation-in-blazor/

A LOT of people who want to embed a HTML UI in a winforms or wpf application, will typically use something like bootstrap for the layout. I've done it many times myself in the past and even with easy to use toolkits like CEFSharp (Which I've also blogged about in the past : https://www.codeguru.com/columns/dotnet/if-you-like-it-put-an-html5-ui-on-it.html ) knowing when you have a validation error can be tricky.

That's all I can think of at the moment, sorry it's a bit of a novel :-)

Shawty

[ShaunLoganOracle]

To chime in on a base set of events for WebView2 to expose to C#: I am migrating my usage of System.Windows.Forms.WebBrowser to WebView2 and I would really like to have easy access to the equivalents of these WebBrowser events: Navigated, Navigating, DocumentCompleted.
Thanks.

[shawty]

@ShaunLoganOracle Navigated and Navigating are already available:

Agreed tot he DocumentCompleted event, that's what kicked this discussion off in the first place, there are 2 key events in JavaScript:

"DOMContentLoaded" - This fires when the HTML Has finished loading, but while the browser is still loading CSS, Script and Image files

"load" - This fires when the entire document and all of it's resources are loaded.

I'm highly recommending the use case for providing handlers for both load events.

[liminzhu]

@shawty thanks for the details! I think the DOMContentLoaded, load and unload events are worth consideration. Something we can track on our backlog and see the demand bubble up. I share some reservation on input events. It's a bit of a rabbit hole. I will spend some more time looking into validation.

[shawty]

@liminzhu no worries, if you want any more input feel free to reach out to me :-)