(Proposed Feature) Can we have the ability to intercept object tags?

[shawty]

I'm currently working on a custom browser instance that will be used in a specific setting for developing applications that will run on Smart TV set's.

One of the interfaces in the API that I will be implementing in the custom browser, requires the addition of an <object...> tag with a special mime type to be injected into the page dom.

Most "TV Applications" developed to use this API, perform that operation as follows:

the result of this will result in the following tag:

<object type="video/broadcast" id="MediaPlayer"></object>

being added to the page. This tag when the "play()" method is called on it, will start presenting to the user, the current image present on the broadcast video tuner in the device. This is part of the OIPF/HbbTV smart TV application environment.

At present I'm creating a desktop browser, that allows the developer using it, to load into and debug an HTML Application written to follow this standard, rather than the very difficult closed environment that exists on most TV sets.

In order for me to perform this task, I need to implement the API, that the external applications expect to be present on the device, a task which is now well underway, as WebView2 has to date been the only web control that we've come accros that provides suitable features to allow the API to work the way the specs state it should. The only thing we don't have is the "object" tag handling.

When the object is added, the chromium kernel, displays it as follows:

This is not unexpected, as the browser instance behind the scenes does not have an API object registered within it's API layer, that understands that media type.

In order for me to implement this "object" tag correctly, I need to do 2 things.

1. Be told by the browser when one of these tags is seen and be given a reference to the object, that allows me to work with it.

2. Be told when the object needs to be painted (In the case of this example, it will be painted with a Television broadcast from a DVB-T Tuner) so that I can paint the correct data into the control.

I do have a few JavaScript shims I'm playing about with to try and make something work, but having an interface and event similar to the IObject interface in AngleBrackets would be much more helpful, then we could just add our own properties and methods to the object when created, and paint the controls rectangle as required.

Might I suggest some kind of event that is surfaced to the webview control that restricts access ONLY to that object and nothing else in the DOM, and maybe some C# level functionality that allows us to for EG: blah.addProperty("foo", "fooValue") or blah.addFunction("blahFunction", "//blah function JS code...")

The return should be a bool, thus allowing us in the application to say, yes we implemented this (return true) and if we don't return false, so the browser can show the "isn't supported" UI story.

If the implementation in application hosting the control supports an object of a given type, then it should

1. Respond to the "object event" with a true
2. implement any backend host objects needed
3. add any custom properties, methods needed to the new object and wire up any calls needed to any host objects created
4. Register any paint or other events so the control rectangle can be painted as needed.

To highlight this with an example of what I'm trying to do, If i was to create a "video/broadcast" object, then I would need to implement methods such as "bindToCurrentChannel":

as documented in Volume 5 section 7.13 of the OIPF Specification Version 2.

Some objects are already natively handled, for example to play an MP4 the same process is used, but a media type of "video/mpeg", "video/mpg" or "video/mp4" is used. All the browsers, including WebView2 already handle this, although I have yet to verify that the API calls and methods are the same, but I may be able to patch those using a weak-map on the JS side if not.

If you want more information, or would like a copy of the spec document, I can provide them for you.

Regards
Shawty

AB#26994612

[liminzhu]

Thanks for opening the issue and sorry for the delay! This looks very interesting.

Can you change the web content or they have to use the object tag? The reason I'm asking is that custom element can be pretty useful as you define whatever behavior you want on a custom tag.

[shawty]

Hi @liminzhu

Custom object tags where at one time the way that all browsers hooked things like "Flash Player" and the original Abobe PDF reader.

Classically ALL Browser plugins based on the original and now insecure NPF format, used the object tag and it's mime type to trigger a plugin.

Plugin's would register for a mime type, then when an '' was inserted into the DOM Node, the browser would look in it's registry to see if the mime type was registered.

If a mime type was registered, then the plugin would be loaded, and the browser would pass a record to it with the rectangle occupied by the object tag. If the styles on the object tag caused it to be resized, the browser would send a message to the plugin with the new size.

From that point on, the plugin had 2 choices, it could either accept all responsibility for painting the given rectangle (Sort of like an early canvas control) and would hook into the browsers paint mechanism and paint it's rectangle when ever asked too.

The second choice was that it was handed a DOM reference to the inner HTML in it's tag, and it could do whatever it wanted within the tag. Styles, JavaScript anything... much like the modern iFrame anything done inside the tag was NOT visible outside the object tag, and anything inside was not visible to code outside it. In this second case, the plugin was NOT allowed to change the root level object tag, but there was the ability to use things like postMessage to send messages into the tag, and the tag could postMessage back out.

What I'm looking for specifically is the former case of painting the tag.

If you look at my custom browser (https://github.com/shawty/hbbtvbrowserEXPERIMENTAL) you'll see how I've started to implement the '" tag.

In the OIPF/HbbTV smart TV standard, when this object is placed in a document, the controlling page can then call "bindToCurrentChannel()", which would then cause the televisions subsystem, to start painting the current tuner image in the rectangle occupied by the tag.

As far as the OIPF is concerned, it also needs to be able to add extra functions onto the root object too, in my case I'm attaching them to the prototype by using Object.Attach(a,b) in JS code.

Interestingly enough... IF I use the other video tag specified in the OIPF Standards '' the edge control actually understands and obeys it and allows you to ".play()", ".stop()" the current video. That is it treats it like a tag but with the original video playback API on it. This actually took me a bit by surprise when I was developing my browser, had me scratching my head for a while wondering why something I'd not yet written was working :-)

If you look in the Docs folder inside my experimental browser project, you'll actually find various PDF's among which are the OIPF/OITF Version 2 smart TV standards, volume 5. Section 7 in this document (7.13 if my memory is right) is the exact section you want that describes all of what I'm trying to do.

If you want any more help here, you can find me on Twitter as shawty_ds I'm a UK Microsoft Community Leader, so it's not difficult to find me :-D

Hope that helps.
Shawty