Surface M3U8 tags during playback

[jlacivita]

It would be great to know whenever playback (or seeking) crosses an interesting M3U8 tag, e.g.

EXT-X-DISCONTINUITY

or even custom tags, e.g.

EXT-X-MYCUSTOMTAG: Hello World

This would enable storing (and responding to) meta-data inside the M3U, which is a common practice.

[adamscybot]

We have implemented these changes -- however they are not off-the-shelf ready and are probably not implemented as well as they could be. Would be great to see this in the plugin.

[jlacivita]

@mangui any thoughts on supporting this?

[mangui]

it should be easy to parse custom tags and put their content into org.mangui.hls.model.Fragment.
so storing should be easy.
then we should inject those metadata into the NetStream and retrieve them afterwards, at the right timing.

I am not clear yet on how this could be done, maybe injecting custom FLV tags with metadata/cue point.
then catching them on the fly with NetStream.onCuePoint() or NetStream.onMetaData(), and wrapping them with a new HLSEvent (ONMETADATA...)

we could also eventually dispatch level update using same technics

[mangui]

it is possible to inject FLV Tag (TagType=18 : SCRIPTDATAOBJECT) with custom content.
such tag should be injected with correct timestamp, as follow:

            tag = new Tag(Tag.CUSTOM_METADATA, frag_pts, frag_pts, false);
            var data : ByteArray = new ByteArray();
            data.objectEncoding = ObjectEncoding.AMF0;
            data.writeObject("onHLSCustoMetaData");
            data.writeObject(customtag);
            tag.push(data, 0, data.length);

then we should receive a callback on NetStream.client() with tag content when such tag is reached.

[mangui]

I will start by notifying discontinuity, to validate the model.

then we could move to other interesting stuff.
currently hls.level return the level of current loaded fragment, not actual playback level. it should be possible to also report actual playback level using similar technics.

[jlacivita]

Exciting! I will test it out when you start committing code

[mangui]

you should now be able to be notified when any new fragment starts playing. you should be able to check for discontinuity.

[mangui]

Hi, now you should be able to retrieve custom tags as well.
feedback welcome

[adamscybot]

I have compiled the latest version (chromeless player) but for me onFragmentPlaying is never called in the javascript. Same goes for onFragmentLoaded

[mangui]

strange. I just double check current flashls/master locally, I got js callback triggered

[13:7:45] load URL ../../../../playlists/test_001/stream.m3u8
[13:7:46] new track list
[13:7:46] manifest loaded, playlist duration:510.00
[13:7:48] switching state to PLAYING_BUFFERING
[13:7:48] start playback
[13:7:48] switching level to 0
[13:7:48] new track list
[13:7:48] TS/AAC 0 [257]
[13:7:48] switching audio track to 0
[13:7:48] switching level to 5
[13:7:49] onFragmentLoaded(): bandwidth:210791385 level:5 width:640
[13:7:49] onFragmentPlaying(): level:5 sn:0
[13:7:49] onFragmentLoaded(): bandwidth:67932349 level:5 width:640
[13:7:49] onVideoSize(), 640x360
[13:7:49] onVideoSize(),resize stage to 640x360
[13:7:49] switching state to PLAYING
[13:7:49] onFragmentLoaded(): bandwidth:31255746 level:5 width:640
[13:7:49] onFragmentLoaded(): bandwidth:40467488 level:5 width:640
[13:7:50] onFragmentLoaded(): bandwidth:36566000 level:5 width:640
[13:7:50] onFragmentLoaded(): bandwidth:20715810 level:5 width:640
[13:7:50] onFragmentLoaded(): bandwidth:64701299 level:5 width:640
[13:7:59] onFragmentPlaying(): level:5 sn:1
[13:7:59] onFragmentLoaded(): bandwidth:25899218 level:5 width:640
[13:8:9] onFragmentPlaying(): level:5 sn:2
[13:8:9] onFragmentLoaded(): bandwidth:22893855 level:5 width:640

[adamscybot]

Hi Mangui,

My mistake, I was loading an incorrect version of the player.

However, I noticed only level and sequence num are exposed. This does not include the custom_tags. Would it not be better to expose the whole playMetrics object?

In addition, it would be very useful to me if standard tags like PROGRAM-DATE-TIME also appear via this object (maybe renaming custom_tags to tags). I notice a special case for discontinuity is implemented in the form of a counter. I think it would be useful if tags like this were included alongside custom tags in a tags object/array.

[adamscybot]

Additionally there is bug whereby custom tags that precede an EXTINF tag rather than come after it, do not get populated in the custom_tags object. This behaviour is the opposite of the HLS specification:

EXTINF

It applies only to the media segment that follows it, and MUST be followed by a
media segment URI. Each media segment MUST be preceded by an EXTINF
tag.

[adamscybot]

I will send over a pull request of my fixes in a little while.

[adamscybot]

Here it is. There is one caveat with this approach in that on the first chunk, tag_list will include tags pertaining to the playlist like #EXT-X-VERSION:3 and #EXT-X-TARGETDURATION:5.

This isn't all bad, as their care cases you want access to these to. However, future work may include detecting these tags and adding them to a playlist loaded event rather than the first detected fragment.

You unfortunately can't use EXTINF as a delimiter as before due to the HLS spec requiring this tag to be just before the URL and no where else.

My only other suggestion would be to introduce a flashls custom tag that acts as the delimiter between tags pertaining to the playlist at the top, and tags pertaining to the first chunk.

[mangui]

Hi @adamscybot
indeed some playlist generic tag will be listed in the first fragment.
I think they should be filtered out. they are known standard tag, not related to a fragment.
could you filter these two tags and update the PR ?
thanks ;-)

[adamscybot]

Hi mangui,

I will get on this later today.

[adamscybot]

@mangui I have now updated the pull request with the fix. 8a0baee

[mangui]

looks good thanks

[mangui]

implemented

[jlacivita]

You guys rock! I'll check this out next week.

[ZeROZ7]

Hi mangui, first thanks for this implementation, looks very util.
I was working in the OSMF pluging, but here not seems work the event who fire the function "onHLSFragmentChange". I follow the secuence and verify if the event happens,
and it happens, but looks like the _hls.dipatchEvent does nothing in this case, even the logs it not happens inside this function.
The thing i want to do is get the filename of the current .ts playing. Maybe i m in the wrong place to catch this filename,in any case please i will be very grateful if you can help me with this.
thanks for you time and help.
good lucky.

[mangui]

you should be able to catch any event on OSMF side, using something like

_hls.addEventListener(HLSEvent.FRAGMENT_PLAYING, _fragmentPlayingHandler);

[ZeROZ7]

thanks for the answer. but still can't catch the event.
For exclude any error in my custom code i download the last version and compiled myself. i noted that the log from the HLSNetStream.as file (where is fire the event),not happen.I test put in others log in this file in different parts and the only one who do not work is this. Even i put logs in the part when you write the object onHLSFragmentChange.
I also catch other event in the osmf side, like FRAGMENT_LOADING, FRAGMENT_LOADED and it works, but with FRAGMENT_PLAYING not worked.
thanks again and i hope you can help me.
greetings.

[mangui]

Hi @ZeROZ7 I would advise you to retrieve flashls/master and first check that you can catch the event with chromeless player.
see
https://github.com/mangui/flashls/blob/master/src/org/mangui/chromeless/ChromelessPlayer.as#L418
https://github.com/mangui/flashls/blob/master/src/org/mangui/chromeless/ChromelessPlayer.as#L144
https://github.com/mangui/flashls/blob/master/examples/chromeless/index.html#L476

you should see traces updated on the web page each time FRAGMENT_LOADING event is received.
if this work, then there is no reason you cannot catch the same from OSMF ?

[ZeROZ7]

Yes now i am working in ChromelessPlayer and i see the FRAGMENT_PLAYING event , but still can´t capture the event in OSMF. I'm working with Grindplayer, maybe the problem is in this player, i will try with StrobeMediaPlayback to be sure.
I tell you later if works or not.
Thanks for the help.
Greetings

[ZeROZ7]

i have not lucky with StrobeMediaPlayback. Any idea what can be the problem?.
Thanks again.

[mangui]

Hi @ZeROZ7
indeed I cannot catch FRAGMENT_PLAYING event from OSMF either.
the callback HLSNetStream.onHLSFragmentChange() is not triggered.
i am suspecting it is related to the fact that OSMF framework is setting a client on HLSNetStream object, and thus it is catching the event ... I think I will have to create a client proxy so that flashls can still catch internal NetStream events.

[mangui]

Hi @ZeROZ7
issue should be fixed now. you should be able to catch FRAGMENT_PLAYING event from OSMF

[ZeROZ7]

Hi @mangui, works perfect!.
Thanks for all your help and for this great open source plugin.
see you later =D

[jlacivita]

Thanks!

[scaryguy]

Hi there,

I'm really excited to see that your plugin exists! It looks like a great piece!

@mangui , forgive my -probable- newbie question but how to listen to HLSEvent.FRAGMENT_PLAYING event on JavaScript side? Is that something like:

var video_object; //<FlowPlayer, Video JS etc...>
video_object.on('onFragmentPlaying', function(custom_tags) {
     // enjoying with custom_tags
});

What I want to do is; adding some custom tags to M3U8 file and those custom tags will have some values. I need to use those values from JavaScript in order to trigger some behavior.

Do you think that is this something doable using your plugin?

Thank you!