Investigate adding HLS support.

[ojw28]

No description provided.

[sdhr48]

Heyy I was trying to implement HLS(AAC) support on top of the ExoPlayer implementation and then I realized you are also investigating this. So here is my implementation so far. I have a HLSRendererBuilder and I am creating a HLSAacChunkSource that will have the audio and video representations for different bandwidths. In my case I am interested only in Audio. So this will have only one representation. And the major part I feel is the extractor. (AacChunkExtractor). So it would be great if you can let me know if I am going in the right direction.

[OnlyInAmerica]

Registering my extreme interest in adding HLS support. @sdhr48 could you put your work online? I'd love to help!

[ojw28]

That sounds about right, except the extractor should be called something like Mpeg2Extractor, since I think it would be extracting from the MPEG-2 transport stream?

[ojw28]

And yes, we are looking at this too. We'll provide an update when we have something to share!

[crossle]

Nice

[martinbonnin]

Hi !
I'm also very interested in HLS. We need that badly for our app so I started looking into it. Right now my plan is:

• focus on video-only first as HLS has audio/video interleaved inside chunks and it looks like it is not supported with the existing codebase.
• add HLSRendererBuilder, HLSChunkSource, TSMediaChunk, TSExtractor all in java and all for video-only.
• shoehorn audio support.
• test performance, possibly implement the TSExtractor in native code.
• add more features (live, multi-track audio, subtitles, etc...)

What do you think ?
My fork is at https://github.com/martinbonnin/ExoPlayer/commits/hls. It is 100% work in progress and it does not even compile but I don't want to duplicate effort so I figure out it is better to share progress. Please tell me if you believe what I'm doing is useful. If not, just tell me what could be more useful and I'll be happy to work on whatever can make this project move faster !

[4ntoine]

HLS support would be awesome

[akavrt]

Without HLS support ExoPlayer feels incomplete. Let's hope someday we will get it working!

[nachtien]

@martinbonnin, any luck on your implementation? I need this so bad for a project I am working on.

[martinbonnin]

@nachtien I'll post an update when I get stuff working.

@crossle I was a bit worried to add a big dependency such as ffmpeg requiring jni and stuff but I guess if it works it'll be more efficient than java.

[martinbonnin]

ok so now I can have some video rendered except that the stream pauses/rebuffers all the time. It looks like it's coming from chunk.startTimeUs and endTimeUs that I hardcoded to 0 for now. Problem is HLS is not like smoothstreaming and I cannot know the timestamps before parsing the TS file. @ojw28 : any idea how to circumvent this ?

[ojw28]

Aren't there timestamps in the manifest? If there weren't, then it would be impossible to implement seeking?

[martinbonnin]

HLS has no real 'manifest'. It has 'playlists' with the duration of each chunk in the EXTINF tag so I could use that to guess the timestamps but I am not 100% convinced because:

• the timestamps inside the TS might not start at offset 0.
• the chunk durations might be imprecise. I am not sure if this is a problem or not.

I'm going to try...

[ojw28]

Yeah. Playlists fundamentally must contain timestamps (or durations from which timestamps can be derived) for seeking to be possible. The timestamps must also not drift as you get further down the list, otherwise seeking would become inaccurate the further down the playlist you get.

If there needs to be some timestamp adjustment to the sample timestamps themselves, something similar to sampleOffsetUs in Mp4MediaChunk might do the trick, depending on what exactly is required. I'm hoping to get more up to speed with HLS in the next few weeks, and will be able to provide a slightly more informed opinion as opposed to random thoughts :).

[martinbonnin]

Ok so at first sight this seems to work but actually the incessant rebuffering problem was somewhere else. It was due to the fact that my TSMediaChunk.read() could return false (if reading an audio sample while a video one is expected for an example). In that case, the renderer stops feeding its input buffers and exoplayer believes it needs to rebuffer. To workaround that, I now discard audio completely but this is not a long-term solution...

[johnluther]

+1 for HLS support.

[nachtien]

@martinbonnin, awesome work so far!! Do you have an ETA for when think you'll have this done?
I tried using one of our audio only HLS files and it got stuck on preparing. It might be because it is an encrypted stream, or it might be because of something else.

[martinbonnin]

@nachtien do you have a test url ?

[nachtien]

@martinbonnin -- It actually gets through the preparing phase, but then jumps to idle. It looks like it is throwing an index out of bounds exception on the getTrackInfo method. I tracked the issue down to the MainPlaylist file lines 55-69 (parse method). The m3u8 file we have has EXTINF:5.0 or 4.0 above every url, so its not loading anything in the playlist.

Here is the URL we are using: https://api.beatgasm.com/v1/play/4301.m3u8?type=user&type_id=2190&bitrate=64000&X-Auth-Token=d4328aa4-46e2-452f-90f5-8228dd22a54c

Thank you!

[nachtien]

@martinbonnin, I saw the changes you committed today. So awesome. Please let know if you want help with this... If you really think this is possible to get the audio from the URL I attached above to start streaming in under 3 seconds, I could probably dedicate some time to help or allocate some other resources to help.

[martinbonnin]

@nachtien I must admit I do not focus on audio-only right now as I want our own streams to play first. I gave a quick look at your url and I see at least 3 things missing to playback your stream:

• AES decryption
• ID3 + AAC extractor (unless it is encapsulated in a TS file but I doubt it)
• a way to tell the HLSRendererBuilder to only use audio, which should not be the most complicated part but which I find a bit complicated to do cleanly...

As I am still digging and messing with the source code, I have no real ETA yet. Maybe somewhere next week... If you could think of a nice way to implement AES decryption, that'll definitely be useful !

[nachtien]

Thanks! I'll see what I can do.

[martinbonnin]

@nachtien I can now have audio-only working on my hls branch (https://github.com/martinbonnin/ExoPlayer/tree/hls). Making your stream play should not be very difficult from there. It's basically missing AES decryption. But your url seems offline. Can you put it back online or post a new one ?

Thanks,

[johnluther]

This is great progress, thanks everyone.

As an aside to the earlier seeking discussion here, the Android issue discussed here is causing us a lot of headaches: https://groups.google.com/forum/#!topic/android-developers/JZ9bd5FpsSQ.

We (JW Player) have customers who really need granular video seeking (accurate to the second) in HLS, but this segment boundary seeking limitation has left us stuck.

@ojw28, it feasible that ExoPlayer w/ HLS could work around it?

Thanks--
JL

[nachtien]

@martinbonnin, wow, thank you so much for getting this done. We were close to purchasing alternatives and this might really save us some serious time. The URL above still works (I just opened it up in quicktime). https://api.beatgasm.com/v1/play/4301.m3u8?type=user&type_id=2190&bitrate=64000&X-Auth-Token=d4328aa4-46e2-452f-90f5-8228dd22a54c.

[martinbonnin]

Ah, right. It is strange it does not work with 'curl' or 'wget' but opening it with Chrome is OK. Sorry for the false alarm.

[martinbonnin]

@nachtien I could decrypt the AES stream but then, the audio codec inside is MPEG AUDIO, which is unfortunately not supported at the moment (only AAC is). I'll take a look see if this is easy to add...

[nachtien]

Hey @martinbonnin, we want to send you a gift card or something if you get this working for us. Can you email me? Nachtien at gmail.

[martinbonnin]

@nachtien got it working :). It should work (tm) if you take the tip of my hls branch (https://github.com/martinbonnin/ExoPlayer/commits/hls) and you should be able to listen to the techno track under the 'Audio only - AES' item. Not sure if this qualifies for a gift though as seeking is not implemented yet and also it's only tested on my nexus5 so far. I'll email you some details.

[crossle]

@ojw28 I reproduce this many times, when i playback LIVE url.
BTW, why source.getBufferedPositionUs() < getCurrentPositionUs() when live streaming .

[ojw28]

You need to provide more information for us to be able to investigate. Which url specifically, so that we can try it ourselves? After how long does the problem occur, on which device? If possible, please also provide a full bugreport captured shortly after the problem is encountered (adb bugreport).

Thanks!

[crossle]

@ojw28 You can try http://itv08.digizuite.dk/tv2b/ngrp:ch1_all/chunklist-b1089536.m3u8?wowzasessionid=1128324428 live streaming, I used Nexus 5, Android 4.4.4, OOM problem not occur, OOM only occur on Samsung i9100. but it can't playback when player buffering a time, it buffering...all the time。

[crossle]

@ojw28 I found the buffering error, when buffering

ExoPlayer/library/src/main/java/com/google/android/exoplayer/ExoPlayerImplInternal.java

Line 336 in 79c2f53

|| rendererBufferedPositionUs >= positionUs + minBufferDurationUs

always return false, because MediaCodecTrackRenderer.java getBufferedPositionUs always return max value, so buffering all the time. see the print log, you can reproduce the problem use http://www.twitch.tv/ home HLS live stream.

ExoPlayerImplInternal.java.rendererReadyOrEnded  minBufferDurationUs=5000000,rendererBufferedPositionUs=63619655, rendererDurationUs=-1 , positionUs=63825000
MediaCodecTrackRenderer.java.getBufferedPositionUs  sourceBufferedPosition =18522622, getCurrentPositioin=63619655

[andudo]

We have implemented the main HLS stack with all the features we have originally targeted. I'm closing this issue. If you'd like to discuss a bug, missing feature, or any idea/suggestion, please open a separate specific issue.

[ojw28]

@AndroidNoob Do you have any sample streams for HLS containing MP3?