Support for HLS Streams

[derrod]

Hello,

I'm currently using your awesome plugin on my Raspberry Pi to watch streams on my TV without having to use my PC, but sometimes there are issues with lag.

Twitch will be switching to HLS in the future, but it's already available so I would like to have a switch in the settings to switch to HLS when RTMP streams are lagging.

The twitch beta page uses a new API to get stream content, it responds either with a m3u8 playlist containing different quality (source, high, medium, low, mobile) playlists for HLS or the same response you'd get with the old API that your plugin currently uses. This is probably due to balancing but on most streams with higher viewercounts (~1000+) I would constantly get HLS playlists.

There's also a distinct API that always will give you HLS content because it is for the mobile app, this will also work for very small channels.

XBMC supports HLS since Frodo so I think it's reasonable to support HLS for twitch in the future.

~Rodney

[ccaspers]

Can you provide us with some links to the relevant parts in their new API and some info on when the change is going to happen? What kind of video are they serving right now via HLS?

Greetings, Chris

[ElectronicWar]

HSL is already live and used for mobile streams and for the consoles.
The webplayer for PCs seems to support it already as well and seems to stream using HLS already for bigger streams (like the last Dreamhack which was 100% HLS), partnered Streams are already served with HLS as well on a random base.

Details are here: http://blog.twitch.tv/2013/12/service-update-major-video-system-update-and-prague-is-online/

RiotGames LCS Streams seem to use HLS as well, at least I am receiving data via port 80 and its "buffering" from time to time which seems to be a feature from HLS.

[derrod]

HSL is already live and used for mobile streams and for the consoles.
The webplayer for PCs seems to support it already as well and seems to stream using HLS already for bigger streams (like the last Dreamhack which was 100% HLS), partnered Streams are already served with HLS as well on a random base.

Every stream I've watched in the last few days was delivered via HLS, no matter if they had 1 or 100k viewers (I'm in the EU, may differ for NA).

Can you provide us with some links to the relevant parts in their new API and some info on when the change is going to happen? What kind of video are they serving right now via HLS?

Greetings, Chris

Sure! They already switched most if not all streams to be served via HLS, at least for EU servers.

Their new API URL is http://usher.twitch.tv/select/$CHANNEL.json?nauthsig=&nauth=&allow_source=true

It will either send you a HLS playlist (m3u8) containing links to playlists for the different qualities or in rare cases the data for rtmp you currently get. Note that currently it does not require a token and signature, it may do in the future, you can get both from here: http://api.twitch.tv/api/channels/$CHANNEL/access_token (the token has to be urlencoded)

Non-partnered channels or channels that just don't have enough viewers will only have one URL in the response, channels that are partnered/have enough viewers and are transcoded will contain URL's for "Source", "High", "Medium", "Low" and "Mobile" quality levels.

Example for responses:

Transcoded channel:

#EXTM3U
#EXT-X-TWITCH-INFO:NODE="video8.ams01",CLUSTER="ams01",MANIFEST-CLUSTER="ams01",USER-IP="1.2.3.4",MANIFEST-NODE="video8.ams01"
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="source",NAME="Source",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=0,BANDWIDTH=2657671,RESOLUTION=1920x1080,VIDEO="source"
http://video8.ams01.hls.justin.tv/hls71/taketv_7865992192_43994311/source/index.m3u8?token=id=4856697168322189206,bid=7865992192,exp=1387227105,node=video8-1.ams01.hls.justin.tv,nname=video8.ams01,fmt=source&sig=9e462f520ea5b03df23bd7acba4f8805382812e8&
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="high",NAME="High",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1760000,VIDEO="high"
http://video8.ams01.hls.justin.tv/hls71/taketv_7865992192_43994311/high/index.m3u8?token=id=4856697168322189206,bid=7865992192,exp=1387227105,node=video8-1.ams01.hls.justin.tv,nname=video8.ams01,fmt=high&sig=cd27d825525ca9a42605ae3de0442a0e6841226f&
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="medium",NAME="Medium",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=928000,VIDEO="medium"
http://video8.ams01.hls.justin.tv/hls71/taketv_7865992192_43994311/medium/index.m3u8?token=id=4856697168322189206,bid=7865992192,exp=1387227105,node=video8-1.ams01.hls.justin.tv,nname=video8.ams01,fmt=medium&sig=90eccbe7fe64cd5e6f7473b74679d01aef5ed0c5&
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="low",NAME="Low",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=596000,VIDEO="low"
http://video8.ams01.hls.justin.tv/hls71/taketv_7865992192_43994311/low/index.m3u8?token=id=4856697168322189206,bid=7865992192,exp=1387227105,node=video8-1.ams01.hls.justin.tv,nname=video8.ams01,fmt=low&sig=f2893c1ec072c9d383e36db414fe23b2b07e1617&
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="mobile",NAME="Mobile",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=164000,VIDEO="mobile"
http://video8.ams01.hls.justin.tv/hls71/taketv_7865992192_43994311/mobile/index.m3u8?token=id=4856697168322189206,bid=7865992192,exp=1387227105,node=video8-1.ams01.hls.justin.tv,nname=video8.ams01,fmt=mobile&sig=4c536cf350a2015db6f6050a053e3f0d23eeea5b&

Example for non-transcoded channel:

#EXTM3U
#EXT-X-TWITCH-INFO:NODE="video25.sfo01",CLUSTER="sfo01",MANIFEST-CLUSTER="sfo01",USER-IP="1.2.3.4",MANIFEST-NODE="video25.sfo01"
#EXT-X-MEDIA:TYPE=VIDEO,GROUP-ID="chunked",NAME="Source",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1462593,RESOLUTION=1366x768,VIDEO="chunked"
http://video25.sfo01.hls.justin.tv/hls28/krisgaming_7870455808_44132410/chunked/index-live.m3u8?token=id=7483289906744142728,bid=7870455808,exp=1387227197,node=video25-1.sfo01.hls.justin.tv,nname=video25.sfo01,fmt=chunked&sig=fed7e14a273f93051fcf1b2445fceaa45921a3cd&

I don't know how XBMC works internally, but you probably just need to pass the url of the desired quality to it's video player.

[ElectronicWar]

Thanks for the round up, derrod!

Non-partnered channels or channels that just don't have enough viewers will only have one URL in the response, channels that are partnered/have enough viewers and are transcoded will contain URL's for "Source", "High", "Medium", "Low" and "Mobile" quality levels.

Transcoding will be available for non-partnered Streams as well in future, depending on the current load of the servers. Not sure if this will make any difference for the implementation tho.

[derrod]

Transcoding will be available for non-partnered Streams as well in future, depending on the current load of the servers. Not sure if this will make any difference for the implementation tho.

It shouldn't, the playlists should be the same as for partnered/transcoded channels now.
If the implementation works it shouldn't matter.

[ccaspers]

Thanks for your detailed investigation :) I will try to implement HLS-support during the holidays, as my days are pretty packed with other things until then.

[Dthibault]

I sent a request to " http://usher.twitch.tv/select/$CHANNEL.json?nauthsig=&nauth=&allow_source=true" URL. I've got many streams (Source, High, Medium, Low, Mobile) and i past the Source (Or High) URL to a .strm file.

I played this .strm file in XBMC and it works!

[GrimD]

Thanks Dthibault just tried that and it worked for me too. Not the most efficient way to watch twitch but will do for now.

[StateOfTheArt89]

After a long pause, I'm planing to get back working on the twitch plugin. Since I recently finished my bachelor's degree, there is a little more time to work on the plugin.
HLS support will be the first thing to be implemented.

[derrod]

That's good to hear as the twitch app is currently unusable because of the switch to HLS.

RTMP is completely disabled as it seems.

[Dthibault]

With HLS, the processor work less! It's a good news ;)

[lwis]

@GrimD @Dthibault which part are you folks copying into the strm file? Just tried it and it's not working for me 👎

[GrimD]

When you go to http://usher.twitch.tv/select/$CHANNEL.json?nauthsig=&nauth=&allow_source=true it should down load a file. I just changed the extension to .strm, copied it up to my raspberry pi and then opened the file in XBMC.

[Dthibault]

I write a tutorial:

For example, i want to view this stream on XBMC (Raspberry Pi, etc..) : http://fr.twitch.tv/yellowstarr

1. Go to http://usher.twitch.tv/select/yellowstarr.json?nauthsig=&nauth=&allow_source=true
   (Replace yellowstarr by the name of the streamer)
2. Open the .json file with a text editor (Like Notepad++ or other)
3. Choose the quality what you want
   (For example, quality HIGH: http://video12.prg01.hls.justin.tv/hls46/yellowstarr_7904356016_45158007/high/index.m3u8?token=id=6790145345996856612,bid=7904356016,exp=1387489645,node=video12-1.prg01.hls.justin.tv,nname=video12.prg01,fmt=high&sig=8576a59ef38908614c7fc22c0faa101964c85303&
   )
4. Create a new file and name it "xxxx.strm" (xxxx can be replaced ;) )
5. Copy the URL to this file and save it
6. If you have a Raspberry Pi, upload the file on SD Card.
7. Go to System->Files (Or Video->Files, i think it's possible)
8. Choose the .strm file
9. Enjoy ;)

EDIT: The GrimD method is functionnal, but by default, XBMC load the Source quality :/

[lwis]

Interesting, even without changing the channel name and using your method I'm unable to start a stream. My log;

21:56:10 T:2859058256  NOTICE: Thread JobWorker start, auto delete: true
21:56:10 T:2825503824  NOTICE: Thread BackgroundLoader start, auto delete: false
21:56:12 T:3044483664  NOTICE: Previous line repeats 1 times.
21:56:12 T:3044483664  NOTICE: COMXPlayer: Opening: http://video2.ams01.hls.justin.tv/hls46/yellowstarr_7904356016_45158007/source/index.m3u8?token=id=7591526601632606701,bid=7904356016,exp=1387490123,node=video2-1.ams01.hls.justin.tv,nname=video2.ams01,fmt=source&sig=074ccba083224f2ccdd31cb619e75abaf7feba1e&
21:56:12 T:3044483664 WARNING: CDVDMessageQueue(player)::Put MSGQ_NOT_INITIALIZED
21:56:12 T:2825503824  NOTICE: Thread OMXPlayer start, auto delete: false
21:56:12 T:2825503824  NOTICE: Creating InputStream
21:56:12 T:2876609616  NOTICE: Thread FileCache start, auto delete: false
21:56:12 T:2825503824  NOTICE: Creating Demuxer
21:56:12 T:2825503824   ERROR: OpenDemuxStream - Error creating demuxer
21:56:12 T:2825503824  NOTICE: COMXPlayer::OnExit()
21:56:12 T:2825503824  NOTICE: COMXPlayer::OnExit() deleting input stream
21:56:12 T:3044483664   ERROR: Playlist Player: skipping unplayable item: 0, path [http://video2.ams01.hls.justin.tv/hls46/yellowstarr_7904356016_45158007/source/index.m3u8?token=id=7591526601632606701,bid=7904356016,exp=1387490123,node=video2-1.ams01.hls.justin.tv,nname=video2.ams01,fmt=source&sig=074ccba083224f2ccdd31cb619e75abaf7feba1e&]
21:56:12 T:2876609616  NOTICE: Thread BackgroundLoader start, auto delete: false
21:56:12 T:3044483664  NOTICE: DVDPlayer: finished waiting

[GrimD]

Mine didn't load the source by default. When opening the file it then took me to a menu with several options to chose from. They are all the same name as the file doesn't contain what ever XBMC is expecting to name them correctly but they are the different stream qualities listed in the file with source at the top then high etc going down to mobile.

Personaly I found this use full as my contention ratio on my ADSL is pants so I keep going back from the stream to the menu to change the quality up and down depending on how well its streaming

[Dthibault]

GrimD: Hum, yes, your method is better than mine finally 👍
lwis: In my link, there is a token that must change for each request :/

[lwis]

@Dthibault I regenerated the link for my connection :)

[Dthibault]

My log for this stream:

22:36:18 T:2690278464  NOTICE: Thread Jobworker start, auto delete: true
22:36:21 T:3041014400  NOTICE: COMXPlayer: Opening: http://video6.ams01.hls.justin.tv/hls46/yellowstarr_7904356016_451580$
22:36:21 T:3041014400 WARNING: CDVDMessageQueue(player)::Put MSGQ_NOT_INITIALIZED
22:36:21 T:2698667072  NOTICE: Thread COMXPlayer start, auto delete: false
22:36:21 T:2698667072  NOTICE: Creating InputStream
22:36:21 T:2630329408  NOTICE: Thread CFileCache start, auto delete: false
22:36:21 T:2698667072  NOTICE: Creating Demuxer
22:36:27 T:2698667072  NOTICE: Opening video stream: 0 source: 256
22:36:27 T:2698667072  NOTICE: Creating video thread
22:36:27 T:2630329408  NOTICE: Thread COMXPlayerVideo start, auto delete: false
22:36:27 T:2698667072  NOTICE: Opening audio stream: 1 source: 256
22:36:27 T:2698667072  NOTICE: Creating audio thread
22:36:27 T:2744489024  NOTICE: Thread COMXPlayerAudio start, auto delete: false
22:36:27 T:2630329408   ERROR: Output - renderer not started
22:36:27 T:2698667072  NOTICE: OMXClock using audio as reference
22:36:27 T:2630329408   ERROR: Output - renderer not started
22:36:27 T:2630329408   ERROR: Previous line repeats 6 times.
22:36:27 T:2630329408  NOTICE: Display resolution DESKTOP : 1920x1080 @ 60.00 - Full Screen (16)
22:36:27 T:3041014400  NOTICE: Using GL_TEXTURE_2D

I use Raspbmc (Latest Version) on my Raspberry Pi.

[lwis]

Interesting, I'm on the latest nightly of OpenELEC :( (The stream URL is working because I'm able to play it through VLC)

[derrod]

Personally I just put

http://usher.twitch.tv/select/destiny.m3u8?nauthsig=&nauth=&allow_source=true

in the .strm file, so you don't need to redo it when the stream is restarted and your URL is not vaild anymore.
You can remove "&allow_source=true" if you don't want source quality.

I use OpenELEC on my Raspberry Pi btw.

Edit: ooops hit the wrong button, accidently closed the issue...

Edit 2: In most cases my raspberry pi seems to select "source", but not always.

[lwis]

@derrod brilliant, that works! Thanks buddy.

[MrSprigster]

Hey Guys, I took the liberty of making a temporary fix for the addon until it is properly updated.

I hope that's okay CCaspers (feel free to incorporate any part of my fix into your next version or even release it on the git hub).

To implement my fix:

1. Download default.py from http://www53.zippyshare.com/v/64499362/file.html
2. Replace the default.py in (appdata) XBMC\addons\plugin.video.twitch\ folder with the one you just downloaded
3. Fixed!

Merry Christmas Guys!

Dave

[Dthibault]

Thanks for your temporary fix! 👍

[kokarn]

If the temporary fix is something interesting, there's now a pull for it. #47

[MrSprigster]

I've created a much improved fix with Quality options working and Auth added in as a future proofing solution. There's now a pull for this. #49

[StateOfTheArt89]

MrSprigsters fix is now merged into this repository. As the feedback so far states, it seems to be working for most people. A code clean up will follow