New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: twitch.tv / justin.tv archived video (VOD) support #70

Closed
ph03 opened this Issue Dec 23, 2012 · 26 comments

Comments

Projects
None yet
@ph03

ph03 commented Dec 23, 2012

Hi!

I am wondering if it could be possible to extend the justin.tv plugin to also support archived videos that have urls like

http://nl.twitch.tv/jeffroiscool/b/299079587

The important part is the 299079587 video id. This ID can be used with the twitch.tv api to obtain the video url by, e.g.,

http://api.justin.tv/api/clip/show/299079587.xml

(see http://www.fakegaming.eu/TwitchTV/source/download.txt / http://www.fakegaming.eu/TwitchTV/download.php for a php implementation). Perhaps there is also another way to do this using some established livestreamer features. I primarily want to use it to automatically download archived videos to watch them, e.g., on my mobile phone.

@chrippa

This comment has been minimized.

Show comment
Hide comment
@chrippa

chrippa Dec 23, 2012

Owner

I will add support for this soon, but there is a feature missing that needs to be added first. Twitch VODs are split up into 30 min segments, so a FLV joiner needs to be added to support this.

Owner

chrippa commented Dec 23, 2012

I will add support for this soon, but there is a feature missing that needs to be added first. Twitch VODs are split up into 30 min segments, so a FLV joiner needs to be added to support this.

@athoik

This comment has been minimized.

Show comment
Hide comment
@athoik

athoik Jan 4, 2013

Contributor

Maybe this project helps:

Python project to join flv files with same options(normally splitted by video server) together losslessly

https://github.com/hanenoshino/FLVCopyCat

Contributor

athoik commented Jan 4, 2013

Maybe this project helps:

Python project to join flv files with same options(normally splitted by video server) together losslessly

https://github.com/hanenoshino/FLVCopyCat

@john-peterson

This comment has been minimized.

Show comment
Hide comment
@john-peterson

john-peterson Jan 16, 2013

Contributor

Alternative solution by using youtube-dl

If you're reading this thread because you want to download all parts

  • this rg3/youtube-dl#621 is the youtube-dl code to download all parts (but not joining them)
Contributor

john-peterson commented Jan 16, 2013

Alternative solution by using youtube-dl

If you're reading this thread because you want to download all parts

  • this rg3/youtube-dl#621 is the youtube-dl code to download all parts (but not joining them)
@jabol3k

This comment has been minimized.

Show comment
Hide comment
@jabol3k

jabol3k Mar 15, 2013

Any update on this feature ?

jabol3k commented Mar 15, 2013

Any update on this feature ?

@chrippa

This comment has been minimized.

Show comment
Hide comment
@chrippa

chrippa Mar 15, 2013

Owner

Any update on this feature ?

Afraid not. Perhaps I was a bit overly optimistic in my earlier post, what I really meant was that I WANT to add it, but I don't know when, there are other things that needs to be done that are higher priority right now.

If anyone else wants to take a stab at it, here are some of my thoughts. The place to start is probably to take the timestamp adjust code from stream/hds.py and create a generic FLV concat class that you can feed a FLV tag and get a timestamp-adjusted tag back. You could then create a Stream class that reads tags from a list of Stream objects (for example a HTTPStream for each VOD segment) and processes them with previously mentioned class to get correct timestamps.

I don't know for sure if this approach will work though, the H264 timestamps may need to be adjusted aswell which would make this alot harder. Using ffmpeg might be the best option if that's the case. While that would probably limit us to the transports ffmpeg supports, that would be okay in this scenario since it's just HTTP links.

Owner

chrippa commented Mar 15, 2013

Any update on this feature ?

Afraid not. Perhaps I was a bit overly optimistic in my earlier post, what I really meant was that I WANT to add it, but I don't know when, there are other things that needs to be done that are higher priority right now.

If anyone else wants to take a stab at it, here are some of my thoughts. The place to start is probably to take the timestamp adjust code from stream/hds.py and create a generic FLV concat class that you can feed a FLV tag and get a timestamp-adjusted tag back. You could then create a Stream class that reads tags from a list of Stream objects (for example a HTTPStream for each VOD segment) and processes them with previously mentioned class to get correct timestamps.

I don't know for sure if this approach will work though, the H264 timestamps may need to be adjusted aswell which would make this alot harder. Using ffmpeg might be the best option if that's the case. While that would probably limit us to the transports ffmpeg supports, that would be okay in this scenario since it's just HTTP links.

@jabol3k

This comment has been minimized.

Show comment
Hide comment
@jabol3k

jabol3k Mar 15, 2013

Your approach is to feed vlc with fake, huge file and jumping to a timestamp would trigger livestreamer to feed appropriete timestamp from file list to vlc ? I'm I understanding it correctly ? Can we just simplify this for now and do it dirty an unelegant way. The first thing that comes to mind is playlist of all files. That way You are able to curcumvent the issue with timestamps. (Leave it for better times :P).

And as to a proper issue. Can You elaborate on timestams and h264 thing? I have little to no knoledge of it, but when You talked about ffmpeg, You proposed encoding it ? Am I understanding it correctly ?

jabol3k commented Mar 15, 2013

Your approach is to feed vlc with fake, huge file and jumping to a timestamp would trigger livestreamer to feed appropriete timestamp from file list to vlc ? I'm I understanding it correctly ? Can we just simplify this for now and do it dirty an unelegant way. The first thing that comes to mind is playlist of all files. That way You are able to curcumvent the issue with timestamps. (Leave it for better times :P).

And as to a proper issue. Can You elaborate on timestams and h264 thing? I have little to no knoledge of it, but when You talked about ffmpeg, You proposed encoding it ? Am I understanding it correctly ?

@chrippa

This comment has been minimized.

Show comment
Hide comment
@chrippa

chrippa Mar 15, 2013

Owner

Your approach is to feed vlc with fake, huge file and jumping to a timestamp would trigger livestreamer to feed appropriete timestamp from file list to vlc ? I'm I understanding it correctly ?

My approach is simply to basically rewrite the FLV container with correct timestamps. Most of this is already implemented in stream/hds.py, but HDS is designed to be segmented so I don't know for sure if the same approach will work on these FLV files that Twitch provides.

Can we just simplify this for now and do it dirty an unelegant way. The first thing that comes to mind is playlist of all files. That way You are able to curcumvent the issue with timestamps. (Leave it for better times :P).

Livestreamer is designed to output a raw stream of bytes to a output, such as a player or file, this is to be as compatible as possible. If we start feeding URLs to VLC or whatever, we suddenly face alot of corner cases where different players use different parameters and it would also make file output impossible.

If you want to write small script yourself that simply launches VLC with the segment URLs, the relevant API URL is http://api.justin.tv/api/broadcast/by_archive/VODID.xml. JSON is also available by changing the extension to .json.

And as to a proper issue. Can You elaborate on timestams and h264 thing? I have little to no knoledge of it, but when You talked about ffmpeg, You proposed encoding it ? Am I understanding it correctly ?

The reason we need to mess with timestamps is because if we don't, the resulting output will not be a valid FLV file. The player would play the first segment normally, but when it encounters a tag with timestamp 0 it would simply stop. So if we adjust the timestamp to continue from the last segment it should be fine, unless the H264/AAC timestamps also starts at 0, in which case we may have a problem.

I haven't looked at specifics for ffmpeg, but it might be possible to tell ffmpeg to avoid re-encoding by copying the existing data and simply rewrite the FLV container, much like the approach I was thinking of implementing in Livestreamer. But in a worst case scenario it would have to re-encode it.

Owner

chrippa commented Mar 15, 2013

Your approach is to feed vlc with fake, huge file and jumping to a timestamp would trigger livestreamer to feed appropriete timestamp from file list to vlc ? I'm I understanding it correctly ?

My approach is simply to basically rewrite the FLV container with correct timestamps. Most of this is already implemented in stream/hds.py, but HDS is designed to be segmented so I don't know for sure if the same approach will work on these FLV files that Twitch provides.

Can we just simplify this for now and do it dirty an unelegant way. The first thing that comes to mind is playlist of all files. That way You are able to curcumvent the issue with timestamps. (Leave it for better times :P).

Livestreamer is designed to output a raw stream of bytes to a output, such as a player or file, this is to be as compatible as possible. If we start feeding URLs to VLC or whatever, we suddenly face alot of corner cases where different players use different parameters and it would also make file output impossible.

If you want to write small script yourself that simply launches VLC with the segment URLs, the relevant API URL is http://api.justin.tv/api/broadcast/by_archive/VODID.xml. JSON is also available by changing the extension to .json.

And as to a proper issue. Can You elaborate on timestams and h264 thing? I have little to no knoledge of it, but when You talked about ffmpeg, You proposed encoding it ? Am I understanding it correctly ?

The reason we need to mess with timestamps is because if we don't, the resulting output will not be a valid FLV file. The player would play the first segment normally, but when it encounters a tag with timestamp 0 it would simply stop. So if we adjust the timestamp to continue from the last segment it should be fine, unless the H264/AAC timestamps also starts at 0, in which case we may have a problem.

I haven't looked at specifics for ffmpeg, but it might be possible to tell ffmpeg to avoid re-encoding by copying the existing data and simply rewrite the FLV container, much like the approach I was thinking of implementing in Livestreamer. But in a worst case scenario it would have to re-encode it.

@ghost ghost assigned chrippa Sep 2, 2013

@chrippa chrippa closed this in bb65e11 Sep 27, 2013

@ph03

This comment has been minimized.

Show comment
Hide comment
@ph03

ph03 Dec 15, 2013

I really like the VOD support for twitch.tv. However, sometimes VODs are multiple hours long. Is it somehow possible to start streaming at a specific time? Can this be encoded by the URL like explained in

http://twitchtv.wordpress.com/2011/10/24/new-functionality-link-to-a-specific-time-in-a-vod/

or maybe as a command line parameter? I would love this functionality!

ph03 commented Dec 15, 2013

I really like the VOD support for twitch.tv. However, sometimes VODs are multiple hours long. Is it somehow possible to start streaming at a specific time? Can this be encoded by the URL like explained in

http://twitchtv.wordpress.com/2011/10/24/new-functionality-link-to-a-specific-time-in-a-vod/

or maybe as a command line parameter? I would love this functionality!

@chrippa

This comment has been minimized.

Show comment
Hide comment
@chrippa
Owner

chrippa commented Dec 16, 2013

http://twitchtv.wordpress.com/2011/10/24/new-functionality-link-to-a-specific-time-in-a-vod/

The format described in this blog post is now implemented as of 2194206.

@ph03

This comment has been minimized.

Show comment
Hide comment
@ph03

ph03 Dec 17, 2013

Awesome, works like a charm! Thanks a lot!

ph03 commented Dec 17, 2013

Awesome, works like a charm! Thanks a lot!

@bundat

This comment has been minimized.

Show comment
Hide comment
@bundat

bundat Jan 10, 2014

All the vod links here don't work (except the first few ones):
http://www.reddit.com/r/speedrun/comments/1ui4dn/agdq_vods_thread/

Probably since the params are all in hours, like:
?t=8h23m
And the linked source in 2194206 only parses minutes and seconds.

bundat commented Jan 10, 2014

All the vod links here don't work (except the first few ones):
http://www.reddit.com/r/speedrun/comments/1ui4dn/agdq_vods_thread/

Probably since the params are all in hours, like:
?t=8h23m
And the linked source in 2194206 only parses minutes and seconds.

@ph03

This comment has been minimized.

Show comment
Hide comment
@ph03

ph03 Jan 10, 2014

I kept converting hours to minutes and add them together, but I didn't knew that the API also supports hours.

ph03 commented Jan 10, 2014

I kept converting hours to minutes and add them together, but I didn't knew that the API also supports hours.

@chrippa

This comment has been minimized.

Show comment
Hide comment
@chrippa

chrippa Jan 10, 2014

Owner

I added support for hours a couple of weeks ago (009360f) and will be available in the next release.

Owner

chrippa commented Jan 10, 2014

I added support for hours a couple of weeks ago (009360f) and will be available in the next release.

@bundat

This comment has been minimized.

Show comment
Hide comment
@bundat

bundat Jan 10, 2014

Excellent work! Love the fast response.
Thanks a lot!

Absolutely love livestreamer, infinitely better than the SLOW loading twitch player that randomly buffers for no reason.

bundat commented Jan 10, 2014

Excellent work! Love the fast response.
Thanks a lot!

Absolutely love livestreamer, infinitely better than the SLOW loading twitch player that randomly buffers for no reason.

@Joopiter

This comment has been minimized.

Show comment
Hide comment
@Joopiter

Joopiter Dec 24, 2014

Hi. It seems that "time feature" doesn't work everytime, some videos always start at the beginning. For example it doesn't work on "Past broadcast videos" here:
http://www.twitch.tv/pvplive/profile/past_broadcasts

Can it be fixed or are these just broken videos?

Joopiter commented Dec 24, 2014

Hi. It seems that "time feature" doesn't work everytime, some videos always start at the beginning. For example it doesn't work on "Past broadcast videos" here:
http://www.twitch.tv/pvplive/profile/past_broadcasts

Can it be fixed or are these just broken videos?

@aleksijuvani

This comment has been minimized.

Show comment
Hide comment
@aleksijuvani

aleksijuvani May 18, 2015

I'm also having trouble getting the time parameter to work with "past broadcasts". Here's the video that I tried this on. It always starts at the beginning.

aleksijuvani commented May 18, 2015

I'm also having trouble getting the time parameter to work with "past broadcasts". Here's the video that I tried this on. It always starts at the beginning.

@Joopiter

This comment has been minimized.

Show comment
Hide comment
@Joopiter

Joopiter May 18, 2015

Yeah, it is still not working for me at most videos, even with latest realese.

Joopiter commented May 18, 2015

Yeah, it is still not working for me at most videos, even with latest realese.

@skulblakka

This comment has been minimized.

Show comment
Hide comment
@skulblakka

skulblakka May 19, 2015

Contributor

Twitch uses two different systems for VODs. One is HLS based and the other one is Flash based. While you should be able to use the time parameter on the Flash based VODs it will not work on the HLS based one because Livestreamer does currently not support HLS seeking.

Contributor

skulblakka commented May 19, 2015

Twitch uses two different systems for VODs. One is HLS based and the other one is Flash based. While you should be able to use the time parameter on the Flash based VODs it will not work on the HLS based one because Livestreamer does currently not support HLS seeking.

@DerpDarako

This comment has been minimized.

Show comment
Hide comment
@DerpDarako

DerpDarako May 20, 2015

Ok thank you for this information, because this issue is very annoying... :(

DerpDarako commented May 20, 2015

Ok thank you for this information, because this issue is very annoying... :(

@aleksijuvani

This comment has been minimized.

Show comment
Hide comment
@aleksijuvani

aleksijuvani May 20, 2015

As a workaround, I've found that you can use the player-passthrough parameter with MPC-HC and then you're able to seek the video in the player. I couldn't get it to work with VLC though.

livestreamer --player-passthrough hls --player "C:\Program Files\MPC-HC\mpc-hc64.exe" www.twitch.tv/angryjoeshow/v/5158018 best

aleksijuvani commented May 20, 2015

As a workaround, I've found that you can use the player-passthrough parameter with MPC-HC and then you're able to seek the video in the player. I couldn't get it to work with VLC though.

livestreamer --player-passthrough hls --player "C:\Program Files\MPC-HC\mpc-hc64.exe" www.twitch.tv/angryjoeshow/v/5158018 best
@Joopiter

This comment has been minimized.

Show comment
Hide comment
@Joopiter

Joopiter May 20, 2015

@aleksijuvani It is not working for me. Video still starts always at the beginning, no matter if I use VLC or MPC-HC.

BTW you didn't add time stomp to the link in your code.

Joopiter commented May 20, 2015

@aleksijuvani It is not working for me. Video still starts always at the beginning, no matter if I use VLC or MPC-HC.

BTW you didn't add time stomp to the link in your code.

@aleksijuvani

This comment has been minimized.

Show comment
Hide comment
@aleksijuvani

aleksijuvani May 20, 2015

Oh, it'll still start at the beginning, but I was able to seek to the middle of the video with that bar at the bottom.

this thing

The timestamp is not needed when using this method. I'm using version 1.7.8 (6fcba1b) of MPC-HC. Maybe it doesn't work on older versions? Oh, and I think the video has to use the newer HLS format for this to work.

aleksijuvani commented May 20, 2015

Oh, it'll still start at the beginning, but I was able to seek to the middle of the video with that bar at the bottom.

this thing

The timestamp is not needed when using this method. I'm using version 1.7.8 (6fcba1b) of MPC-HC. Maybe it doesn't work on older versions? Oh, and I think the video has to use the newer HLS format for this to work.

@Joopiter

This comment has been minimized.

Show comment
Hide comment
@Joopiter

Joopiter May 20, 2015

Oh, right, I didn't notice. It's working, for my video at least, thanks :)

Joopiter commented May 20, 2015

Oh, right, I didn't notice. It's working, for my video at least, thanks :)

@Hubro

This comment has been minimized.

Show comment
Hide comment
@Hubro

Hubro Jun 18, 2015

Seeking in VODs using hls passthrough also works in VLC 2.2.0 on Linux, but there are no time stamps. The current and end time just say "00:00:00".

Hubro commented Jun 18, 2015

Seeking in VODs using hls passthrough also works in VLC 2.2.0 on Linux, but there are no time stamps. The current and end time just say "00:00:00".

@MaddXav

This comment has been minimized.

Show comment
Hide comment
@MaddXav

MaddXav Aug 5, 2015

I'm getting the same issue. No timestamps on VLC, and adding the timestamp to the video url still makes it start at beginning.

MaddXav commented Aug 5, 2015

I'm getting the same issue. No timestamps on VLC, and adding the timestamp to the video url still makes it start at beginning.

@dmitriy-korotayev

This comment has been minimized.

Show comment
Hide comment
@dmitriy-korotayev

dmitriy-korotayev Apr 28, 2016

Are there any updates on VLC timestamp issue? Player-passthrough parameter works for enabling seeking, but still no exact time

dmitriy-korotayev commented Apr 28, 2016

Are there any updates on VLC timestamp issue? Player-passthrough parameter works for enabling seeking, but still no exact time

javiercantero pushed a commit to javiercantero/livestreamer that referenced this issue Jan 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment