YT signature decrypt, direct stream URL and FoxTube

[SanderDK]

With all the ongoing YT cipher voodoo, 'security' updates and layout changes, it recently got to my attention that the iOS app FoxTube does not seem to be affected by those changes. It obviously uses direct video stream URL's since you can 'cache' (e.i. download) videos, and the app is updated way to infrequently to keep up with the latest signature changes and lengths we constantly have to deal with. This has either got to mean there is or are better ways to get the stream URL for a video which always has the right signature, or there is a way to dynamically decrypt the signatures. I think the latter is the LESS likely solution, since the algorithms used to 'encrypt' the signatures could and have changed at random times. Perhaps there is even a way to get the stream URL from the YT video player directly that too already has the right signature.

I know this is not really an issue, but I believe it is something we have to investigate and should figure out alternative, better ways to get the direct stream URL.

[ghost]

Can it play VEVO videos?

Other youtube videos work fine, in my experience it's always just VEVO. The issue is that we are bypassing their ads which cost them (YouTube & VEVO) revenue. YouTube will continue making daily/weekly changes to prevent this type of access until the end of time.

[SanderDK]

You're absolutely right, the issue lies with videos that have content owned by VEVO, WMG (Videos from Bruno Mars for example) etc. Even those videos have been and are downloadable with the FoxTube app for a long time now without the app seeing an update to add the new signatures YouTube added or changed recently. I guess they could use a remote script to extract the video source URL, though I wonder if that is how they actually do it. I know for a fact the FoxTube app CAN'T play EXPLICIT VEVO videos which require age verification (07FYdnEawAQ for example), so perhaps we can narrow the method with which they extract the source URL down a bit knowing this. When one tries to play an explicit VEVO video it throws an error message, saying the same thing you get when you try to embed a VEVO video on a mobile device: 'This video contains content from VEVO. It is restricted from playback on certain sites'. Note that regular videos which are explicit and require age verification (Q7GVSx7yMaA for example) DO work.

[strex]

I made some tests with a logging proxy server.
It seems the app is just requesting the http://www.youtube.com/watch?v=%videoID% page such as we do.

But I also noticed POST requests to http://data.flurry.com/aas.do
Maybe they are updating their algo automatically.

[SanderDK]

I guess that is rather exciting news, although Flurry is used to track user
behavior for iOS apps, kind of like Google Analytics does for websites, so
that can't be used to update their algorithm. Did you notice any other GET
(or POST) requests? Smart thinking btw!

[Paco8]

I use in my smartphone (android) the app tubemate. At first it was affected by the signature changes (it failed to download VEVO videos) but a couple of weeks ago it was released a new version and since then it can download all VEVO videos, although only two resolutions are available for those videos: 630x360 (mp4) and 320x240 (mp4).
So I guess there must be a way to download the VEVO videos even though in low resolution.

[SanderDK]

I setup my own logging proxy server and logged the requests that were coming from FoxTube, and can confirm the requested URL's are the following:

• When the user requests a video feed or searches for a video using FoxTube, a GET request is send to http://gdata.youtube.com/feeds/api/videos?
• When the user taps on a video received from this request another GET request is send before the video can start playing, this time to http://www.youtube.com/watch?
• When this request has finished, the next GET request is send to http://r14---sn-5hn7snee.c.youtube.com/videoplayback? to stream and buffer the video selected by the user

HOWEVER, I have some other very interesting insights. Since this is a publicly open repo, I would prefer to talk to about the things I found out via a private GitHub repo or an IM service of some sort. Let me know what you guys think, or if someone could create a temporary private GitHub repo to further discuss this.

@phihag @jaimeMF

[strex]

This app mentioned by @Paco8 "TubeMate" is also very interesting.
It seems they are using the official gdata API to get the direct links for the vevo videos, so they don´t need to decrypt signatures because the links returned by the gdata API are working directly.

Normally the gdata API isn't returning the direct links, but the App is sending a special header "X-GData-Device" like the offical YouTube Apps for iOS and Android do.
But I have no Idea how this Header is generated.

[SanderDK]

Very interesting indeed @Paco8 & @strex . I don't have an android device to run and test the app TubeMate myself, though the header could too be a great find. I have looked for a special header like "X-GData-Device", but I did not find it being send from the official YouTube app for iOS. Along with which GET request did you find out it was being send @strex ?

[yasoob]

Hey guys I think is not the best place to discuss these headers. This repo is public and this discussion can be seen even by youtube's developers.

On Jul 25, 2013, at 6:39 PM, SanderDK notifications@github.com wrote:

Very interesting indeed @Paco8 & @strex . I don't have an android device to run and test the app TubeMate myself, though the header could too be a great find. I have looked for a special header like "X-GData-Device", but I did not find it being send from the official YouTube app for iOS. Along with which GET request did you find out it was being send @strex ?

—
Reply to this email directly or view it on GitHub.

[SanderDK]

I agree @yasoob . If someone could create a temporary private GitHub repo that would suffice, but perhaps there is a better alternative?

[aran-arunakiri]

I am also working on an android app which depends on youtube streams, and facing the same problems as you guys. Can you invite me to your private repo ?

[JonasGessner]

Does this private repo exist now? If yes I'd like to join, I am also working on this same solution using the "X-GData-Device" header.

[SanderDK]

As far as I am aware of there is no private repo to discuss this in existence, again if anyone can do this let us know here so we can join the discussion!

[JonasGessner]

I would create the repo but I already reached my limit, sorry! Somebody else needs to create this repo, I'd really appreciate it! ;)

[SanderDK]

I will look into it and invite everyone here as soon as I have created the repo.

Op 4 sep. 2013 om 23:35 heeft Jonas Gessner notifications@github.com het volgende geschreven:

I would create the repo but I already reached my limit, sorry! Somebody needs to create this repo! ;)

—
Reply to this email directly or view it on GitHub.

[leg10nMX]

Hi there,I'm the developer of an iOS App: aTube (Cydia), I'm looking into this issue as well, we can share some insights of what I've researched, please invite me into the conversation.

[Elite]

It seems "X-GData-Device" will need an auth-key supplied to premium partners of Google. Check - http://youtube-direct.googlecode.com/svn-history/r418/branches/2.0/src/com/google/ytd/youtube/VideoDownloadRedirect.java

[cujo30227]

Hi there. I've developed a video ios app as well and would be interested to join the private repo. Could you invite me as well?

[Elite]

@SanderDK waiting eagerly to get into the private repo and see what can be done.

[DeveloperXX]

Hi guys I'm working on a solution right now for this same issue . If a private repo is/has started, please invite as well.

[phihag]

I'm closing this issue now, because the discussion has become off-topic and signatures now are decrypted automatically.

[phihag]

@ksy5662 and everyone else commenting here: As far as I know, there is no private repo. In any case, this is a bugtracker for youtube-dl and unrelated to any private repository. Please refrain from posting further comments asking for access to the mystical private repo. I'll delete all of them.

[phihag]

Locked to prevent further off-topic discussion.