New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ITV failure to download after site change #211
Comments
Hello @njtaylor (I'm an "old" friend from the Get iPlayer mailing list)! I can confirm the latest itv.pm v0.08 is broken since ca. July 17th 2017; I am on Windows Vista SP2 32bit, using portable Strawberry Perl 5.24.01 (x86) and the combined GFV script, v1.25.92
HOWEVER, I have some important additional info to contribute here:
However, it is only when one downloads to disk and then opens in a text editor that one realises both video+audio streams are encrypted; I am attaching a previously downloaded MPD for you to inspect
So, the RTMP (Flash) unencrypted streams ARE STILL THERE, somehow the itv.pm code must be patched to re-gain access to them; FWIW, python code for itvplayer.py (from streamlink) is here; sadly, streamlink doesn't fetch subtitles, it is targeted more to live streams (being a fork of livestreamer); VOD support is just an added bonus, I guess...
That was then quickly (before the token expired) dumped to disk via either FFmpeg or streamlink/livestreamer:
The drawback of the mobile streams is the highest available quality is 504p@1500kbps, while the highest Flash quality variant is 504p@1800kbps; but it should be adequate if all other routes become inaccessible... I trust you to make good use of my invaluable (hopefully) info... Hoping for a GFV fix soon! |
That's very helpful. I'm using OpenBSD, and chrome which has a user agent string of I will look at the itvplayer.py and rtmp streams first. |
Issue has been identified as a change to the productionId=x/xxxx/xxxx#xxx to productionId=url/x_xxxx_xxxx.xxx so the extracted id is wrong. Itv.pm has been updated. |
Hi again, Nigel 😄 It is indeed weird that on OpenBSD+Google Chrome v59.0.3071.115 you are being served unencrypted MPEG-DASH streams, while on Windows (Vista+) and recent Firefox they are serving encrypted ones. When the MPD is inspected, it definitely references ENCRYPTED streams:
Furthermore, I tried downloading that MPD (while its token was still valid) with youtube-dl, but the tool FAILED to initiate the download :-( . So I'm asking again: Are you absolutely positive you're being served plain unencrypted DASH streams in your setup?
Can you please elaborate? Was it a playable unencrypted file you ended up with?
As ever, greatly appreciated 👍; pending a new combined-get_flash_videos release, I had to patch the previous release (combined-get_flash_videos-1.25.92) manually with a text editor as per commit e55ddb9 (or use the "dev" edition extracted into a folder). Things seem to work well again:
Seeing that they have now practically decommissioned their RTMP (Flash) streams, is there a contingency plan in the works for when/if they completely shut down their RTMP VOD infrastructure? Finally, and I don't want to sound ungrateful or greedy, I do want to request an enhancement to the Itv.pm, possibly echoing many overseas users:
As I posted previously, the RTMP streams themselves are not geo-blocked at CDN level, only the request to their playlist API must come from a whitelisted UK IP address. I am currently using a whitelisted UK HTTP proxy and the
The above is an excerpt from --help; in the case of many geo-restricted plugins (itv.py included), there's a built-in mechanism that kicks in automatically, see the Python code here... I fully understand that currently GFV is quite understaffed, but could something like that be implemented for Itv.pm too, so that no need for UK proxy/VPN when fetching from overseas? A randomly generated UK IP address is less prone to becoming blacklisted! With the hope my suggestion be considered in the future, many thanks again for all the hard work you have put thus far towards maintaining (often single-handedly) the ITV plugin 👍 |
I will create a release with the download files shortly. Gone back and checked and was wrong dash is being encrypted, there is a but here, OpenBSD chrome I use is built from the source, which means somewhere in the source is the means to decrypt the dash, or a supplied binary that does it (maybe a jar file). A partial proxy I can look at, however being located in the UK it may prove difficult to test. Adding a header such as X-Forwarded-For is simpler. May consider a mean of adding own headers. There is no code to handle any dash in GFV yet. |
Yes, extremely thankful for that! Both combined-get_flash_videos-1.25.93 and get_flash_videos-1.25.93 work fine on my setup for ITV Hub VOD!
Thought as much... Thanks for re-checking and verifying the bad news...
There are many free resources to mask your actual physical location; being in the UK is a plus in your case, since you simply have to choose one of the many overseas VPN locations; e.g. if OpenVPN is available in OpenBSD (or you can compile from source), then take a look here...
Unfortunately, this is a more generalised problem ;-( . Apart from get_iplayer 3.01, which specialises only in BBC iPlayer TV+Radio DASH streams, and youtube-dl, which can only fetch "static" - non-live - MPDs, there's a lack of free tools/scripts to download MPEG-DASH streams; which is kind of an odd thing, because MPEG-DASH is an open source implementation.
... Will be keeping an eye in this repo for this... |
where to get combined-get_flash_videos-1.25.93 and get_flash_videos-1.25.93 ? |
You get from here, click on code 2-3rd line down at top, should see appear below, commits, branches, releases, contributors. Select releases - combined-get_flash_videos-1.25.93 and get_flash_videos-1.25.93 should appear under the Downloads. If not, can use link below... |
THANKS, njtaylor! |
The hulu.pm was a separate project. 2014 hulu switched to newer flash, and became subscription only. I only maintain the itv plugin, other site plugins are supplied by others, I can create releases that include that they supply. Alternatives like youtube-dl, and kodi don't support hulu any more. |
In addition to the above, all hulu streams, at least on desktop browsers, are heavily encrypted, which is a no go by itself; unless you're a member of a highly expert hacker team (like the one that hacked HBO), use a screen recording software, if you can... |
Allavsoft still works great to download ITV videos. |
Keep to the topic ITV downloads work for most content, only content with additional encryption fails to download, comments aren't for advertising paid for software that only work on limited platforms. |
@njtaylor: The user above joined GitHub less than a month ago, with sole agenda apparently to spam repositories related to media content with plugs for paid-for third party Windows software; proof:
I would permanently block and report to GitHub (... but sadly those spammers tend to come back under a different alias) ... Regards |
ITV not longer downloads. It appears flash videos have been disabled and replace with dash. This doesn't work with firefox only chrome, firefox still ends up trying to download a flash plugin. Video does play in chrome should be possible to replace flash with dash, dash has been implemented for other sites.
The text was updated successfully, but these errors were encountered: