Cannot Download from ITV

[sxmpete]

Cannot download Good Karma Hospital. Get "WARNING: hlsnative has detected features it does not support, extraction will be delegated to ffmpeg". DL then appears to start, but Keep getting error messages about "Invalid Timestamps". This is the same problem as Issues 353 and 363.
Mac OS 10.14, GIA 1.21.13, same problems using both ExpressVPN and Nord VPN.
Verbose log attached
GIA 1.21.13 ITV Fail 26Jan22.docx

[sleaux-meaux]

Those messages can often be ignored if the download finishes successfully. Let it run to completion and then check the file.

[skovatch]

This happens for me on an M1 MacBook because the crypto library wasn't built for the same architecture as the included version of python. Youtube-dl can't handle decrypting the video segments, so it passes it off to ffmpeg, which is what generates all of those warnings. The video still downloads correctly if you let it run to completion.

I'm guessing, though, that you don't have one of the newer Macs?

[sxmpete]

2019 iMac running Mojave (for app compatibility reasons).
I gave up after 32 log pages of errors, but will rty to leave it longer to see if it does indeed finish...

[sxmpete]

Let attempted DL run for 40 minutes. It did not complete, there was no longer any network activity and the log had stopped updating after 1286 pages.

[sleaux-meaux]

It could be that slow if downloading at or below realtime. It's not unheard of with a crummy VPN - you would have to check the file to see how much of the programme was downloaded. However, it would be more profitable to address the underlying problem. I see now that you have spliced in python3 for GiA, which I suspect is the root cause. It will not be able to load modules from pycrypto package included with GiA, which is compiled for Python 2.7, so youtube-dl will fail over to ffmpeg for downloading. The Mojave-supplied Python 2.7 should work OK. so remove the link to python3 (I assume you have followed #277).

[sxmpete]

Thanks! Yes I followed 277 suggestion and it worked, for a while. I will remove link and give it a try

…

On Sat, Jan 29, 2022 at 1:23 PM sleaux-meaux ***@***.***> wrote: It could be that slow if downloading at or below realtime. It's not unheard of - you would have to check the file to see how much of the programme was downloaded. However, it would be more profitable to address the underlying problem. I see now that you have spliced in python3 for GiA, which I suspect is the root cause. It will not be able to load PyCrypto module included with GiA, which is compiled for Python 2.7, so youtube-dl will fail over to ffmpeg for downloading. The Mojave-supplied Python 2.7 should work OK. so remove the link to python3 (I assume you have followed #277 <#277>). — Reply to this email directly, view it on GitHub <#372 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGZX3OOSFX3QKNP72AXISM3UYQWDXANCNFSM5NAVZABA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Regards, Peter

[sxmpete]

Per sleaux-meaux suggestion, reverted to MacOS version of Python (2.7.10).
Attempted DL started ok (no hand-off to ffmpeg this time) but failed on what appears to be a python related error, log attached
GIA ITV Fail 30Jan22.docx
:

[sleaux-meaux]

Similar problem already reported upstream: ytdl-org/youtube-dl#30544

There is nothing GiA can do in the face of network/server problems like that. Keep retrying the download. It should pick up where previous attempt left off and eventually finish.

[nicacy]

It will not be able to load modules from pycrypto package included with GiA, which is compiled for Python 2.7, so youtube-dl will fail over to ffmpeg for downloading. The Mojave-supplied Python 2.7 should work OK. so remove the link to python3 (I assume you have followed #277).

Hijacking this thread slightly, to query specifically this response. I've been having no luck downloading anything for the past few days, and as all happened to be ITV programmes I looked to this thread. But I'm running macOS 12.3 beta, in which the already deprecated Python 2.7 has been removed. I'm guessing this is the reason that Get iPlayer Automator no longer works (for me)?

[skovatch]

Hijacking this thread slightly, to query specifically this response. I've been having no luck downloading anything for the past few days, and as all happened to be ITV programmes I looked to this thread. But I'm running macOS 12.3 beta, in which the already deprecated Python 2.7 has been removed. I'm guessing this is the reason that Get iPlayer Automator no longer works (for me)?

This is correct -- Apple removed Python 2.7, and GiA looks for "/usr/bin/python" when starting youtube-dl for ITV downloads.

This is now a minor fire drill for me, since this breaks GiA in a couple of ways, and not just for ITV. I'm building a Python 3.x distribution that I can include inside of GiA, but the process isn't documented very well. I'm convinced it's doable, though. :-)

[sleaux-meaux]

For now, you can just install Python 2 from python.org and splice it into GiA as in #277 (comment). Python 3 would also probably work as long as you install the pycryptodome package to avoid delegation to ffmpeg for downloading (unverified).

[skovatch]

Yes, PyCryptodome does work with python3. I verified that part. I was able to create a python3 distribution and bundle it, but it leaves hard-coded paths in random places causing it to fail when I put the app on another machine.

I don't mind if you want to install Python yourself, but you shouldn't have to. Python 2.7 is outdated and I'd really like to avoid requiring any kind of additional installation, as it goes against the philosophy of dragging one thing to the Applications folder and launching the app.

[sleaux-meaux]

Sorry, that comment was for @nicacy

[cgareth]

For now, you can just install Python 2 from python.org and splice it into GiA as in #277 (comment). Python 3 would also probably work as long as you install the pycryptodome package to avoid delegation to ffmpeg for downloading (unverified).

To sleaux-meaux - I've just followed your advice to reread the comment link, and thank you. I remain on OSX10.12 with the latest GiA and have accepted not being able to download from ITV for over a year. I think I'd messed around with updating and reverting Python too much and had just broken something.

For what it's worth, I followed the instructions in the post I (having forgotten I'd ever done it) dumbed down and requoted from Steps39 on Feb 14th 2020. I don't know how to link directly to it from here.

Thanks again Scott, and to all you others who remain invaluable.

[sleaux-meaux]

@skovatch Consider switching to yt-dlp. It is patched for ITV and one of their build scripts could be used to generate a standalone executable (via pyinstaller --onedir) that presumably could be dropped into GiA. You may need to build it yourself for compatibility with older macOS (unless all dependencies are supplied via wheels). Also, their build was broken in latest release 2022.01.21, though previous release 2021.12.27 would be OK for ITV. I don't think you need to invoke python directly since it is no longer used to convert ITV subtitles.

[skovatch]

@sleaux-meaux yt-dlp does look like a good option. I'm finding that building a python distribution on macOS is not for the faint of heart, particularly when it comes to backwards compatibility.

[sleaux-meaux]

FWIW, I've verified that it's possible to build yt-dlp with pyinstaller to run on a clean 10.10 (Intel) system if built with python.org Python 3. The release build doesn't seem to work on macOS earlier than 10.15, which I think is probably due to their use of Xcode-supplied Python 3.

[skovatch]

Yup, I have a build now with a PyInstaller-built yt-dlp from a 3.9 python I built myself. It pulls in openssl and pycryptodomex and is built for arm64 and x86_64. You have to set MACOSX_DEPLOYMENT_TARGET=10.10

[skovatch]

I tested on my M1 MacBook pro and my work laptop which is an intel MBP running Big Sur. I can't put my VPN on my work laptop so I couldn't completely test but it did invoke yt-dlp and fetch page metadata.

[nicacy]

I'm building a Python 3.x distribution that I can include inside of GiA, but the process isn't documented very well. I'm convinced it's doable, though. :-)

Fingers crossed!

[nicacy]

For now, you can just install Python 2 from python.org and splice it into GiA as in #277 (comment). Python 3 would also probably work as long as you install the pycryptodome package to avoid delegation to ffmpeg for downloading (unverified).

Thanks very much for the response. I installed P2 and tried following the info in the other thread, but nothing would make GiA work with ITV downloads. I have zero experience with Python — so no idea how to go about troubleshooting — so for the time being at least it looks as though only BBC downloads are in my future!

[cgareth]

For now, you can just install Python 2 from python.org and splice it into GiA as in #277 (comment). Python 3 would also probably work as long as you install the pycryptodome package to avoid delegation to ffmpeg for downloading (unverified).

Thanks very much for the response. I installed P2 and tried following the info in the other thread, but nothing would make GiA work with ITV downloads. I have zero experience with Python — so no idea how to go about troubleshooting — so for the time being at least it looks as though only BBC downloads are in my future!

I was in the same boat as you until a couple of days ago. This works for me (I'm on Sierra and the newest GiA)

#277 (comment)

[nicacy]

I was in the same boat as you until a couple of days ago. This works for me (I'm on Sierra and the newest GiA)

#277 (comment)

Encouraged by this, I tried again today on a different Mac (also running 12.3) using those step-by-step instructions, and it worked — ITV downloads are back 😊.

[sxmpete]

Latest workaround suggested in issue 354 fixes this problem on older Mac Os's up to Mojave by installing python 2.7.17 which contains TLS 1.02.
ITV downloads are back!

[cgareth]

OS10.12
Latest GiA
Tried downloading Trigger Point.

Having worked again last week, ITV isn't downloading again for me; I'd wouldn't normally question this but this part of the log caught my eye -

youtube-dl version 2021.04.26-gia_01
[debug] Python version 2.7.10 (CPython) - Darwin-16.5.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.2-tessus
[debug] Proxy map: {}
[debug] Using fake IP 25.240.109.211 (GB) as X-Forwarded-For.

My python version is in fact 2.7.16 (I checked this on Terminal).
I'm not using a fake IP.

I've tried the identical download on the previous two versions of GiA, to no avail.
Don't know if this is of interest / any use, but one never knows.

[skovatch]

OS10.12 Latest GiA Tried downloading Trigger Point.

My python version is in fact 2.7.16 (I checked this on Terminal). I'm not using a fake IP.

This means that your environment that is started from Terminal.app is finding your 2.7.16 version of python in your path, while the system environment is finding /usr/bin/python first, which is 2.7.10.

What does /usr/bin/env python give you?

[cgareth]

Hi Scott,
Do ignore my first reply below. I've reinstalled the necessary Python and all is good (I presume installing the updates to GiA mucked things up for me).

Every day here is a lesson!

This means that your environment that is started from Terminal.app is finding your 2.7.16 version of python in your path, while the system environment is finding /usr/bin/python first, which is 2.7.10.

What does /usr/bin/env python give you?

Hi Scott,
It gives me this:

Python 2.7.17 (v2.7.17:c2f86d86e6, Oct 19 2019, 16:24:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

(I haven't done anything (to my knowledge at least) to GiA directly to re-halt ITV downloads.)
Is that reply notable to any degree?

[sleaux-meaux]

Every time you update GiA, you need to re-create symbolic link for python as in #277 (comment) for 10.12 and below

[skovatch]

I'm closing this as fixed in v1.23.0