truTV giving ERROR: Unsupported URL

[bcoakiot]

Checklist

• [x ] I'm reporting a broken site support issue
• [x ] I've verified that I'm running youtube-dl version 2020.02.16
• [x ] I've checked that all provided URLs are alive and playable in a browser
• [x ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
• [x ] I've searched the bugtracker for similar bug reports including closed ones
• [x ] I've read bugs section in FAQ

Verbose log

youtube-dl https://www.trutv.com/shows/hot-ones-the-game-show/season-1/episode-1/beware-the-bucket -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.trutv.com/shows/hot-ones-the-game-show/season-1/episode-1/beware-the-bucket', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.02.16
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg 4.2.2
[debug] Proxy map: {}
[generic] beware-the-bucket: Requesting header
WARNING: Falling back on generic information extractor.
[generic] beware-the-bucket: Downloading webpage
[generic] beware-the-bucket: Extracting information
ERROR: Unsupported URL: https://www.trutv.com/shows/hot-ones-the-game-show/season-1/episode-1/beware-the-bucket
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpuz77jgzq\build\youtube_dl\YoutubeDL.py", line 796, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpuz77jgzq\build\youtube_dl\extractor\common.py", line 530, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpuz77jgzq\build\youtube_dl\extractor\generic.py", line 3350, in _real_extract
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.trutv.com/shows/hot-ones-the-game-show/season-1/episode-1/beware-the-bucket

Description

Error when trying to download from TruTV. It seems as if it defaults to the generic extractor instead of using the specific one. TBS seems to still work, which has the same player and URL structure.

[jhalvors-school]

So, did a little bit of exploring on this. It's maybe do-able, but you need an authentication token. When changing the regex in trutv.py, you'd want the regex to parse as

series_slug = hot-ones-the-game-show
clip_slug = season-1/episode-1/beware-the-bucket
id = None

The API link yielded by the extractor would be https://api.trutv.com/v2/web/series/clip/hot-ones-the-game-show/season-1/episode-1/beware-the-bucket

Unfortunately, visiting it gives {"message":"Missing Authentication Token"}

[nhlfarm]

I was able to download videos on trutv at 1080p using the m3u8 method. You can download videos this way for now until it is fixed.

1. Open trutv video and login to TV provider if needed
2. Ctrl + Shift + i and search for m3u8. Reload video and page if it doesnt show up
3. There will be a master.m3u8 and a 'layer' link. Copy the link address of the master.m3u8 and execute on youtube-dl. This should give you the options to the videos up to 720p.
4. To get 1080p, copy the link address for the 'layer' one instead, then change the number after the word 'layer' (this appears twice) to 7, which is 1080p and the highest possible resolution. I tried 8 and above but they don't give you a higher resolution than 1080p. Numbers lower than 7 are just resolutions that are downloadable from copying the master.m3u8 link.

[roadzy]

@nhlfarm does browser matter? I'm trying chrome and not having any luck. https://www.trutv.com/shows/tacoma-fd/season-1/episode-1/on-the-hot-seat-explicit

[ThatNerdyPikachu]

@nhlfarm This does not work on videos with ads, it just grabs up to the first ad.

(This is the same issue I ran into with my PR)

[nhlfarm]

@roadzy I don't think browser matters. I just tried downloading the tacoma fd episode and I was able to get the 'layer' link on both chrome and firefox: https://i.imgur.com/ef1LtB7.jpg

After that, I just copied the link from chrome by right clicking -> Copy -> Copy link address. Then, I changed the two times 'layer3' appears into 'layer7' and executed it on youtube-dl: https://i.imgur.com/VkoKoRV.png

[nhlfarm]

@ThatNerdyPikachu I am able to download full episodes with the method I posted above. I have ublock origin on both chrome and firefox, and thought that maybe it was able to give a layer link that bypasses the ads when downloading with youtube-dl. But then I turned off ublock origin on chrome, then got the m3u8 layer link: https://i.imgur.com/t7Lkg8I.png
I executed it on youtube-dl and was still able to download the full episode: https://i.imgur.com/ey8Sbuq.png
I haven't had any instance where youtube-dl only downloads up to the first ad; I've only downloaded the full episodes. If you need any additional info I am glad to help

[roadzy]

@nhlfarm seems to work except, I keep getting this:
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:0; previous: 55544263, current: 11171171; changing to 55544264. This may result in incorrect timestamps in the output file.
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:0; previous: 55544264, current: 11174174; changing to 55544265. This may result in incorrect timestamps in the output file.
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:0; previous: 55544265, current: 11177177; changing to 55544266. This may result in incorrect timestamps in the output file.
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:1; previous: 27224276, current: 5459487; changing to 27224277. This may result in incorrect timestamps in the output file.
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:1; previous: 27224277, current: 5460511; changing to 27224278. This may result in incorrect timestamps in the output file.
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:1; previous: 27224278, current: 5461536; changing to 27224279. This may result in incorrect timestamps in the output file.
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:1; previous: 27224279, current: 5462559; changing to 27224280. This may result in incorrect timestamps in the output file.
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:1; previous: 27224280, current: 5463583; changing to 27224281. This may result in incorrect timestamps in the output file.
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:1; previous: 27224281, current: 5464607; changing to 27224282. This may result in incorrect timestamps in the output file.
[mp4 @ 0x3bda9c0] Non-monotonous DTS in output stream 0:1; previous: 27224282, current: 5

Looks like its chopped into 3 sections on the page. The file it creates is only part 1 which is only 10 minutes. I know for Nick Jr/MTV, it downloads each part and I have a script that merges them as one file.

[nhlfarm]

@roadzy I got that error before too, and I believe it's because it's downloading the video using ffmpeg, which is the default HLS downloader. However, I was still able to download the episodes fully, although the fps was not maximized (i.e. 29.970000 fps). I would download and get the same errors you're getting (non-monotonous DTS), which would cause the videos to be like 28.84415 fps for example.

The fix that worked for me (which should also work for you as well) was to choose the HLS downloader as HLS native instead. You just have to add "--hls-prefer-native" without the quotations into the command and you should download the video using HLS native, as seen in the example of my previous post. I didn't get any errors after doing this and was able to download them fully at 29.970000 fps all the time.

[roadzy]

@nhlfarm what are you doing to get the full 24 minutes? I'm only getting to the first ad break. Mine is like 10 minutes. I've tried multiple episodes.

[nhlfarm]

@roadzy I'm literally just doing the exact steps of what I posted above, and I'm able to get the full episodes. I haven't encountered the error that you and nerdypikachu are getting where only up to the first ad break is downloaded. I have all the full Adam Ruins Everything episodes downloaded and tried multiple episodes of tacoma pd and haven't downloaded any 10 minute ones; only the full episodes. Have you tried it with/without adblock or with a different browser, or updating youtube-dl and ffmpeg? I would try those and also try using the master link and downloading the 720p version and see if it downloads the full episode for you.

[roadzy]

@nhlfarm just curious... can you try this and see if you get the whole clip or just 12 minutes like i am?

youtube-dl --hls-prefer-native https://tve.cdn.turner.com/trutv/c52b7d45944f5d3a7c115e6584e230f4/layer7/layer7_bk.m3u8?hdntl=exp=1587546364~acl=%2ftrutv%2fc52b7d45944f5d3a7c115e6584e230f4%2f*~hmac=0ff4fb37759d472ae5350f3403d0a54769cd96369c44dc8ad1b5feab739dd7d5

[nhlfarm]

@roadzy I just tried to execute that but got HTTP Error 403: Forbidden, probably because the session already expired since I am trying it 15 hours after you posted it. Instead, I downloaded and uploaded a random episode of tacoma, with screenshots of every step I did:

1. Selected tacoma episode. Opened network activity with ctrl+ shift + i and searched for 'm3u8,' then played the episode. Copied layer link: https://i.imgur.com/8MGTTY5.png
2. Changed the 2 instances of 'layer3' to 'layer7' then executed on youtube-dl: https://i.imgur.com/yFoTZSk.png
   (The first command I executed was the one you provided, which gave the HTTP 403 error)
3. After it finished downloading, this is what my cmd looked like: https://i.imgur.com/EtoUifw.png
   It downloaded a ~900 MB episode.
4. I opened the .mp4 file on VLC and this is what I got: https://i.imgur.com/BzJuRyB.png
   Notice the codec info: 1920x1080, 29.970000 fps, and it's 24:32 long, which is the full episode.
5. I uploaded it to mega: /file/OJc0VYZD#FxTBAMEjrc3fNkdgd0dw0E4kuAfvJeJNK6yQ-9t8EPk
   (add that after mega.nz)

[roadzy]

@nhlfarm I figured it out. The issue was with ffmpeg. When you ran --hls-prefer-native it would fail over to ffmpeg. From the first day I started using youtube-dl, it has always just used ffmpeg. I didn't know there was another option. So digging further, I found out I needed pycrypto to get the native to work. Once it switched to the native, 10 minute episode became 24 minute. Also I had the DTS issue with fox.com using ffmpeg and it fixed that too! What is it using to encode? Should I always be using hls-prefer-native ? Thanks for all your help!

[nhlfarm]

@roadzy For me, since I started using youtube-dl it has defaulted to using the native HLS downloader, which is what was always used to download videos for me, and I never got any problems. When trying to download from trutv is when I realized youtube-dl could also use ffmpeg to download videos, but I kept getting errors unlike with native HLS. It's weird that your youtube-dl defaults to using ffmpeg instead of native HLS, maybe it's an outdated version. But I'm glad to hear you got it working and are able to download the full episodes. When using the '--hls-prefer-native' I am not 100% sure of what it is using to encode, I just know that it's amazing and I will always use it over ffmpeg. So I just add '--hls-prefer-native' to all of commands now, and I recommend you do too, since you said your youtube-dl defaults to ffmpeg.

[sofakng]

@nhlfarm I've had the same issue with TruTV and FFmpeg's HLS downloader. The downloaded files also played incorrectly. After switching to the native HLS implementation it downloaded without HLS errors and it played back perfectly.

I wonder the status of the pull request that fixes TruTV? (#24466)

[ThatNerdyPikachu]

I wonder the status of the pull request that fixes TruTV? (#24466)

I'll test my PR with the native HLS mode and report back.