CSPAN gives 403 errors for each fragment

[dhelma]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2020.03.24
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.c-span.org/video/?3461-1/biden-campaign-speech']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.03.24
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg git-2020-01-15-0dc0837, ffprobe git-2020-01-15-0dc0837
[debug] Proxy map: {}
[CSpan] 3461: Downloading webpage
[CSpan] 151793: Downloading JSON metadata
[CSpan] 151793: Downloading XML
[CSpan] 151793: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://iphone.c-spanvideo.org/m3u8/program.151793.MP4.M20.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 366
[download] Destination: Biden Campaign Speech-151793.mp4
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 10 of 10)...
[download] Skipping fragment 1...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 2 (attempt 10 of 10)...
[download] Skipping fragment 2...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 3 (attempt 10 of 10)...
[download] Skipping fragment 3...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 4 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 4 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 4 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 4 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 4 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 4 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 4 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 4 (attempt 8 of 10)...

# I've omitted the output doing this for the remaining 300 or so fragments, before... 

[download] 100% of 0.00B in 05:29
[debug] ffmpeg command line: ffprobe -show_streams "file:Biden Campaign Speech-151793.mp4"

Description

I've tried downloading multiple videos from C-Span. It seems like the playlist is found, but the server is returning a 403 Forbidden, but I'm able to watch just fine in the browser.

[JNuss71]

I am getting this too with 2020.05.08

[frisch1]

Note we are seeing this on cspan as well. Based on the origin, this may be more generalized to AWS Cloudfront HLS content protection:
https://docs.aws.amazon.com/elastictranscoder/latest/developerguide/content-protection.html

Speculation only, but the respond pattern is similar to when this is engaged.

[bubbaprog]

I don't know how to implement this in code, but the content can be downloaded properly by providing the server with a proper Referer and User-Agent header.

[altaway]

True, the access is forbidden unless the referer is explicitly specified.

youtube-dl --referer "https://iphone.c-spanvideo.org/" <your_url>

The program already uses a sensible user-agent (check with --dump-user-agent and compare it with your browser's), so you don't have to specify it.