Pluralsight - Unable to download webpage: HTTP Error 429

[mirfilip]

Hi,
I will start with an acknowledgement I've read all the 429 related issues here, none seem to fit.

I'm running the latest version. I'm trying to download a course with options --username "my-user-name" --password "my-password" --user-agent "my-user-agent-here" --sleep-interval X --no-overwrites --verbose "course-url"

It downloads fine but hits 429 after a period of time:

[pluralsight] course-name-here-video: Downloading webpage
[pluralsight] course-name-here-video: Downloading mp4-high URL
WARNING: Unable to download webpage: HTTP Error 429:
[pluralsight] course-name-here-video: Waiting for 2 seconds to avoid throttling
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 666, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 316, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/pluralsight.py", line 226, in _real_extract
    self._sort_formats(formats)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 829, in _sort_formats
    raise ExtractorError('No video formats found')
ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Surely, it's not youtube-dls fault that I got 429. Nor is changing the location you download from the solution. Simply, by waiting some period of time and then rerunning the script, it works flawlessly.

I'm running with default retries parameter (docs say "default is 10") so I would expect to have the request retried 10 times. From error message, it doesn't look so. Also, sleeping sleep-interval before retrying would be good. Does that happen?

It looks to me like a totally recoverable error, but instead the script halts. Am I missing something?

[dstftw]

--retries has effect only on the actual final file downloading. I doubt 429 is recoverable at all since it's a sign of overuse.
You can skip videos that encountered an error during extraction with -i.

[mirfilip]

@dstftw It is recoverable. Every time I encounter 429 and script hard fails and exit, I'm able to rerun it a minute or two later and it works. By definition 429 is

The user has sent too many requests in a given amount of time
so it's recoverable by sleeping for some time before retrying.

The docs you brought advise to contact the provider if 429 happens. With Pluralsight, 429 is a sign you start to hit a rate limit, not that your account is blocked (if that happens, you get 402 or simply 403). So you just wait some time and you're fine.

When hitting a rate limit, some sites would even provide extra info with Retry-After header set. Sleeping for that period of time is enough.

If that use case is not covered currently, please consider this task a feature request.