[youtube] Retry requests for page listings after temporary errors

[Zirro]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2020.12.26
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

[youtube:tab] CNN: Downloading webpage
[download] Downloading playlist: CNN - Videos
[youtube:tab] Downloading page 1
[youtube:tab] Downloading page 2
[youtube:tab] Downloading page 3
[youtube:tab] Downloading page 4
[youtube:tab] Downloading page 5
...more pages here...
[youtube:tab] Downloading page 1215
[youtube:tab] Downloading page 1216
[youtube:tab] Downloading page 1217
[youtube:tab] Downloading page 1218
[youtube:tab] Downloading page 1219
WARNING: Unable to download JSON metadata: HTTP Error 500: Internal Server Error
[youtube:tab] playlist CNN - Videos: Downloading 36570 videos
[download] Downloading video 1 of 36570

Description

URL: https://www.youtube.com/user/CNN/videos

YouTube will occasionally return an Error 500: Internal Server Error while downloading the pages of a playlist. This currently causes youtube-dl to proceed to the next step of downloading videos without having received the complete list.

These particular errors seem to be temporary one-offs that quickly resolve themselves with a retry, so it would probably be preferable for youtube-dl to reattempt the same request again to be able to continue retrieving the remaining pages from the list. This would be particularly helpful when downloading large playlists like the one above, as the alternative of restarting it from the beginning risks running into issues with rate limiting instead.