[Pornhub] TypeError: cannot unpack non-iterable bool object

[rtentser]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2020.07.28
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

$ youtube-dl -v https://www.pornhub.com/view_video.php?viewkey=ph5f29eb606fbb4
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.pornhub.com/view_video.php?viewkey=ph5f29eb606fbb4']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 3.8.5 (CPython) - Linux-5.8.3-6-tkg-pds-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1
[debug] Proxy map: {}
[PornHub] ph5f29eb606fbb4: Downloading pc webpage
[PornHub] ph5f29eb606fbb4: Downloading m3u8 information
[PornHub] ph5f29eb606fbb4: Downloading m3u8 information
[PornHub] ph5f29eb606fbb4: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 500: Internal Server Error
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 33, in <module>
    sys.exit(load_entry_point('youtube-dl==2020.7.28', 'console_scripts', 'youtube-dl')())
  File "/usr/lib/python3.8/site-packages/youtube_dl/__init__.py", line 474, in main
    _real_main(argv)
  File "/usr/lib/python3.8/site-packages/youtube_dl/__init__.py", line 464, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 2018, in download
    res = self.extract_info(
  File "/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/pornhub.py", line 321, in _real_extract
    formats.extend(self._extract_m3u8_formats(
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 1614, in _extract_m3u8_formats
    res = self._download_webpage_handle(
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/pornhub.py", line 33, in _download_webpage_handle
    webpage, urlh = dl(*args, **kwargs)
TypeError: cannot unpack non-iterable bool object

Description

Strangely, it works with -v option after url. It doesn't work without. It doesn't work if i provide -f option after -v.

$ youtube-dl https://www.pornhub.com/view_video.php?viewkey=ph5f29eb606fbb4 -f 1080p -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.pornhub.com/view_video.php?viewkey=ph5f29eb606fbb4', '-f', '1080p', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 3.8.5 (CPython) - Linux-5.8.3-6-tkg-pds-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1
[debug] Proxy map: {}
[PornHub] ph5f29eb606fbb4: Downloading pc webpage
[PornHub] ph5f29eb606fbb4: Downloading m3u8 information
[PornHub] ph5f29eb606fbb4: Downloading m3u8 information
[PornHub] ph5f29eb606fbb4: Downloading m3u8 information
[PornHub] ph5f29eb606fbb4: Downloading m3u8 information
[PornHub] ph5f29eb606fbb4: Downloading m3u8 information
[debug] Invoking downloader on 'https://cv.phncdn.com/videos/202008/04/339442501/1080P_8000K_339442501.mp4?zDJw-mu16aEVe3EGWvK2pAcpKs5aXHKutnTpcSDDpdKT6krZ12z9qtZOR_5-L3gFUnmqZufo4NbZ3QSrzVybtZpFjtYqUn674LGUTqtXJfNkbArGZXz8i-go4lNJlCGJcptRtyt1x5Gmr6-cFCnfpeDjSa6KYf-x0rQwdEYB5TIt0W99U2ccS5YgHVYRfhWI5B9yu4X-L-Uh'
[download] Destination: Sexy Teen Naked Booty Dance Panty Try Haul-ph5f29eb606fbb4.mp4
[download] 100% of 294.36MiB in 09:46

[TheRealDude2]

I have now tested it for a while and also on different days. The download seems to work only arbitrarily.

I have tested without option, with "-v", with "-f 1080p" and with "-f 1080p -v", in random order.

The best chance that the download starts is with the option "-f 1080p". Here the download worked in about half of the attempts, in the other combinations about 1 time from 5 to 10 attempts. Strange is, that without options the download works almost as bad as with "-v".

I suspect a Timinig problem behind it. The longer the request to the server takes, the more probable a "HTTP Error 500: Internal Server Error" appears.

[msilletti]

I tried appending the both those options multiple times but am getting the same error. I just build from the latest master.

file=https://www.pornhub.com/view_video.php?viewkey=ph5f3d14b2c5db0
youtube-dl $file -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.pornhub.com/view_video.php?viewkey=ph5f3d14b2c5db0', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 3.8.5 (CPython) - Linux-5.4.61-1-lts-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1
[debug] Proxy map: {}
[PornHub] ph5f3d14b2c5db0: Downloading pc webpage
[PornHub] ph5f3d14b2c5db0: Downloading m3u8 information
[PornHub] ph5f3d14b2c5db0: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 500: Internal Server Error
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 33, in <module>
    sys.exit(load_entry_point('youtube-dl==2020.7.28', 'console_scripts', 'youtube-dl')())
  File "/usr/lib/python3.8/site-packages/youtube_dl/__init__.py", line 474, in main
    _real_main(argv)
  File "/usr/lib/python3.8/site-packages/youtube_dl/__init__.py", line 464, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 2018, in download
    res = self.extract_info(
  File "/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/pornhub.py", line 321, in _real_extract
    formats.extend(self._extract_m3u8_formats(
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 1614, in _extract_m3u8_formats
    res = self._download_webpage_handle(
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/pornhub.py", line 33, in _download_webpage_handle
    webpage, urlh = dl(*args, **kwargs)
TypeError: cannot unpack non-iterable bool object

[TheRealDude2]

I think it really is the server. I added a small loop in the extractor common.py to the function def _extract_m3u8_formats.
As expected there are still errors, but the retry starts the download on every attempt.

Since the extractor common.py is an important basis for all extractors, I don't really dare to do this for the master, since I cannot judge the interactions for other extractors.

Maybe someone could test it or write a nicer loop. I have adapted the function to something like (common.py, line 1618):

def _extract_m3u8_formats(self, m3u8_url, video_id, ext=None,
entry_protocol='m3u8', preference=None,
m3u8_id=None, note=None, errnote=None,
fatal=True, live=False, data=None, headers={},
query={}):
i=0
while i<=3:
try:
res = self._download_webpage_handle(
m3u8_url, video_id,
note=note or 'Downloading m3u8 information',
errnote=errnote or 'Failed to download m3u8 information',
fatal=fatal, data=data, headers=headers, query=query)
i = i + 1
except:
continue
else:
break