Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems and inconsistencies downloading Pornhub playlists (favorites) #11444

Closed
bestwaytodesign opened this issue Dec 14, 2016 · 17 comments
Closed
Labels

Comments

@bestwaytodesign
Copy link

@bestwaytodesign bestwaytodesign commented Dec 14, 2016

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like that [x])
  • Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.12.12. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

  • [ x] I've verified and I assure that I'm running youtube-dl 2016.12.12

Before submitting an issue make sure you have:

  • [ x] At least skimmed through README and most notably FAQ and BUGS sections
  • [ x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • [x ] Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue


If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

$ youtube-dl -u PORNHUB_USER -p PASSWD --verbose --min-sleep-interval 70 --max-sleep-interval 110 http://www.pornhub.com/users/PORNHUB_USER/videos/favorites?o=oldest&page=1
[1] 24526
os_user@PCNAME:/PATH/TO/PATH$ [debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'--verbose', u'--min-sleep-interval', u'70', u'--max-sleep-interval', u'110', u'http://www.pornhub.com/users/PORNHUB_USER/videos/favorites?o=oldest']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.12.12
[debug] Python version 2.7.9 - Linux-3.2.40-armv7l-with-debian-8.5
[debug] exe versions: ffmpeg 3.1.1, ffprobe 3.1.1
[debug] Proxy map: {}
[PornHubUserVideos] PORNHUB_USER: Downloading page 1
[PornHubUserVideos] PORNHUB_USER: Downloading page 2
[PornHubUserVideos] PORNHUB_USER: Downloading page 3
[PornHubUserVideos] PORNHUB_USER: Downloading page 4
[PornHubUserVideos] PORNHUB_USER: Downloading page 5
[PornHubUserVideos] PORNHUB_USER: Downloading page 6
[PornHubUserVideos] PORNHUB_USER: Downloading page 7
[PornHubUserVideos] PORNHUB_USER: Downloading page 8
[PornHubUserVideos] PORNHUB_USER: Downloading page 9
[PornHubUserVideos] PORNHUB_USER: Downloading page 10
[PornHubUserVideos] PORNHUB_USER: Downloading page 11
[download] Downloading playlist: PORNHUB_USER
[PornHubUserVideos] playlist PORNHUB_USER: Collected 40 video ids (downloading 40 of them)
[download] Downloading video 1 of 40
[PornHub] ph5822dc94ab070: Downloading webpage
ERROR: Unable to extract title; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 694, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 357, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/pornhub.py", line 134, in _real_extract
    webpage, 'title', group='title')
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 660, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract title; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

...
<end of log>

Description of your issue, suggested solution and other information

Trying to download a full playlist from pornhub using youtube-dl is not consistent and errors out sometimes (see log). The "pages" it tries to download (messages [PornHubUserVideos] PORNHUB_USER: Downloading page #) isn't consistent either, sometimes it's 11 others like 40. Something is odd here.

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Dec 14, 2016

Post concrete examples.

@bestwaytodesign
Copy link
Author

@bestwaytodesign bestwaytodesign commented Dec 14, 2016

@dstftw do you want the user and password too? I described the issue and provided the log, you don't need anything else in order to reproduce it on your end.

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Dec 14, 2016

You must provide concrete example URLs that reproduce the behavior you consider incorrect.

@bestwaytodesign
Copy link
Author

@bestwaytodesign bestwaytodesign commented Dec 14, 2016

@dstftw any favorites playlist url will do.

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Dec 14, 2016

I'm not going to find example URLs myself and fortune tell what's not consistent. Providing complete report is your responsibility.

@dstftw dstftw closed this Dec 14, 2016
@dstftw dstftw added the invalid label Dec 14, 2016
@bestwaytodesign
Copy link
Author

@bestwaytodesign bestwaytodesign commented Dec 14, 2016

Lol, it's your script dude, if you don't want to fix it suit yourself.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Dec 15, 2016

@bestwaytodesign You didn't provide a link that is broken so nobody is able to fix it. "http://www.pornhub.com/users/PORNHUB_USER/videos/favorites?o=oldest&page=1" is not enough as things may be different for different users. It's a waste of time if we fix it for another user account but the result does not fit your need.

@bestwaytodesign
Copy link
Author

@bestwaytodesign bestwaytodesign commented Dec 15, 2016

@yan12125 it's very simple: try to reproduce it for another account and if you can't just let it be then. If you can reproduce it then fix it and it will probably fix it for all other accounts too. You guys chose the third option of just letting it be and that's fine by me: it's your script so if you're happy leaving it alone then I'm happy too.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Dec 15, 2016

Sorry, I guess I can't fix it.

$ youtube-dl -vF "http://www.pornhub.com/users/zoe_ph/videos/public"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-vF', 'http://www.pornhub.com/users/zoe_ph/videos/public']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.12.12
[debug] Git HEAD: 30918999f
[debug] Python version 3.5.2 - Linux-4.8.13-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 3.2.2, ffprobe 3.2.2, rtmpdump 2.4
[debug] Proxy map: {}
[PornHubUserVideos] zoe_ph: Downloading page 1
[PornHubUserVideos] zoe_ph: Downloading page 2
[PornHubUserVideos] zoe_ph: Downloading page 3
[PornHubUserVideos] zoe_ph: Downloading page 4
[PornHubUserVideos] zoe_ph: Downloading page 5
[PornHubUserVideos] zoe_ph: Downloading page 6
[download] Downloading playlist: zoe_ph
[PornHubUserVideos] playlist zoe_ph: Collected 257 video ids (downloading 257 of them)
[download] Downloading video 1 of 257
[PornHub] ph5847a35e1d661: Downloading webpage
[info] Available formats for ph5847a35e1d661:
format code  extension  resolution note
240P-400K    mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 2 of 257
[PornHub] 1070030428: Downloading webpage
[info] Available formats for 1070030428:
format code  extension  resolution note
vl-240-0     mp4        unknown    
vl-240-1     mp4        unknown    
vl-720       mp4        unknown    
480P-600k    mp4        480p        600k  (best)
[download] Downloading video 3 of 257
[PornHub] 625732834: Downloading webpage
[info] Available formats for 625732834:
format code  extension  resolution note
vl-240P-0    mp4        unknown    
vl-240P-1    mp4        unknown    
vl-480P      mp4        unknown    (best)
[download] Downloading video 4 of 257
[PornHub] 227863594: Downloading webpage
^C
ERROR: Interrupted by user
@bestwaytodesign
Copy link
Author

@bestwaytodesign bestwaytodesign commented Dec 15, 2016

@yan12125 If you don't want to do something about it just don't but please spare the subterfuges: the url I put in my log was obviously from the 'favorites' playlist.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Dec 15, 2016

Well, basically public videos are the same as favorite videos. Tried this user several times and always get 13 videos.

$ youtube-dl -vF "http://www.pornhub.com/users/bgfpclips/videos/favorites"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-vF', 'http://www.pornhub.com/users/bgfpclips/videos/favorites']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.12.12
[debug] Git HEAD: 30918999f
[debug] Python version 3.5.2 - Linux-4.8.13-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 3.2.2, ffprobe 3.2.2, rtmpdump 2.4
[debug] Proxy map: {}
[PornHubUserVideos] bgfpclips: Downloading page 1
[PornHubUserVideos] bgfpclips: Downloading page 2
[download] Downloading playlist: bgfpclips
[PornHubUserVideos] playlist bgfpclips: Collected 13 video ids (downloading 13 of them)
[download] Downloading video 1 of 13
[PornHub] ph57965c442a734: Downloading webpage
[info] Available formats for ph57965c442a734:
format code  extension  resolution note
240P-400K-0  mp4        240p        400k 
240P-400K-1  mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 2 of 13
[PornHub] ph57e6c010e1ca0: Downloading webpage
[info] Available formats for ph57e6c010e1ca0:
format code  extension  resolution note
240P-400K-0  mp4        240p        400k 
240P-400K-1  mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 3 of 13
[PornHub] 228258877: Downloading webpage
[info] Available formats for 228258877:
format code  extension  resolution note
240P-338K-0  mp4        240p        338k 
240P-338K-1  mp4        240p        338k 
480P-362K    mp4        480p        362k  (best)
[download] Downloading video 4 of 13
[PornHub] 432755412: Downloading webpage
[info] Available formats for 432755412:
format code  extension  resolution note
vl-240P-0    mp4        unknown    
vl-240P-1    mp4        unknown    
vl-480P      mp4        unknown    (best)
[download] Downloading video 5 of 13
[PornHub] ph57732ec66899f: Downloading webpage
[info] Available formats for ph57732ec66899f:
format code  extension  resolution note
240P-400K-0  mp4        240p        400k 
240P-400K-1  mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 6 of 13
[PornHub] 1287925133: Downloading webpage
[info] Available formats for 1287925133:
format code  extension  resolution note
vl-240-0     mp4        unknown    
vl-240-1     mp4        unknown    
480P-600K    mp4        480p        600k  (best)
[download] Downloading video 7 of 13
[PornHub] 1068766821: Downloading webpage
[info] Available formats for 1068766821:
format code  extension  resolution note
vl-240P-0    mp4        unknown    
vl-240P-1    mp4        unknown    
vl-480P      mp4        unknown    
vl-720P      mp4        unknown    (best)
[download] Downloading video 8 of 13
[PornHub] 872479661: Downloading webpage
[info] Available formats for 872479661:
format code  extension  resolution note
vl-240P-0    mp4        unknown    
vl-240P-1    mp4        unknown    
vl-480P      mp4        unknown    (best)
[download] Downloading video 9 of 13
[PornHub] 803489516: Downloading webpage
[info] Available formats for 803489516:
format code  extension  resolution note
vl-240P-0    mp4        unknown    
vl-240P-1    mp4        unknown    
vl-480P      mp4        unknown    (best)
[download] Downloading video 10 of 13
[PornHub] ph580554a0ee995: Downloading webpage
[info] Available formats for ph580554a0ee995:
format code  extension  resolution note
240P-400K    mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 11 of 13
[PornHub] ph580544e395dcc: Downloading webpage
[info] Available formats for ph580544e395dcc:
format code  extension  resolution note
240P-400K    mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 12 of 13
[PornHub] ph57ec57bb588ec: Downloading webpage
[info] Available formats for ph57ec57bb588ec:
format code  extension  resolution note
240P-400K-0  mp4        240p        400k 
240P-400K-1  mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 13 of 13
[PornHub] ph57ec4eb14ae4b: Downloading webpage
[info] Available formats for ph57ec4eb14ae4b:
format code  extension  resolution note
240P-400K    mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Finished downloading playlist: bgfpclips
@bestwaytodesign
Copy link
Author

@bestwaytodesign bestwaytodesign commented Dec 15, 2016

@yan12125 Using the same command you used and on another installation of youtube-dl (on another pc, and apparently more uptodate script) I get the same:

youtube-dl -vF "http://www.pornhub.com/users/bgfpclips/videos/favorites"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-vF', u'http://www.pornhub.com/users/bgfpclips/videos/favorites']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.12.15
[debug] Python version 2.7.9 - Linux-3.16.0-4-amd64-x86_64-with-debian-8.6
[debug] exe versions: avconv 11.8-6, avprobe 11.8-6, ffmpeg N-79947-gcd244fa-static, ffprobe N-79947-gcd244fa-static
[debug] Proxy map: {}
[PornHubUserVideos] bgfpclips: Downloading page 1
[PornHubUserVideos] bgfpclips: Downloading page 2
[download] Downloading playlist: bgfpclips
[PornHubUserVideos] playlist bgfpclips: Collected 13 video ids (downloading 13 of them)
[download] Downloading video 1 of 13
[PornHub] 742332421: Downloading webpage
[info] Available formats for 742332421:
format code  extension  resolution note
vl-240-0     mp4        unknown    
vl-240-1     mp4        unknown    
vl-480       mp4        unknown    
vl-720       mp4        unknown    (best)
[download] Downloading video 2 of 13
[PornHub] ph58161fcb54759: Downloading webpage
[info] Available formats for ph58161fcb54759:
format code  extension  resolution note
240P-400K-0  mp4        240p        400k 
240P-400K-1  mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 3 of 13
[PornHub] ph579fa1895ec03: Downloading webpage
[info] Available formats for ph579fa1895ec03:
format code  extension  resolution note
240P-400K-0  mp4        240p        400k 
240P-400K-1  mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 4 of 13
[PornHub] ph5713bfcf0269a: Downloading webpage
[info] Available formats for ph5713bfcf0269a:
format code  extension  resolution note
240P-400K-0  mp4        240p        400k 
240P-400K-1  mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 5 of 13
[PornHub] ph57732ec66899f: Downloading webpage
[info] Available formats for ph57732ec66899f:
format code  extension  resolution note
240P-400K-0  mp4        240p        400k 
240P-400K-1  mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 6 of 13
[PornHub] 1287925133: Downloading webpage
[info] Available formats for 1287925133:
format code  extension  resolution note
vl-240-0     mp4        unknown    
vl-240-1     mp4        unknown    
480P-600K    mp4        480p        600k  (best)
[download] Downloading video 7 of 13
[PornHub] 1068766821: Downloading webpage
[info] Available formats for 1068766821:
format code  extension  resolution note
vl-240P-0    mp4        unknown    
vl-240P-1    mp4        unknown    
vl-480P      mp4        unknown    
vl-720P      mp4        unknown    (best)
[download] Downloading video 8 of 13
[PornHub] 872479661: Downloading webpage
[info] Available formats for 872479661:
format code  extension  resolution note
vl-240P-0    mp4        unknown    
vl-240P-1    mp4        unknown    
vl-480P      mp4        unknown    (best)
[download] Downloading video 9 of 13
[PornHub] 803489516: Downloading webpage
[info] Available formats for 803489516:
format code  extension  resolution note
vl-240P-0    mp4        unknown    
vl-240P-1    mp4        unknown    
vl-480P      mp4        unknown    (best)
[download] Downloading video 10 of 13
[PornHub] ph580554a0ee995: Downloading webpage
[info] Available formats for ph580554a0ee995:
format code  extension  resolution note
240P-400K    mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 11 of 13
[PornHub] ph580544e395dcc: Downloading webpage
[info] Available formats for ph580544e395dcc:
format code  extension  resolution note
240P-400K    mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 12 of 13
[PornHub] ph57ec57bb588ec: Downloading webpage
[info] Available formats for ph57ec57bb588ec:
format code  extension  resolution note
240P-400K-0  mp4        240p        400k 
240P-400K-1  mp4        240p        400k 
480P-600K    mp4        480p        600k 
720P-1500K   mp4        720p       1500k  (best)
[download] Downloading video 13 of 13
[PornHub] ph57ec4eb14ae4b: Downloading webpage
ERROR: Unable to extract title; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/media/home/username/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 694, in extract_info
    ie_result = ie.extract(url)
  File "/media/home/username/bin/youtube-dl/youtube_dl/extractor/common.py", line 357, in extract
    return self._real_extract(url)
  File "/media/home/username/bin/youtube-dl/youtube_dl/extractor/pornhub.py", line 134, in _real_extract
    webpage, 'title', group='title')
  File "/media/home/username/bin/youtube-dl/youtube_dl/extractor/common.py", line 660, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract title; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

That's weird, is there a way to get a more verbose output? Looking at the video title it doesn't seem like it has any weird characters in the name or anything. It could be that the site is timing me out too. Sometimes it starts to download the videos and then it errors out due to a timeout. You don't seem to suffer from that though although you only downloaded the playlist metadata, it doesn't seem you downloaded any of the videos per se.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Dec 16, 2016

is there a way to get a more verbose output?

youtube-dl -vF "http://www.pornhub.com/users/bgfpclips/videos/favorites" --dump-pages
@bestwaytodesign
Copy link
Author

@bestwaytodesign bestwaytodesign commented Dec 16, 2016

@yan12125 that only printed mostly gibberish, wasn't exactly what I was after. Oh well

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Dec 16, 2016

If you know what's the cause of RegexNotFoundError, you'll know why I ask you for the output with --dump-pages

@bestwaytodesign
Copy link
Author

@bestwaytodesign bestwaytodesign commented Dec 16, 2016

dump.txt
Oh I see. I attached the dump. Hopefully it helps.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Dec 17, 2016

In the latest log everything is downloaded fine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.