youtube-dl fails to download Facebook video on one server while working fine on the other for the same link

[osimit]

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.10.02. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2016.10.02

Before submitting an issue make sure you have:

• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

$ youtube-dl --write-pages -j -v https://www.facebook.com/BoredPandaArt/videos/570024353184694/?hc_ref=NEWSFEED
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--write-pages', u'-j', u'-v', u'https://www.facebook.com/BoredPandaArt/videos/570024353184694/?hc_ref=NEWSFEED']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.10.02
[debug] Python version 2.7.8 - Linux-3.16.0-31-generic-x86_64-with-Ubuntu-14.10-utopic
[debug] exe versions: ffmpeg 2.5.10-0ubuntu0.15.04.1, ffprobe 2.5.10-0ubuntu0.15.04.1
[debug] Proxy map: {}
ERROR: Cannot parse data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 694, in extract_info
    ie_result = ie.extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 355, in extract
    return self._real_extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/facebook.py", line 352, in _real_extract
    video_id, fatal_if_no_video=True)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/facebook.py", line 276, in _extract_from_url
    raise ExtractorError('Cannot parse data')
ExtractorError: Cannot parse data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description of your issue, suggested solution and other information

First of all, youtube-dl is great piece of software and has worked flawlessly for a long time for me! Good job keeping up with ever changing sites! :)

I run youtube-dl on two different servers (both Ubuntu VPS from same provider) and one of them has recently stopped downloading facebook videos with the explanation above. The other one works nicely and without any errors. I have also used the --write-pages option to dump pages and inspect them, and I can see that dumped pages actually do contain the videos.

youtube-dl on both server is invoked exactly the same and it works on one, but does not work on other, without any clear explanation. I think it's important to clarify that this only happens to Facebook videos while Youtube still works.

Dumps are also attached (added .html for easier viewing)
dump.zip

Thank you and keep up the good work!

[yas9999]

Hello, i guess it's something related with permission ! like u can't see some videos if you are not joined to the group/page ! it's also the same rules for youtube-dl he download videos that only visible to all public maybe if there is any possibility to use username:password in youtube-dl i guess u will be able to download more videos !

[osimit]

@yaccz I don't think that is the case here as it works on another identical server. Also, as you can see in my dump, page is perfectly visible to youtube-dl (in attachments) so it should be able to parse video, but it is not parsing it. Clearly there is some sort of error here.

I repeat, both machines are identical, one works and the other does not.

[yaccz]

Um, what is happening? How did I get here?

[osimit]

@yaccz My bad, I wanted to reply to @yas9999 but the autocomplete failed me

[yas9999]

@osimit in my case i can download the video are u sure that u have the last version of youtube-dl !
$ youtube-dl --version
2016.10.02

[osimit]

@yas9999 As you can see from my pasted output:

[debug] youtube-dl version 2016.10.02

As I already said, I can download it too on another server, but on this particular one it is not downloading, and I am looking into how to track what might the issue be

[yas9999]

you have the problem as #1269

[osimit]

@yas9999 No, I do not, as you can see from the dump that I attached, Facebook sees the video (both dumps are in the .zip file that I attached ) and is perfectly able to download it on another server using the same configuration and same version. If the video was private, how did you download it, as you said that you were able to download it?

@osimit in my case i can download the video are u sure that u have the last version of youtube-dl !
$ youtube-dl --version
2016.10.02

[yan12125]

I haven't looked into details, but it looks like a slightly different facebook page layout. (different parameters in handleServerJS) Seems a new bug and should be fixed in youtube-dl.

[osimit]

@yan12125 Seems so, any idea why that might happen? Maybe Facebook is gradually rolling some changes? Thought had crossed my mind, but I figured that since both servers are in Germany with same provider, they should see the changes at the same time. Might be wrong idea tho

[yan12125]

Facebooks often generate quite different HTML for the same URL, even on the same IP and the same browser, so it's likely that others can't reproduce the error. Dumped files are definitely help. Tthanks for providing them!

[osimit]

No problem, hopefully this can be resolved quickly

[yan12125]

Thanks for the report. Please check whether 38588ab fixes the problem or not.

[osimit]

Yeah, seems to be ok now. Thanks

When can I expect it to be merged and available via --update ?

$ python -m youtube_dl -j -v "https://www.facebook.com/BoredPandaArt/videos/570024353184694/?hc&_ref=NEWSFEED"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-j', u'-v', u'https://www.facebook.com/BoredPandaArt/videos/570024353184694/?hc&_ref=NEWSFEED']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.10.02
[debug] Git HEAD: 38588ab
[debug] Python version 2.7.8 - Linux-3.16.0-31-generic-x86_64-with-Ubuntu-14.10-utopic
[debug] exe versions: ffmpeg 2.5.10-0ubuntu0.15.04.1, ffprobe 2.5.10-0ubuntu0.15.04.1
[debug] Proxy map: {}
{"display_id": "570024353184694", "extractor": "facebook", "fps": null, "upload_date": "20160930", "requested_subtitles": null, "stretched_ratio": null, "height": 640, "_filename": "Artist uses paint and water drops to create spontaneous dress designs -https -...-570024353184694.mp4", "uploader": "Bored Panda Art", "format_id": "570025539851242v+570025446517918add", "id": "570024353184694", "requested_formats": [{"asr": null, "tbr": 894, "protocol": "https", "language": "eng", "format": "570025539851242v - 640x640 (DASH video)", "url": "https://video.xx.fbcdn.net/v/t42.1790-2/14553927_570025543184575_1795101810659164160_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfdjNfNjQwX2NyZl8yM19tYWluXzMuMF9mcmFnXzJfdmlkZW8ifQ==&oh=805316c171c03600e77ee53a7667c96d&oe=57F7B9E1", "vcodec": "avc1.4d401e", "format_note": "DASH video", "height": 640, "width": 640, "ext": "mp4", "filesize": null, "fps": null, "manifest_url": null, "format_id": "570025539851242v", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "acodec": "none"}, {"asr": 48000, "tbr": 49, "protocol": "https", "language": "eng", "format": "570025446517918add - audio only (DASH audio)", "url": "https://video.xx.fbcdn.net/v/t42.1790-2/14538811_570025449851251_825107988300693504_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfdjNfNDI2X2NyZl8yM19tYWluXzMuMF9mcmFnXzJfYXVkaW8ifQ==&oh=3ef94ae0469226065fcaef141743310c&oe=57F7D364", "vcodec": "none", "format_note": "DASH audio", "height": null, "width": null, "ext": "m4a", "filesize": null, "fps": null, "manifest_url": null, "format_id": "570025446517918add", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "acodec": "mp4a.40.29"}], "playlist": null, "timestamp": 1475258400, "title": "Artist uses paint and water drops to create spontaneous dress designs:\nhttps:...", "extractor_key": "Facebook", "vcodec": "avc1.4d401e", "format": "570025539851242v - 640x640 (DASH video)+570025446517918add - audio only (DASH audio)", "abr": null, "playlist_index": null, "vbr": null, "ext": "mp4", "webpage_url": "https://www.facebook.com/BoredPandaArt/videos/570024353184694/?hc&_ref=NEWSFEED", "formats": [{"asr": 48000, "tbr": 49, "protocol": "https", "language": "eng", "format": "570025446517918a - audio only (DASH audio)", "url": "https://video.xx.fbcdn.net/v/t42.1790-2/14538811_570025449851251_825107988300693504_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfdjNfNDI2X2NyZl8yM19tYWluXzMuMF9mcmFnXzJfYXVkaW8ifQ==&oh=3ef94ae0469226065fcaef141743310c&oe=57F7D364", "vcodec": "none", "format_note": "DASH audio", "height": null, "width": null, "ext": "m4a", "filesize": null, "fps": null, "manifest_url": null, "format_id": "570025446517918a", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "acodec": "mp4a.40.29"}, {"asr": 48000, "tbr": 49, "protocol": "https", "language": "eng", "format": "570025446517918add - audio only (DASH audio)", "url": "https://video.xx.fbcdn.net/v/t42.1790-2/14538811_570025449851251_825107988300693504_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfdjNfNDI2X2NyZl8yM19tYWluXzMuMF9mcmFnXzJfYXVkaW8ifQ==&oh=3ef94ae0469226065fcaef141743310c&oe=57F7D364", "vcodec": "none", "format_note": "DASH audio", "height": null, "width": null, "ext": "m4a", "filesize": null, "fps": null, "manifest_url": null, "format_id": "570025446517918add", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "acodec": "mp4a.40.29"}, {"asr": null, "tbr": 486, "protocol": "https", "language": "eng", "format": "570025443184585vdd - 426x426 (DASH video)", "url": "https://video.xx.fbcdn.net/v/t42.1790-2/14538596_570025453184584_7377907787575066624_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfdjNfNDI2X2NyZl8yM19tYWluXzMuMF9mcmFnXzJfdmlkZW8ifQ==&oh=13785e975bfb5efc46b7bb0a0cb850af&oe=57F7BDD0", "vcodec": "avc1.4d401e", "format_note": "DASH video", "height": 426, "width": 426, "ext": "mp4", "filesize": null, "fps": null, "manifest_url": null, "format_id": "570025443184585vdd", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "acodec": "none"}, {"asr": null, "tbr": 894, "protocol": "https", "language": "eng", "format": "570025539851242v - 640x640 (DASH video)", "url": "https://video.xx.fbcdn.net/v/t42.1790-2/14553927_570025543184575_1795101810659164160_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfdjNfNjQwX2NyZl8yM19tYWluXzMuMF9mcmFnXzJfdmlkZW8ifQ==&oh=805316c171c03600e77ee53a7667c96d&oe=57F7B9E1", "vcodec": "avc1.4d401e", "format_note": "DASH video", "height": 640, "width": 640, "ext": "mp4", "filesize": null, "fps": null, "manifest_url": null, "format_id": "570025539851242v", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "acodec": "none"}, {"protocol": "https", "format": "dash_sd_src - unknown", "url": "https://video.xx.fbcdn.net/v/t42.1790-2/14535645_1763188887267052_3788455063979032576_n.mp4?efg=eyJybHIiOjk2NCwicmxhIjo2MzYsInZlbmNvZGVfdGFnIjoidjNfNDI2X2NyZl8yM19tYWluXzMuMF9zZCJ9&rl=964&vabr=536&oh=ae3c19e0666c69e91e2bd3fba7ef08b5&oe=57F7D31B", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "ext": "mp4", "preference": 0, "format_id": "dash_sd_src"}, {"protocol": "https", "format": "dash_sd_src_no_ratelimit - unknown", "url": "https://video.xx.fbcdn.net/v/t42.1790-2/14535645_1763188887267052_3788455063979032576_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6InYzXzQyNl9jcmZfMjNfbWFpbl8zLjBfc2QifQ%3D%3D&oh=ae3c19e0666c69e91e2bd3fba7ef08b5&oe=57F7D31B", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "ext": "mp4", "preference": 0, "format_id": "dash_sd_src_no_ratelimit"}, {"protocol": "https", "format": "dash_hd_src - unknown", "url": "https://video.xx.fbcdn.net/v/t43.1792-2/14416249_1576693892640259_3492460731280916480_n.mp4?efg=eyJybHIiOjE1MDAsInJsYSI6MTAyNCwidmVuY29kZV90YWciOiJzdmVfaGQifQ%3D%3D&rl=1500&vabr=710&oh=67258fcdb49c043626b86a0c0c51181b&oe=57F7D2DF", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "ext": "mp4", "preference": 5, "format_id": "dash_hd_src"}, {"protocol": "https", "format": "dash_hd_src_no_ratelimit - unknown", "url": "https://video.xx.fbcdn.net/v/t43.1792-2/14416249_1576693892640259_3492460731280916480_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6InN2ZV9oZCJ9&oh=67258fcdb49c043626b86a0c0c51181b&oe=57F7D2DF", "http_headers": {"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7", "Accept-Language": "en-us,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)"}, "ext": "mp4", "preference": 5, "format_id": "dash_hd_src_no_ratelimit"}], "fulltitle": "Artist uses paint and water drops to create spontaneous dress designs:\nhttps:...", "resolution": null, "width": 640, "webpage_url_basename": "570024353184694", "acodec": "mp4a.40.29"}

[yan12125]

FYI: It's already on the master branch and will be included in the next version.