Skip to content

/ youtube-dl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dailymotion download fails #20961

Closed
Maxou44 opened this issue May 5, 2019 · 11 comments
Closed

Dailymotion download fails #20961

Maxou44 opened this issue May 5, 2019 · 11 comments
Labels
cant-reproduce

Comments

@Maxou44
Copy link

@Maxou44 Maxou44 commented May 5, 2019 •
edited

Checklist

  • [ X] I'm reporting a broken site support issue
  • [ X] I've verified that I'm running youtube-dl version 2019.04.30
  • [ X] I've checked that all provided URLs are alive and playable in a browser
  • [ X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • [ X] I've searched the bugtracker for similar bug reports including closed ones
  • [ X] I've read bugs section in FAQ

Verbose log

root# youtube-dl -v https://www.dailymotion.com/video/x77etcs
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.dailymotion.com/video/x77etcs']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.04.30
[debug] Python version 2.7.12 (CPython) - Linux-4.4.0-130-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: ffmpeg 4.0.1-0york0, ffprobe 4.0.1-0york0
[debug] Proxy map: {}
[dailymotion] x77etcs: Downloading webpage
[dailymotion] x77etcs: Downloading metadata JSON
[dailymotion] x77etcs: Downloading m3u8 information
[dailymotion] x77etcs: Downloading m3u8 information
[dailymotion] x77etcs: Downloading m3u8 information
[dailymotion] x77etcs: Downloading m3u8 information
[dailymotion] x77etcs: Downloading m3u8 information
[dailymotion] x77etcs: Downloading m3u8 information
[dailymotion] x77etcs: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://www.dailymotion.com/cdn/H264-1920x1080/video/x77etcs.mp4?sec=TLkyXUVASQiUwwiDAP3V2LCMl9jcAamoNbjegvoK00S2ysmYIO8nz9F337m8qfaD'
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1915, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1854, in dl
    return fd.download(name, info)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 364, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 341, in real_download
    establish_connection()
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 109, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2227, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 467, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 654, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

Description

I can't download any Dailymotion video, I think the Dailymotion API/routes changed...

Urls to test:
https://www.dailymotion.com/video/x77exhx
https://www.dailymotion.com/video/x7706ll
@dstftw
Copy link
Collaborator

@dstftw dstftw commented May 5, 2019 

> py -3.7 .\youtube_dl\__main__.py https://www.dailymotion.com/video/x7706ll
[dailymotion] x7706ll: Downloading webpage
[dailymotion] x7706ll: Downloading metadata JSON
[dailymotion] x7706ll: Downloading m3u8 information
[dailymotion] x7706ll: Downloading m3u8 information
[dailymotion] x7706ll: Downloading m3u8 information
[dailymotion] x7706ll: Downloading m3u8 information
[dailymotion] x7706ll: Downloading m3u8 information
[dailymotion] x7706ll: Downloading m3u8 information
[dailymotion] x7706ll: Downloading m3u8 information
[download] Destination: Les niches fiscales des entreprises dans le collimateur de Bercy-x7706ll.mp4
[download]   1.2% of 136.93MiB at 600.15KiB/s ETA 03:50
ERROR: Interrupted by user
@dstftw dstftw closed this May 5, 2019
@Maxou44
Copy link
Author

@Maxou44 Maxou44 commented May 8, 2019

I have additionnal information, it looks like Dailymotion CDN banned ranges of server IP address, the solution is to use a proxy 🙂
@RealDeviil
Copy link

@RealDeviil RealDeviil commented May 13, 2019

I am having the same issue:

root@waw:/tmp/youtube-dl# youtube-dl --verbose --write-pages http://www.dailymotion.com/video/x31j7ik_chapman-stick_music
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'--write-pages', u'http://www.dailymotion.com/video/x31j7ik_chapman-stick_music']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.05.11
[debug] Python version 2.7.9 (CPython) - Linux-4.14.63-xxxx-std-ipv6-64-x86_64-with-Debian-8
[debug] exe versions: ffmpeg N-87327-g18821e3ba1-static, ffprobe N-87327-g18821e3ba1-static
[debug] Proxy map: {}
[dailymotion] x31j7ik: Downloading webpage
[dailymotion] Saving request to x31j7ik_https_-_www.dailymotion.com_video_x31j7ik.dump
[dailymotion] x31j7ik: Downloading metadata JSON
[dailymotion] Saving request to x31j7ik_https_-_www.dailymotion.com_player_metadata_video_x31j7ikembedder=http%3A%2F%2Fwww.dailymotion.com%2Fvideo%2Fx31j7ik_chapman-stick_music_GK_PV5_NEON=1_integration=inline.dump
[dailymotion] x31j7ik: Downloading m3u8 information
[dailymotion] Saving request to x31j7ik_https_-_www.dailymotion.com_cdn_manifest_video_x31j7ik.m3u8sec=x2w05vBlNMAHkSG7f3Wf0Xb1ypGf9AvFxnqAqIOm7zGZ7EDWct9oiYqs8KFrcfsn_include=144.dump
[dailymotion] x31j7ik: Downloading m3u8 information
[dailymotion] Saving request to x31j7ik_https_-_www.dailymotion.com_cdn_manifest_video_x31j7ik.m3u8sec=x2w05vBlNMAHkSG7f3Wf0Xb1ypGf9AvFxnqAqIOm7zGZ7EDWct9oiYqs8KFrcfsn.dump
[dailymotion] x31j7ik: Downloading m3u8 information
[dailymotion] Saving request to x31j7ik_https_-_www.dailymotion.com_cdn_manifest_video_x31j7ik.m3u8sec=x2w05vBlNMAHkSG7f3Wf0Xb1ypGf9AvFxnqAqIOm7zGZ7EDWct9oiYqs8KFrcfsn_include=380.dump
[dailymotion] x31j7ik: Downloading m3u8 information
[dailymotion] Saving request to x31j7ik_https_-_www.dailymotion.com_cdn_manifest_video_x31j7ik.m3u8sec=x2w05vBlNMAHkSG7f3Wf0Xb1ypGf9AvFxnqAqIOm7zGZ7EDWct9oiYqs8KFrcfsn_include=240.dump
[dailymotion] x31j7ik: Downloading m3u8 information
[dailymotion] Saving request to x31j7ik_https_-_www.dailymotion.com_cdn_manifest_video_x31j7ik.m3u8sec=x2w05vBlNMAHkSG7f3Wf0Xb1ypGf9AvFxnqAqIOm7zGZ7EDWct9oiYqs8KFrcfsn_include=720.dump
[dailymotion] x31j7ik: Downloading m3u8 information
[dailymotion] Saving request to x31j7ik_https_-_www.dailymotion.com_cdn_manifest_video_x31j7ik.m3u8sec=x2w05vBlNMAHkSG7f3Wf0Xb1ypGf9AvFxnqAqIOm7zGZ7EDWct9oiYqs8KFrcfsn_include=480.dump
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://www.dailymotion.com/cdn/H264-1280x720/video/x31j7ik.mp4?sec=mGzGq-Rt8hcoLMUUrI3ha7x2cNV_04RgYEAVOKXM7Dediqkp14TIIkU9Swx2S6mQ'
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1915, in process_info
    success = dl(filename, info_dict)
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1854, in dl
    return fd.download(name, info)
  File "/usr/bin/youtube-dl/youtube_dl/downloader/common.py", line 364, in download
    return self.real_download(filename, info_dict)
  File "/usr/bin/youtube-dl/youtube_dl/downloader/http.py", line 341, in real_download
    establish_connection()
  File "/usr/bin/youtube-dl/youtube_dl/downloader/http.py", line 109, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2227, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 469, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 656, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

I'm running youtube-dl version 2019.05.11 on OVH dedicated servers in Poland and Canada.

I've tested these URLs:
https://www.dailymotion.com/video/x1ces6k
https://www.dailymotion.com/video/x31j7ik

I think this issue deserves more attention and testing. Even if Sergey cannot reproduce the issue, it is very possible that the issue affects a significant percentage of youtube-dl users.

Maxime, can you tell me more about your solution? What kind of proxy did you use to circumvent this issue? In what country is your proxy located?

Thank you.
@Maxou44
Copy link
Author

@Maxou44 Maxou44 commented May 13, 2019

My server is located in France in an Online datacenter, I use a Surfshark proxy to solve this issue
@flackune
Copy link

@flackune flackune commented May 13, 2019

I've got a Server in an OVH datacenter and another also in an Online datacenter. I opened a ticket to Dailymotion's support to find out more. I will keep you informed as soon as I have more details :)
@RealDeviil
Copy link

@RealDeviil RealDeviil commented May 15, 2019

I guess if I have to use a proxy workaround, I will. But sometimes if you dig around in the source code, you can find a code workaround there as well. I would prefer a code workaround.

Using my own "test" code, I attempted to download videos from different sources, and pass different cookies and request headers, but nothing worked for me. :/
@DeHackEd
Copy link

@DeHackEd DeHackEd commented May 19, 2019

I've experienced the same thing. Changing IP addresses isn't necessary though - downloading again using wget (use -v to make youtube-dl print the download URL it tries to fetch) has been successful for me.
@mcmarkj
Copy link

@mcmarkj mcmarkj commented Sep 15, 2019

Can this be reopened? I'm having ongoing issues with this.
@IRGC
Copy link

@IRGC IRGC commented Nov 6, 2019

@flackune any update bruh?
@Karif
Copy link

@Karif Karif commented Nov 11, 2019 •
edited

Problem continues...

youtube -v "https://www.dailymotion.com/video/x6uoseg"

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.dailymotion.com/video/x6uoseg']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.11.05
[debug] Python version 2.7.15+ (CPython) - Linux-4.15.0-66-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6
[debug] Proxy map: {}
[dailymotion] x6uoseg: Downloading webpage
[dailymotion] x6uoseg: Downloading metadata JSON
[dailymotion] x6uoseg: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/bin/youtube/youtube_dl/YoutubeDL.py", line 796, in extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube/youtube_dl/extractor/common.py", line 530, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/youtube/youtube_dl/extractor/dailymotion.py", line 260, in _real_extract
self._sort_formats(formats)
File "/usr/local/bin/youtube/youtube_dl/extractor/common.py", line 1327, in _sort_formats
raise ExtractorError('No video formats found')
ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
@remitamine
Copy link
Collaborator

@remitamine remitamine commented Nov 11, 2019 

youtube-dl https://www.dailymotion.com/video/x6uoseg
[dailymotion] x6uoseg: Downloading webpage
[dailymotion] x6uoseg: Downloading metadata JSON
[dailymotion] x6uoseg: Downloading m3u8 information
[download] Destination: Tele1 TV - Canlı Yayın-x6uoseg.mp4
ffmpeg version n4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9.2.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-nvdec --enable-nvenc --enable-omx --enable-shared --enable-version3
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-VERSION:3')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:09.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:12.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:15.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:18.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:21.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:24.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:27.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:30.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:33.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:36.000+0100')
[hls @ 0x55b82e8fb340] Opening 'https://stream-05.dc3.dailymotion.com/sec(oGEyQzojvNQy_SUQu81YX_Y007_YwMWBJdzZruuf6M0)/dm/3/x6uoseg/s/stream-3-1573431808.ts' for reading
[hls @ 0x55b82e8fb340] Opening 'https://stream-05.dc3.dailymotion.com/sec(oGEyQzojvNQy_SUQu81YX_Y007_YwMWBJdzZruuf6M0)/dm/3/x6uoseg/s/stream-3-1573431811.ts' for reading
Input #0, hls, from 'https://stream-05.dc3.dailymotion.com/sec(oGEyQzojvNQy_SUQu81YX_Y007_YwMWBJdzZruuf6M0)/dm/3/x6uoseg/s/live-3.m3u8':
  Duration: N/A, start: 1620.896322, bitrate: N/A
  Program 0 
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
Output #0, mp4, to 'file:Tele1 TV - Canlı Yayın-x6uoseg.mp4.part':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 25 fps, 25 tbr, 90k tbn, 90k tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=   41 fps=0.0 q=-1.0 size=     256kB time=00:00:01.69 bitrate=1237.4kbits/frame=   52 fps= 47 q=-1.0 size=     512kB time=00:00:02.08 bitrate=2007.2kbits/[hls @ 0x55b82e8fb340] Skip ('#EXT-X-VERSION:3')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:12.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:15.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:18.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:21.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:24.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:27.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:30.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:33.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:36.000+0100')
[hls @ 0x55b82e8fb340] Skip ('#EXT-X-PROGRAM-DATE-TIME:2019-11-11T01:23:39.000+0100')
[https @ 0x55b82ef39e00] Opening 'https://stream-05.dc3.dailymotion.com/sec(oGEyQzojvNQy_SUQu81YX_Y007_YwMWBJdzZruuf6M0)/dm/3/x6uoseg/s/stream-3-1573431814.ts' for reading
frame=   74 fps= 32 q=-1.0 size=     512kB time=00:00:02.99 bitrate=1400.4kbits
@ytdl-org ytdl-org locked and limited conversation to collaborators Nov 11, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
cant-reproduce
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants
@DeHackEd @dstftw @RealDeviil @mcmarkj @Maxou44 @remitamine @Karif @flackune @IRGC
You can’t perform that action at this time.