stops working on Apple Trailers

[beew]

Example http://trailers.apple.com/trailers/fox/kungfupanda3/

youtube-dl --verbose http://trailers.apple.com/trailers/fox/kungfupanda3/
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'http://trailers.apple.com/trailers/fox/kungfupanda3/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.02.01
[debug] Python version 2.7.10 - Linux-4.4.1-040401-generic-x86_64-with-Ubuntu-15.10-wily
[debug] exe versions: avconv 2.8.3-1, avprobe 2.8.3-1, ffmpeg 2.8.3-1, ffprobe 2.8.3-1, rtmpdump 2.4
[debug] Proxy map: {}
[appletrailers] kungfupanda3: Downloading XML
[appletrailers] kungfupanda3-clp-kaiarrives: Downloading settings json
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 352, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 1903, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

youtube-dl version 2016.02.01

Thanks.

[tedmx]

Hello beew, can you confirm whether you were able to successfully download "Clip: Kai Arrives" in previous builds?

It's not just YDL breaking

That first failing clip is a relatively fresh one, posted on 2016-01-22 (that date is from Playlist file: http://trailers.apple.com/trailers/fox/kungfupanda3/includes/playlists/itunes.inc#).

YDL's failing part is one that fetches metadata JSON for a particular video.

Fetching of JSON fails for "Clip: Kai Arrives" (you can try that in browser):

http://trailers.apple.com/trailers/fox/kungfupanda3/includes/settings/kungfupanda3-clp-kaiarrives.json

but it works, for example, for an older trailer from the same set, the trailer named "Trailer 2":

http://trailers.apple.com/trailers/fox/kungfupanda3/includes/settings/kungfupanda3-tlr2.json

Similar case

Random freshly updated playlist out there.

User-friendly page:
http://trailers.apple.com/trailers/paramount/zoolander2/

Playlist URL:
http://trailers.apple.com/trailers/paramount/zoolander2/includes/playlists/itunes.inc#

New trailer JSON is missing:
http://trailers.apple.com/trailers/paramount/zoolander2/includes/settings/zoolander2-usca-clip-kiss.json

Older trailer JSON is here:
http://trailers.apple.com/trailers/paramount/zoolander2/includes/settings/zoolander2-usca-tlr1.json

Problems

So there are two sides.

• Actions from Apple's side:
  ⋅⋅* Either Apple has globally changed the way JSON files are provided. For all videos uploaded since particular unknown date, JSON files are placed somewhere else.
  ⋅⋅* In other case, Apple uploads JSON file for a clip to usual location, but only when the clip has been posted long enough.
• Problem of youtube-dl:
  ⋅⋅* For Apple trailers lists, even if all videos are available for download (links are available from Playlist .../includes/playlists/itunes.inc), the whole batch process fails if JSON for the first trailer is unavailable.

Proposal

It will be great to make JSON acquiring an optional step for Apple trailers download. If for a particular video JSON file is absent, youtube-dl should fall back to data from .../includes/playlists/itunes.inc

[beew]

Hi,
Yes, it just stopped working. It worked before, but I don't know exactly when it stopped. I visited the Apple sites just occasionally to test mpv with the ytdl hook.

[tedmx]

Hey beew! The only way to fix the issue now is to fix the code. If I'll be able to submit a patch, I'll come back here and let you know.

[remitamine]

it will be fixed in the next version.