Skip to content

/ youtube-dl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fox.com] unable to get season_number and episode_number #12219

Closed
jaylweb opened this issue Feb 22, 2017 · 5 comments
Closed

[fox.com] unable to get season_number and episode_number #12219

jaylweb opened this issue Feb 22, 2017 · 5 comments

Comments

@jaylweb
Copy link

@jaylweb jaylweb commented Feb 22, 2017 •
edited by dstftw

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like that [x])
  • Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.02.21. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2017.02.21

Before submitting an issue make sure you have:

  • At least skimmed through README and most notably FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ap-mso', 'DTV', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '-v', '--get-filename', '-o', '%(season)s s%(season_number)se%(episode_number)s - %(title)s.%(ext)s', 'http://www.fox.com/watch/796751939630/7684301568']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.02.17
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-82092-g89ec4ad, ffprobe N-82092-g89ec4ad, rtmpdump 2.4
[debug] Proxy map: {}
NA sNAeNA - Chris Has Got a Date, Date, Date, Date, Date.mp4

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information

Unable to get season, season_number, and episode_number. Output results in NA for each field. I am able to grab the title. The links I provided clearly show the season number and episode number on the web page. I think the info is contained in the following script example from the last link I provided above.

<script type="application/ld+json">{"@context":"http:\/\/schema.org","@id":"801550915931","@type":"TVEpisode","name":"Chris Has Got a Date, Date, Date, Date, Date","description":"Taylor Swift agrees to go to a homecoming dance with Chris after he asks her via online video.","url":"http:\/\/www.fox.com\/watch\/796751939630\/7684301568","image":"http:\/\/www.fox.com\/sites\/default\/files\/video-image-thumb\/eacx02_376_03a_0152_hires2_1280x720_801543747530.jpg","episodeNumber":"5","partOfSeason":{"@type":"TVSeason","seasonNumber":"15"},"partOfSeries":{"@type":"TVSeries","name":"Family Guy","sameAs":"http:\/\/www.imdb.com\/title\/tt0182576\/","url":"http:\/\/www.fox.com\/family-guy"},"releasedEvent":{"@type":"PublicationEvent","startDate":"2016-11-06","location":{"@type":"Country","name":"US"}},"potentialAction":[{"@type":"WatchAction","target":["http:\/\/www.fox.com\/watch\/796751939630\/7684301568?cmpid=xtra:-FOX-Goog_Watch","android-app:\/\/com.fox.now\/foxapp\/video_fullepisode\/freewheel\/796751939630?cmpid=xtra:-FOX-Goog_Watch","ios-app:\/\/571096102\/foxapp\/video_fullepisode\/freewheel\/796751939630?cmpid=xtra:-FOX-Goog_Watch"],"expectsAcceptanceOf":[{"@type":"Offer","availabilityStarts":"1969-12-31T19:00:00-05:00","availabilityEnds":"2016-11-14T03:01:00-05:00","category":"externalSubscription","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]},{"@type":"Offer","availabilityStarts":"2016-11-14T03:01:00-05:00","availabilityEnds":"2017-01-09T02:59:00-05:00","category":"free","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]},{"@type":"Offer","availabilityStarts":"2017-01-09T02:59:00-05:00","availabilityEnds":"2017-06-29T02:59:00-04:00","category":"externalSubscription","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]}]},{"@type":"WatchAction","target":[{"@type":"EntryPoint","urlTemplate":"http:\/\/www.fox.com\/watch\/796751939630\/7684301568?cmpid=xtra:-FOX-Goog_Watch","actionPlatform":"http:\/\/schema.org\/DesktopWebPlatform","inLanguage":"en"},{"@type":"EntryPoint","urlTemplate":"android-app:\/\/com.fox.now\/foxapp\/video_fullepisode\/freewheel\/796751939630?extcmp=xtra_FOXNOW_Goog_Watch","actionPlatform":"http:\/\/schema.org\/AndroidPlatform","inLanguage":"en"},{"@type":"EntryPoint","urlTemplate":"http:\/\/www.fox.com\/watch\/796751939630\/7684301568?extcmp=xtra_FOXNOW_Goog_Watch","actionPlatform":"http:\/\/schema.org\/IOSPlatform","inLanguage":"en"}],"expectsAcceptanceOf":[{"@type":"Offer","availabilityStarts":"1969-12-31T19:00:00-05:00","availabilityEnds":"2016-11-14T03:01:00-05:00","category":"externalSubscription","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]},{"@type":"Offer","availabilityStarts":"2016-11-14T03:01:00-05:00","availabilityEnds":"2017-01-09T02:59:00-05:00","category":"free","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]},{"@type":"Offer","availabilityStarts":"2017-01-09T02:59:00-05:00","availabilityEnds":"2017-06-29T02:59:00-04:00","category":"externalSubscription","eligibleRegion":[{"@type":"Country","name":"US"},{"@type":"Country","name":"AS"},{"@type":"Country","name":"GU"},{"@type":"Country","name":"PR"},{"@type":"Country","name":"VI"},{"@type":"Country","name":"UM"}]}]}],"duration":"PT21M36S"}</script><meta name="description" content="FULL EPISODE - &quot;Chris Has Got a Date, Date, Date, Date, Date&quot; - Taylor Swift agrees to go to a homecoming dance with Chris after he asks her via online video."/>

This issue only occurs on Fox. I do not have this issue with other major networks.
@gkoelln
Copy link
Contributor

@gkoelln gkoelln commented Feb 22, 2017

Here's my fix:

# coding: utf-8
from __future__ import unicode_literals

from .adobepass import AdobePassIE
from ..utils import (
    int_or_none,
    smuggle_url,
    update_url_query,
)


class FOXIE(AdobePassIE):
    _VALID_URL = r'https?://(?:www\.)?fox\.com/watch/(?P<id>[0-9]+)'
    _TEST = {
        'url': 'http://www.fox.com/watch/255180355939/7684182528',
        'md5': 'ebd296fcc41dd4b19f8115d8461a3165',
        'info_dict': {
            'id': '255180355939',
            'ext': 'mp4',
            'title': 'Official Trailer: Gotham',
            'description': 'Tracing the rise of the great DC Comics Super-Villains and vigilantes, Gotham reveals an entirely new chapter that has never been told.',
            'duration': 129,
            'timestamp': 1400020798,
            'upload_date': '20140513',
            'uploader': 'NEWA-FNG-FOXCOM',
        },
        'add_ie': ['ThePlatform'],
    }

    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)

        video_metadata = self._search_json_ld(webpage, video_id, fatal=False)
        settings = self._parse_json(self._search_regex(
            r'jQuery\.extend\(Drupal\.settings\s*,\s*({.+?})\);',
            webpage, 'drupal settings'), video_id)
        fox_pdk_player = settings['fox_pdk_player']
        release_url = fox_pdk_player['release_url']
        query = {
            'mbr': 'true',
            'switch': 'http'
        }
        if fox_pdk_player.get('access') == 'locked':
            ap_p = settings['foxAdobePassProvider']
            rating = ap_p.get('videoRating')
            if rating == 'n/a':
                rating = None
            resource = self._get_mvpd_resource('fbc-fox', None, ap_p['videoGUID'], rating)
            query['auth'] = self._extract_mvpd_auth(url, video_id, 'fbc-fox', resource)

        return {
            '_type': 'url_transparent',
            'ie_key': 'ThePlatform',
            'url': smuggle_url(update_url_query(release_url, query), {'force_smil_url': True}),
            'id': video_id,
            'series': video_metadata.get('series'),
            'season_number': int_or_none(video_metadata.get('season_number')),
            'episode_number': int_or_none(video_metadata.get('episode_number')),
            'episode': video_metadata.get('episode'),
        }
@jaylweb
Copy link
Author

@jaylweb jaylweb commented Feb 22, 2017 •
edited

Here is the latest verbose output with version 2017.02.22. I have two computers and one was a few days out of date, although both had the same results.

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ap-mso', 'DTV', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '-v', '--get-filename', '-o', '%(season)s s%(season_number)se%(episode_number)s - %(title)s.%(ext)s', 'http://www.fox.com/watch/796751939630/7684301568']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.02.22
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-82092-g89ec4ad, ffprobe N-82092-g89ec4ad, rtmpdump 2.4
[debug] Proxy map: {}
NA sNAeNA - Chris Has Got a Date, Date, Date, Date, Date.mp4

Am I able to test the above fix proposed by @gkoelln?

I installed Python 3.4.4, downloaded the current Master branch code, updated fox.py with gkoelln's suggested code, tried running python -m YoutubDl.py and get a SystemError about the Parent module '' not loaded. I am not sure where to go from there. I am trying though.
@jaylweb
Copy link
Author

@jaylweb jaylweb commented Feb 23, 2017

Ok, I figured out how to test the code and it was successful!! Here is my output.

C:\Python34\Scripts>python -m youtube_dl --ap-mso DTV --ap-username 'PRIVATE' --ap-password 'PRIVATE" -v --get-filename -o "%(series)s s%(season_number)se%(episode_number)s - %(title)s.%(ext)s" http://www.fox.com/watch/796751939630/7684301568

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ap-mso', 'DTV', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '-v', '--get-filename', '-o', '%(series)s s%(season_number)se%(episode_number)s - %(title)s.%(ext)s', 'http://www.fox.com/watch/796751939630/7684301568']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.02.22
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-82092-g89ec4ad, ffprobe N-82092-g89ec4ad, rtmpdump 2.4
[debug] Proxy map: {}
Family Guy s15e5 - Chris Has Got a Date, Date, Date, Date, Date.mp4

How do we get this code submitted for review? I am new to all of this.
@gkoelln
Copy link
Contributor

@gkoelln gkoelln commented Feb 23, 2017

Well, either the developers can make my proposed changes above, or I can make those changes on my fork and do a pull request. Though, I haven't had much success with any of my pull requests being merged so far.
@jaylweb jaylweb changed the title fox.com unable to get season_number and episode_number [fox.com] unable to get season_number and episode_number Mar 7, 2017
@jaylweb
Copy link
Author

@jaylweb jaylweb commented Mar 7, 2017

@dstftw, what do you think about this?
@gkoelln gkoelln mentioned this issue Mar 7, 2017
Merged
3 of 8 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

[Fox] Add metadata extraction
2 participants
@jaylweb @gkoelln
You can’t perform that action at this time.