Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Pandora.tv] Extractor giving error "ValueError: time data '/x/y/xyz' does not match format '%Y%m%d'" #12846

Closed
khckhc opened this issue Apr 26, 2017 · 7 comments
Labels
bug

Comments

@khckhc
Copy link

@khckhc khckhc commented Apr 26, 2017

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like that [x])
  • Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.04.26. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2017.04.26

Before submitting an issue make sure you have:

  • At least skimmed through README and most notably FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue


If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

C:\>youtube-dl.exe --version
2017.04.26

C:\>youtube-dl.exe "http://channel.pandora.tv/channel/video.ptv?ch_userid=gogoucc&prgid=54721744" -v -f best
[debug] System config: []
[debug] User config: ['-f', '22/35/34/18', '-o', '%(title)s.%(ext)s', '--console-title']
[debug] Custom config: []
[debug] Command-line args: ['http://channel.pandora.tv/channel/video.ptv?ch_userid=gogoucc&prgid=54721744', '-v', '-f', 'best']
[debug] Encodings: locale cp950, fs mbcs, out cp950, pref cp950
[debug] youtube-dl version 2017.04.26
[debug] Python version 3.4.4 - Windows-XP-5.1.2600-SP3
[debug] exe versions: none
[debug] Proxy map: {}
[pandora.tv] 54721744: Downloading JSON metadata
[pandora.tv] 54721744: Downloading JSON metadata
[pandora.tv] 54721744: Downloading JSON metadata
[pandora.tv] 54721744: Downloading JSON metadata
[pandora.tv] 54721744: Downloading JSON metadata
Traceback (most recent call last):
  File "__main__.py", line 19, in <module>
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\__init__.py", line 465, in main
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\__init__.py", line 455, in _real_main
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\YoutubeDL.py", line 1896, in download
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\YoutubeDL.py", line 771, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\YoutubeDL.py", line 825, in process_ie_result
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\YoutubeDL.py", line 1539, in process_video_result
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\YoutubeDL.py", line 1610, in process_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\YoutubeDL.py", line 704, in _match_entry
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\utils.py", line 1288, in __contains__
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpi8nwgst1\build\youtube_dl\utils.py", line 1251, in date_from_str
  File "C:\Python\Python34\lib\_strptime.py", line 507, in _strptime_datetime
  File "C:\Python\Python34\lib\_strptime.py", line 344, in _strptime
ValueError: time data '/k/e/kei' does not match format '%Y%m%d'

Description of your issue, suggested solution and other information

The extractor for pandora.tv recently started not working for some videos. Checking pandoratv.py I can see it is using the URL "http://m.pandora.tv/?c=view&m=viewJsonApi&ch_userid=%s&prgid=%s" to download metadata for the video in JSON format.

After downloading the JSON data, it will extract the video upload date from the field ['data']['rows']['vod_play_info']['result']['fid'], expecting first 8 characters of the field to be in "YYYYMMDD" format. However recently pandora.tv appear to have changed format of this field for some videos.

Working: http://m.pandora.tv/?c=view&m=viewJsonApi&ch_userid=gogoucc&prgid=54730858
fid: 201704262212063498v3c5ric2jmvu

Not working: http://m.pandora.tv/?c=view&m=viewJsonApi&ch_userid=gogoucc&prgid=54721744
fid: /k/e/keigoo/30/20170423175419187mshwpggef3c83

I believe the extractor code need to be modified to work with both formats.

@isaacm
Copy link

@isaacm isaacm commented May 9, 2017

Something like this should work:

--- a/youtube_dl/extractor/pandoratv.py	2017-05-09 13:09:52.000000000 -0500 
+++ b/youtube_dl/extractor/pandoratv.py	2017-05-09 13:11:09.000000000 -0500
@@ -1,5 +1,6 @@
 # coding: utf-8
 from __future__ import unicode_literals
+import re

 from .common import InfoExtractor
 from ..compat import (
@@ -80,13 +81,21 @@
             })
         self._sort_formats(formats)

+        fid = info.get('fid')
+        upload_date = None
+        if fid is not None:
+            matches = re.findall(r'(\d+)', fid)
+            if matches is not None:
+                upload_date = max(matches, key=len)
+                upload_date = upload_date[:8]
+
         return {
             'id': video_id,
             'title': info['subject'],
             'description': info.get('body'),
             'thumbnail': info.get('thumbnail') or info.get('poster'),
             'duration': float_or_none(info.get('runtime'), 1000) or parse_duration(info.get('time')),
-            'upload_date': info['fid'][:8] if isinstance(info.get('fid'), compat_str) else None,
+            'upload_date': upload_date,
             'uploader': info.get('nickname'),
             'uploader_id': info.get('upload_userid'),
             'view_count': str_to_int(info.get('hit')),

I can create a PR if collaborators agree it looks fine.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Jun 15, 2017

In the "not working" example (/k/e/keigoo/30/20170423175419187mshwpggef3c83), the date is still available in the last part (20170423175419187mshwpggef3c83), which should be extracted.

@yan12125 yan12125 added the bug label Jun 15, 2017
@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Jun 15, 2017

By the way, http://channel.pandora.tv/channel/video.ptv?ch_userid=gogoucc&prgid=54721744 is working now:

$ python youtube_dl/__main__.py -vF "http://channel.pandora.tv/channel/video.ptv?ch_userid=gogoucc&prgid=54721744"
[debug] System config: []
[debug] User config: ['-v', '--add-metadata', '--xattrs', '-o', '~/tmp/%(title)s-%(id)s.%(ext)s']
[debug] Custom config: []
[debug] Command-line args: ['-vF', 'http://channel.pandora.tv/channel/video.ptv?ch_userid=gogoucc&prgid=54721744']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.06.12
[debug] Git HEAD: ca2703717
[debug] Python version 3.6.1 - Linux-4.11.4-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 3.3.2, ffprobe 3.3.2
[debug] Proxy map: {}
[pandora.tv] 54721744: Downloading JSON metadata
[pandora.tv] 54721744: Downloading JSON metadata
[pandora.tv] 54721744: Downloading JSON metadata
[pandora.tv] 54721744: Downloading JSON metadata
[pandora.tv] 54721744: Downloading JSON metadata
[info] Available formats for 54721744:
format code  extension  resolution note
240p         flv        240p       
336p         flv        336p       
480p         flv        480p       
720p         flv        720p       (best)

Any other example? If none I guess this can be closed.

@khckhc
Copy link
Author

@khckhc khckhc commented Jun 17, 2017

@yan12125 If you only use "-F" to check available formats then its working (I guess because it does not involve parsing upload_date), but if you really perform a download then its still giving the same error.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Jun 17, 2017

You're got it!

@isaacm Are you still working on this?

@isaacm
Copy link

@isaacm isaacm commented Jun 17, 2017

No, I only investigated the problem to figure out the root cause of the error. You can take up the work based on my findings and code snippet above.

Or close the ticket if it's resolved.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Jun 19, 2017

Fixed in cc2ffe5 :)

@yan12125 yan12125 closed this in cc2ffe5 Jun 19, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.