Skip to content

/ youtube-dl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--metadata-from-title regex optional capture groups #13408

Closed
jrieko opened this issue Jun 17, 2017 · 1 comment
Closed

--metadata-from-title regex optional capture groups #13408

jrieko opened this issue Jun 17, 2017 · 1 comment

Comments

@jrieko
Copy link

@jrieko jrieko commented Jun 17, 2017 •
edited

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.06.12. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2017.06.12

Before submitting an issue make sure you have:

  • At least skimmed through README and most notably FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

${redacted}@${redacted} ~/git/youtube-dl/tmp $ ../youtube-dl -v -f 'bestaudio/best' -i --yes-playlist -x --audio-quality 0 --audio-format 'm4a' --embed-thumbnail --add-metadata --metadata-from-title '^(?P<track_number>\d+) - (?P<track>.+?)(?: - (?P<artist>.+?))??(?: - (?P<album>Life Is Strange Complete Soundtrack))?$' --playlist-items 4-5 https://www.youtube.com/playlist?list=PLp3-KQ922RELlU_q8bmxEMtweK_b8jo0j
[debug] System config: [u'--postprocessor-args', u'-strict -2']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-f', u'bestaudio/best', u'-i', u'--yes-playlist', u'-x', u'--audio-quality', u'0', u'--audio-format', u'm4a', u'--embed-thumbnail', u'--add-metadata', u'--metadata-from-title', u'^(?P<track_number>\\d+) - (?P<track>.+?)(?: - (?P<artist>.+?))??(?: - (?P<album>Life Is Strange Complete Soundtrack))?$', u'--playlist-items', u'4-5', u'https://www.youtube.com/playlist?list=PLp3-KQ922RELlU_q8bmxEMtweK_b8jo0j']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.06.12
[debug] Python version 2.7.12 - Linux-4.4.0-77-generic-x86_64-with-LinuxMint-18.1-serena
[debug] exe versions: ffmpeg 2.8.11-0ubuntu0.16.04.1, ffprobe 2.8.11-0ubuntu0.16.04.1
[debug] Proxy map: {'http': 'http://localhost:8118/', 'https': 'http://localhost:8118/', 'no': 'localhost,127.0.0.0/8,::1,192.168.0.0/16'}
[youtube:playlist] PLp3-KQ922RELlU_q8bmxEMtweK_b8jo0j: Downloading webpage
[download] Downloading playlist: Life Is Strange Complete Soundtrack
[youtube:playlist] PLp3-KQ922RELlU_q8bmxEMtweK_b8jo0j: Downloading page #1
[youtube:playlist] playlist Life Is Strange Complete Soundtrack: Downloading 2 videos
[download] Downloading video 1 of 2
[youtube] yWjb1mj9su0: Downloading webpage
[youtube] yWjb1mj9su0: Downloading video info webpage
[youtube] yWjb1mj9su0: Extracting video information
[youtube] {22} signature length 42.43, html5 player vflZ4aCE3
[youtube] yWjb1mj9su0: Downloading player https://www.youtube.com/yts/jsbin/player-vflZ4aCE3/en_US/base.js
[youtube] {43} signature length 42.43, html5 player vflZ4aCE3
[youtube] {18} signature length 42.43, html5 player vflZ4aCE3
[youtube] {36} signature length 42.43, html5 player vflZ4aCE3
[youtube] {17} signature length 42.43, html5 player vflZ4aCE3
[youtube] {137} signature length 42.43, html5 player vflZ4aCE3
[youtube] {136} signature length 42.43, html5 player vflZ4aCE3
[youtube] {135} signature length 42.43, html5 player vflZ4aCE3
[youtube] {134} signature length 42.43, html5 player vflZ4aCE3
[youtube] {133} signature length 42.43, html5 player vflZ4aCE3
[youtube] {160} signature length 42.43, html5 player vflZ4aCE3
[youtube] {140} signature length 42.43, html5 player vflZ4aCE3
[youtube] yWjb1mj9su0: Downloading MPD manifest
[youtube] yWjb1mj9su0: Downloading thumbnail ...
[youtube] yWjb1mj9su0: Writing thumbnail to: 4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.jpg
[debug] Invoking downloader on u'https://r6---sn-q4f7snld.googlevideo.com/videoplayback/id/c968dbd668fdb2ed/itag/140/source/youtube/requiressl/yes/mn/sn-q4f7snld/mm/31/ms/au/ei/OnVEWaH8BYiXD8vCmrgG/mv/m/pl/17/initcwndbps/555000/ratebypass/yes/mime/audio%2Fmp4/otfp/1/gir/yes/clen/4545619/lmt/1478635906019682/dur/286.162/mt/1497658597/signature/30F4804A3D557DA00390E98382998223B4A2179B.27F76B3501CF964757D7078E05E331D793639DDB/beids/%5B9466591%5D/key/dg_yt0/ip/${redacted}/ipbits/0/expire/1497680282/sparams/ip,ipbits,expire,id,itag,source,requiressl,mn,mm,ms,ei,mv,pl,initcwndbps,ratebypass,mime,otfp,gir,clen,lmt,dur/'
[dashsegments] Total fragments: 30
[download] Destination: 4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.m4a
[download] 100% of 4.33MiB in 00:12
[ffmpeg] Correcting container in "4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.m4a"
[debug] ffmpeg command line: ffmpeg -y -i 'file:4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.m4a' -c copy -f mp4 -strict -2 'file:4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.temp.m4a'
[fromtitle] parsed track: To all of you (Original)
[fromtitle] parsed album: Life Is Strange Complete Soundtrack
[fromtitle] parsed track_number: 4
[fromtitle] parsed artist: Syd Matters
[debug] ffmpeg command line: ffprobe -show_streams 'file:4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.m4a'
[ffmpeg] Post-process file 4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.m4a exists, skipping
[ffmpeg] Adding metadata to '4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.m4a'
[debug] ffmpeg command line: ffmpeg -y -i 'file:4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.m4a' -vn -acodec copy -metadata 'comment=This is the original version from the 2005 album "Someday We Will Foresee Obstacles." It includes an extra minute of guitar riffs at the end.

To support the artist, please listen to the track on Spotify: /redirect?q=https%3A%2F%2Fopen.spotify.com%2Ftrack%2F4Y9TDwvffDYIEx0zPaHiWj&redir_token=QgjtTX3UzFhqZz-9Jc0elIqVQQ98MTQ5Nzc0NTA4MUAxNDk3NjU4Njgx

Lyrics
To all of you
American girls, it'"'"'s sad to
imagine a world without you.
American girls, I'"'"'d like to
be part of the world around you,
driving a car by the seaside,
watching the world from the bright side.

To all of you
American girls in the movies.
No one can tell where your heart is.
American girls like dollies
With shiny smiles and plastic bodies.
I wish I had
an American girlfriend.

I cry sometimes, walking around my own place,
Wondering why she cries sometimes,
Talking about her own place
Somewhere around the mountains.
No one could dry her fountain
Till she got tired to complain.
That'"'"'s when I fly
to the wildland,
to your land.

Full playlist: https://www.youtube.com/playlist?list=PLp3-KQ922RELlU_q8bmxEMtweK_b8jo0j

More details: /redirect?q=https%3A%2F%2Fsteamcommunity.com%2Fsharedfiles%2Ffiledetails%2F%3Fid%3D785603956&redir_token=QgjtTX3UzFhqZz-9Jc0elIqVQQ98MTQ5Nzc0NTA4MUAxNDk3NjU4Njgx

I am in no way affiliated with Dontnod, Square Enix, or any of the musicians. This complete soundtrack is fan work only and has not received their blessing.

Please buy the Life Is Strange Limited Edition for the official soundtrack, which includes exclusive instrumental tracks by Jonathan Morali not included in this soundtrack or the game.' -metadata 'album=Life Is Strange Complete Soundtrack' -metadata 'description=This is the original version from the 2005 album "Someday We Will Foresee Obstacles." It includes an extra minute of guitar riffs at the end.

To support the artist, please listen to the track on Spotify: /redirect?q=https%3A%2F%2Fopen.spotify.com%2Ftrack%2F4Y9TDwvffDYIEx0zPaHiWj&redir_token=QgjtTX3UzFhqZz-9Jc0elIqVQQ98MTQ5Nzc0NTA4MUAxNDk3NjU4Njgx

Lyrics
To all of you
American girls, it'"'"'s sad to
imagine a world without you.
American girls, I'"'"'d like to
be part of the world around you,
driving a car by the seaside,
watching the world from the bright side.

To all of you
American girls in the movies.
No one can tell where your heart is.
American girls like dollies
With shiny smiles and plastic bodies.
I wish I had
an American girlfriend.

I cry sometimes, walking around my own place,
Wondering why she cries sometimes,
Talking about her own place
Somewhere around the mountains.
No one could dry her fountain
Till she got tired to complain.
That'"'"'s when I fly
to the wildland,
to your land.

Full playlist: https://www.youtube.com/playlist?list=PLp3-KQ922RELlU_q8bmxEMtweK_b8jo0j

More details: /redirect?q=https%3A%2F%2Fsteamcommunity.com%2Fsharedfiles%2Ffiledetails%2F%3Fid%3D785603956&redir_token=QgjtTX3UzFhqZz-9Jc0elIqVQQ98MTQ5Nzc0NTA4MUAxNDk3NjU4Njgx

I am in no way affiliated with Dontnod, Square Enix, or any of the musicians. This complete soundtrack is fan work only and has not received their blessing.

Please buy the Life Is Strange Limited Edition for the official soundtrack, which includes exclusive instrumental tracks by Jonathan Morali not included in this soundtrack or the game.' -metadata 'title=To all of you (Original)' -metadata track=4 -metadata 'artist=Syd Matters' -metadata date=20161007 -metadata 'purl=https://www.youtube.com/watch?v=yWjb1mj9su0' -strict -2 'file:4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.temp.m4a'
[atomicparsley] Adding thumbnail to "4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.m4a"
[debug] AtomicParsley command line: AtomicParsley '4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.m4a' --artwork '4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.jpg' -o '4 - To all of you (Original) - Syd Matters - Life Is Strange Complete Soundtrack-yWjb1mj9su0.temp.m4a'
[download] Downloading video 2 of 2
[youtube] sgFPoCYE_e0: Downloading webpage
[youtube] sgFPoCYE_e0: Downloading video info webpage
[youtube] sgFPoCYE_e0: Extracting video information
[youtube] sgFPoCYE_e0: Downloading MPD manifest
[youtube] sgFPoCYE_e0: Downloading thumbnail ...
[youtube] sgFPoCYE_e0: Writing thumbnail to: 5 - The Butterfly - Life Is Strange Complete Soundtrack-sgFPoCYE_e0.jpg
[debug] Invoking downloader on u'https://r1---sn-q4f7snse.googlevideo.com/videoplayback/id/b2014fa02604fded/itag/140/source/youtube/requiressl/yes/pl/17/ei/SnVEWZ_9NJn3uQK0uaSADQ/initcwndbps/637500/mn/sn-q4f7snse/mm/31/mv/m/ms/au/ratebypass/yes/mime/audio%2Fmp4/otfp/1/gir/yes/clen/238550/lmt/1478637282383433/dur/14.976/mt/1497658597/key/dg_yt0/signature/45D7F5A0CDDD17F850BE8BA8CB5B2F2D77035CF5.68FD137DD76F142D6A980A84547ED48397B342E4/beids/%5B9466591%5D/ip/${redacted}/ipbits/0/expire/1497680298/sparams/ip,ipbits,expire,id,itag,source,requiressl,pl,ei,initcwndbps,mn,mm,mv,ms,ratebypass,mime,otfp,gir,clen,lmt,dur/'
[dashsegments] Total fragments: 3
[download] Destination: 5 - The Butterfly - Life Is Strange Complete Soundtrack-sgFPoCYE_e0.m4a
[download] 100% of 232.90KiB in 00:00
[ffmpeg] Correcting container in "5 - The Butterfly - Life Is Strange Complete Soundtrack-sgFPoCYE_e0.m4a"
[debug] ffmpeg command line: ffmpeg -y -i 'file:5 - The Butterfly - Life Is Strange Complete Soundtrack-sgFPoCYE_e0.m4a' -c copy -f mp4 -strict -2 'file:5 - The Butterfly - Life Is Strange Complete Soundtrack-sgFPoCYE_e0.temp.m4a'
[fromtitle] parsed track: The Butterfly
[fromtitle] parsed album: Life Is Strange Complete Soundtrack
[fromtitle] parsed track_number: 5
ERROR: coercing to Unicode: need string or buffer, NoneType found
Traceback (most recent call last):
  File "../youtube-dl/youtube_dl/YoutubeDL.py", line 773, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "../youtube-dl/youtube_dl/YoutubeDL.py", line 827, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "../youtube-dl/youtube_dl/YoutubeDL.py", line 1562, in process_video_result
    self.process_info(new_info)
  File "../youtube-dl/youtube_dl/YoutubeDL.py", line 1900, in process_info
    self.post_process(filename, info_dict)
  File "../youtube-dl/youtube_dl/YoutubeDL.py", line 1965, in post_process
    files_to_delete, info = pp.run(info)
  File "../youtube-dl/youtube_dl/postprocessor/metadatafromtitle.py", line 43, in run
    self._downloader.to_screen('[fromtitle] parsed ' + attribute + ': ' + value)
TypeError: coercing to Unicode: need string or buffer, NoneType found

[download] Finished downloading playlist: Life Is Strange Complete Soundtrack

Description of your issue, suggested solution and other information

I'm downloading a playlist and using a regex with --metadata-from-title. Some of the titles omit some metadata items so I made the respective capture groups optional. However, when youtube-dl gets to such a title where a capture group result is empty, the match object's value is None, which trips up later code expecting a string.

The verbose output above demonstrates two items in the playlist, the first matches all captures groups and succeeds; the second fails as described.

I did verify the regex matches titles in the playlist as expected.
@jrieko
Copy link
Author

@jrieko jrieko commented Jun 17, 2017

I think just skipping the empty match will do the trick (so the attribute is omitted from the metadata dict). I'll whip up a PR right quick unless anybody disagrees.
@dstftw dstftw closed this in be80986 Jun 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
@jrieko
You can’t perform that action at this time.