add support for xattrs #5498

glensc · 2015-04-22T08:08:04Z

would be cool if youtube-dl supported user.xdg.origin.url and user.xdg.referrer.url Extended Attributes

most browsers already do that, for example chrome on linux:

$ getfattr -dR ~/Downloads
getfattr: Removing leading '/' from absolute path names
# file: home/glen/Downloads/jiralinks-2014-11-12.zip
user.xdg.origin.url="https://codeload.github.com/meteotest/jiralinks/zip/2014-11-12"
user.xdg.referrer.url="https://www.dokuwiki.org/plugin:jiralinks"

i have patch for wget as well

yan12125 · 2015-04-22T10:01:08Z

With --xattrs you can have the user.xdg.referrer.url attribute set. Check xattrpp.py for other attributes youtube-dl sets. Anyway, it may be useful if user.xdg.origin.url is added, though some websites, for example youtube, use different links each time, so such a feature is not meaningful.

glensc · 2015-04-22T10:08:11Z

ah, sorry did not know about --xattrs. how about getting it turned on by default? (maybe depending on platform or xattr py module availability)

ytdl-org/youtube-dl#5498

yan12125 · 2015-04-22T10:13:04Z

If you want to enable this option for all downloads, try the configuration file

glensc · 2015-04-22T10:40:56Z

thanks

glensc · 2015-04-22T12:35:17Z

not sure what's the syntax for config?

$ youtube-dl ...
[download] 100% of 31.96MiB in 00:28
[metadata] Writing metadata to file's xattrs
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 9, in <module>
    load_entry_point('youtube-dl==2015.2.28', 'console_scripts', 'youtube-dl')()
  File "/usr/share/python2.7/site-packages/youtube_dl/__init__.py", line 397, in main
  File "/usr/share/python2.7/site-packages/youtube_dl/__init__.py", line 387, in _real_main
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1442, in download
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 654, in extract_info
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 700, in process_ie_result
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1143, in process_video_result
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1425, in process_info
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1480, in post_process
  File "/usr/share/python2.7/site-packages/youtube_dl/postprocessor/xattrpp.py", line 106, in run
  File "/usr/share/python2.7/site-packages/youtube_dl/postprocessor/xattrpp.py", line 40, in write_xattr
TypeError: argument must be string or int

$ cat /etc/youtube-dl.conf 
# default system wide configuration
# https://github.com/rg3/youtube-dl#configuration

# Write metadata to the video file's xattrs (using dublin core and xdg standards)
--xattrs
$

yan12125 · 2015-04-22T13:24:16Z

Your config file is OK. The problem seems coming from python-pyxattr. What is the version of python-pyxattr you use? Note that the latest youtube-dl is version 2015.04.17, and you should always use the latest version. If you still encounter errors with the latest version, post the full verbose log with the -v option.

glensc · 2015-04-22T13:34:44Z

python-pyxattr-0.4.0-4.x86_64, same trace with youtube-dl==2015.2.28

glensc · 2015-04-22T13:36:36Z

the full -v isn't much interesting than already posted

[~] ➔ youtube-dl https://www.youtube.com/watch?v=4BMCTGZ2Rb8 -v
[debug] System config: ['--xattrs']
[debug] User config: []
[debug] Command-line args: ['https://www.youtube.com/watch?v=4BMCTGZ2Rb8', '-v']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.02.28
[debug] Python version 2.7.7 - Linux-3.14.36-1-x86_64-Intel-R-_Core-TM-2_Quad_CPU____Q9400__@_2.66GHz-with-pld-3.0-PLD
[debug] exe versions: ffmpeg 2.5.4, ffprobe 2.5.4, rtmpdump 2.3
[debug] Proxy map: {}
[youtube] 4BMCTGZ2Rb8: Downloading webpage
[youtube] 4BMCTGZ2Rb8: Extracting video information
[youtube] 4BMCTGZ2Rb8: Downloading DASH manifest
[debug] Invoking downloader on u'https://r10---sn-5go7ln7z.googlevideo.com/videoplayback?fexp=900720%2C907263%2C916640%2C921418%2C934954%2C936109%2C938028%2C9406010%2C9406501%2C9407014%2C9407115%2C9407671%2C9408107%2C9408268%2C9408347%2C9408349%2C9408707%2C9408930%2C9409248%2C9409252%2C947233%2C948124%2C948701%2C948703%2C951703%2C952612%2C952637%2C957201&upn=7OECesNs20w&ip=195.222.9.197&ratebypass=yes&expire=1429731309&pl=26&ipbits=0&initcwndbps=3426250&key=yt5&requiressl=yes&itag=18&sparams=dur%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Cmime%2Cmm%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&ms=au&mv=m&mt=1429709545&source=youtube&sver=3&dur=387.935&id=o-AMHY4Ff_vX6NIgXDoqrfDYpGNsb-ERKmLyYgd_3keYeb&mm=31&signature=101214540AB3A4E7B6F890616E6D19E722DB6460.640FF21022DEEE9BFA6AE90D172234097DD641E6&mime=video%2Fmp4'
[download] Destination: EI aktsiisimängudele! 26.04.2015 Toompeale!-4BMCTGZ2Rb8.mp4
[download] 100% of 31.96MiB in 00:37
[metadata] Writing metadata to file's xattrs
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 9, in <module>
    load_entry_point('youtube-dl==2015.2.28', 'console_scripts', 'youtube-dl')()
  File "/usr/share/python2.7/site-packages/youtube_dl/__init__.py", line 397, in main
  File "/usr/share/python2.7/site-packages/youtube_dl/__init__.py", line 387, in _real_main
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1442, in download
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 654, in extract_info
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 700, in process_ie_result
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1143, in process_video_result
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1425, in process_info
  File "/usr/share/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1480, in post_process
  File "/usr/share/python2.7/site-packages/youtube_dl/postprocessor/xattrpp.py", line 106, in run
  File "/usr/share/python2.7/site-packages/youtube_dl/postprocessor/xattrpp.py", line 40, in write_xattr
TypeError: argument must be string or int
[~] ➔

yan12125 · 2015-04-22T13:59:31Z

With youtube-dl 2015.02.28, pyxattr 0.5.0 works for me while 0.4.0 fails with the same TypeError. Please consider update your pyxattr. Still, it's strongly encouraged to use the latest youtube-dl.

ytdl-org/youtube-dl#5498 (comment)

jaimeMF · 2015-04-22T21:23:24Z

I'm closing the issue since it works with the last version of pyxattr.

For more information, see #5498 and changes to convertObj() in iustin/pyxattr@cc84e46

Closes #9054 There are two xattr packages in Python, pyxattr [1] and xattr [2]. They have different APIs. In old days pyxattr supports Linux only and xattr supports Linux, Mac, FreeBSD and Solaris, and pyxattr supports Linux only. Recently pyxattr adds support for Mac OS X. [3] An old version of [2] is shipped with Mac OS X. However, some Linux distributions have pyxattr only, for example PLD-Linux [4] and old Arch Linux. [5] As a result, supporting both is the way to go. [1] https://github.com/iustin/pyxattr [2] https://github.com/xattr/xattr [3] iustin/pyxattr#9 [4] #5498 [5] https://git.archlinux.org/svntogit/community.git/commit/?id=427c4c76401e386d865ccddea4fbfdc74df80492 https://git.archlinux.org/svntogit/community.git/commit/?id=59b40da7b69622a6761d364a8b07909e9cccaa56 python-xattr is added on 2016/06/29 while pyxattr is there for more than 6 years

glensc added a commit to pld-linux/youtube-dl that referenced this issue Apr 22, 2015

require python-pyxattr

d70b68a

ytdl-org/youtube-dl#5498

glensc closed this Apr 22, 2015

glensc reopened this Apr 22, 2015

glensc added a commit to pld-linux/youtube-dl that referenced this issue Apr 22, 2015

require pyxattr 0.5.0

cd3037f

ytdl-org/youtube-dl#5498 (comment)

jaimeMF closed this Apr 22, 2015

yan12125 added a commit that referenced this issue Apr 23, 2015

[xattr] Add version detection for python-pyxattr

Loading status checks…

8c88261

For more information, see #5498 and changes to convertObj() in iustin/pyxattr@cc84e46

yan12125 mentioned this issue Mar 12, 2016

Would like to add extended attributes to saved file #8668

Closed

ytdl-org / youtube-dl

add support for xattrs #5498

add support for xattrs #5498

glensc commented Apr 22, 2015

yan12125 commented Apr 22, 2015

glensc commented Apr 22, 2015

yan12125 commented Apr 22, 2015

glensc commented Apr 22, 2015

glensc commented Apr 22, 2015

yan12125 commented Apr 22, 2015

glensc commented Apr 22, 2015

glensc commented Apr 22, 2015

yan12125 commented Apr 22, 2015

jaimeMF commented Apr 22, 2015

ytdl-org / youtube-dl

Join GitHub today

GitHub is where the world builds software

add support for xattrs #5498

add support for xattrs #5498

Comments

glensc commented Apr 22, 2015

yan12125 commented Apr 22, 2015

glensc commented Apr 22, 2015

yan12125 commented Apr 22, 2015

glensc commented Apr 22, 2015

glensc commented Apr 22, 2015

yan12125 commented Apr 22, 2015

glensc commented Apr 22, 2015

glensc commented Apr 22, 2015

yan12125 commented Apr 22, 2015

jaimeMF commented Apr 22, 2015

Essential cookies

Always active

Analytics cookies