Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search by Filter #11457

Open
linglung opened this issue Dec 16, 2016 · 12 comments
Open

Search by Filter #11457

linglung opened this issue Dec 16, 2016 · 12 comments
Labels

Comments

@linglung
Copy link

@linglung linglung commented Dec 16, 2016

How to search any videos based on FEATURES filter such as 4K, Creative commons or 3D? i can see youtube-dl has ability to sort by upload date, min view etc but i can't find for features. e.g : ytsearch:Chritsmas videos :creative commons

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Dec 16, 2016

youtube-dl's search function is based on YouTube website's. Do you know how to search for specific features on the website?

@linglung
Copy link
Author

@linglung linglung commented Dec 16, 2016

@yan12125 Absolutely i knew that. In this case, it seems that my question is unclear. What i mean here is : I couldn't find what parameters should be used to search for Youtube videos using youtube-dl based on filtering by FEATURES (4K, HD, creative commons, etc).

Then the result of search queries are videos filtered by FEATURES only. For example: i search 10 videos of "christmas song" - filtered by creative commons license, so the result would be 10 videos with CC license only. I assume the parameter is like : ytsearch10:"christmas song: creative commons".

As i'm not a native English speaker, so i attached a screenshot. Hope you'll get it.

features yt

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Dec 17, 2016

Got it. Sounds a useful feature

@yan12125 yan12125 added the request label Dec 17, 2016
@vordep vordep mentioned this issue Dec 19, 2016
3 of 6 tasks complete
@linglung
Copy link
Author

@linglung linglung commented Dec 19, 2016

@vordep Great. Works as expected. Just missing a litle feature for determining of videos count result, where ytsearch or gvsearch already did these job perfectly.
I'm waiting for complete task progress of this request.
Thank you all for making a piece of great software like this

@vordep
Copy link
Contributor

@vordep vordep commented Dec 19, 2016

@linglung Sadly it was not accepted, anyway i believed you were talking about being able to search for more than one videos right, like this?

ytsearch10:"christmas "

@linglung
Copy link
Author

@linglung linglung commented Dec 20, 2016

@vordep Yes, right. More than one video but with certain amount of videos, such as 5, 10 or 12 videos. I know this has been done with ytsearch or gvsearch. i.e : ytsearch5, ytsearch10, ytsearch12. It's only missing of filtering for "Features" as i asked before.

While your latest class update is almost perfect. It has the ability to filter results based on "Features" - especially creative commons filter - as exactly what i need. Sadly, It couldn't limit in a certain amount of videos, or to determine the number of videos that will be downloaded . It just downloaded all the videos on page one.

No problem if it's not possible or not accepted at this moment, the last update was satisfied enough for me.

@vordep
Copy link
Contributor

@vordep vordep commented Dec 20, 2016

@linglung It should work without a hitch since ytsearch works, and my classes are extensions of that, try this

ytsearchcc10:"christmas trees"

this is the output i got, it downloaded the 10 videos i requested.

[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'ytsearchcc10:christmas trees', u'-v', u'--simulate']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.12.18
[debug] Git HEAD: d414423
[debug] Python version 2.7.12 - Linux-4.4.0-53-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: ffmpeg 2.8.10-0ubuntu0.16.04.1, ffprobe 2.8.10-0ubuntu0.16.04.1
[debug] Proxy map: {}
[youtube:search:creative_commons] query "christmas trees": Downloading page 1
[download] Downloading playlist: christmas trees
[youtube:search:creative_commons] playlist christmas trees: Collected 10 video ids (downloading 10 of them)
[download] Downloading video 1 of 10
[youtube] SRpg5RNUXD0: Downloading webpage
[youtube] SRpg5RNUXD0: Downloading video info webpage
[youtube] SRpg5RNUXD0: Extracting video information
[youtube] SRpg5RNUXD0: Downloading MPD manifest
[download] Downloading video 2 of 10
[youtube] CeCTD9B1KM4: Downloading webpage
[youtube] CeCTD9B1KM4: Downloading video info webpage
[youtube] CeCTD9B1KM4: Extracting video information
[youtube] CeCTD9B1KM4: Downloading MPD manifest
[download] Downloading video 3 of 10
[youtube] 0zwY7Jam23s: Downloading webpage
[youtube] 0zwY7Jam23s: Downloading video info webpage
[youtube] 0zwY7Jam23s: Extracting video information
[youtube] 0zwY7Jam23s: Downloading MPD manifest
[download] Downloading video 4 of 10
[youtube] qDt1w4mzI5o: Downloading webpage
[youtube] qDt1w4mzI5o: Downloading video info webpage
[youtube] qDt1w4mzI5o: Extracting video information
[youtube] qDt1w4mzI5o: Downloading MPD manifest
[download] Downloading video 5 of 10
[youtube] YHXZmSvUVvc: Downloading webpage
[youtube] YHXZmSvUVvc: Downloading video info webpage
[youtube] YHXZmSvUVvc: Extracting video information
[youtube] YHXZmSvUVvc: Downloading MPD manifest
[download] Downloading video 6 of 10
[youtube] RXjPX_1kJRE: Downloading webpage
[youtube] RXjPX_1kJRE: Downloading video info webpage
[youtube] RXjPX_1kJRE: Extracting video information
[youtube] RXjPX_1kJRE: Downloading MPD manifest
[download] Downloading video 7 of 10
[youtube] Z1RNP-z32ac: Downloading webpage
[youtube] Z1RNP-z32ac: Downloading video info webpage
[youtube] Z1RNP-z32ac: Extracting video information
[youtube] Z1RNP-z32ac: Downloading MPD manifest
[download] Downloading video 8 of 10
[youtube] 5GdvxnPC0Qw: Downloading webpage
[youtube] 5GdvxnPC0Qw: Downloading video info webpage
[youtube] 5GdvxnPC0Qw: Extracting video information
[youtube] 5GdvxnPC0Qw: Downloading MPD manifest
[download] Downloading video 9 of 10
[youtube] kK1Q2vLGd_A: Downloading webpage
[youtube] kK1Q2vLGd_A: Downloading video info webpage
[youtube] kK1Q2vLGd_A: Extracting video information
[youtube] kK1Q2vLGd_A: Downloading MPD manifest
[download] Downloading video 10 of 10
[youtube] b5yRVJGtBOw: Downloading webpage
[youtube] b5yRVJGtBOw: Downloading video info webpage
[youtube] b5yRVJGtBOw: Extracting video information
[youtube] b5yRVJGtBOw: Downloading MPD manifest
[download] Finished downloading playlist: christmas trees

@linglung
Copy link
Author

@linglung linglung commented Dec 21, 2016

@vordep I tried it and not yet working properly. My details are:

  • Youtube-dl latest update (exe version) from here https://github.com/rg3/youtube-dl/releases/
  • Windows 10 x64.
  • Python version 3.5.2 (cmd windows log said, it's 3.5.2 by python -V command, while youtube-dl log said it's 3.4.4).

was it due to Python version? Here is the log :

$ youtube-dl -v --simulate ytsearchcc10:"christmas trees"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', '--simulate', 'ytsearchcc10:christmas trees']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2016.12.20
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-82095-g5867234, ffprobe N-82095-g5867234
[debug] Proxy map: {}
[generic] ytsearchcc10:christmas trees: Requesting header
WARNING: Could not send HEAD request to ytsearchcc10:christmas trees: <urlopen error unknown url type: ytsearchcc10>
[generic] ytsearchcc10:christmas trees: Downloading webpage
ERROR: Unable to download webpage: <urlopen error unknown url type: ytsearchcc10> (caused by URLError('unknown url type: ytsearchcc10',))
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmph8curlv3\build\youtube_dl\extractor\common.py", line 405, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmph8curlv3\build\youtube_dl\YoutubeDL.py", line 2001, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 464, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 487, in _open
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 1253, in unknown_open
@linglung
Copy link
Author

@linglung linglung commented Dec 21, 2016

A normal log without cc command.

$ youtube-dl -v --simulate ytsearch10:"christmas trees"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', '--simulate', 'ytsearch10:christmas trees']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2016.12.20
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-82095-g5867234, ffprobe N-82095-g5867234
[debug] Proxy map: {}
[youtube:search] query "christmas trees": Downloading page 1
[download] Downloading playlist: christmas trees
[youtube:search] playlist christmas trees: Collected 10 video ids (downloading 10 of them)
[download] Downloading video 1 of 10
[youtube] ux3STlov7Vw: Downloading webpage
[youtube] ux3STlov7Vw: Downloading video info webpage
[youtube] ux3STlov7Vw: Extracting video information
[download] Downloading video 2 of 10
[youtube] CrasjXq3ce0: Downloading webpage
[youtube] CrasjXq3ce0: Downloading video info webpage
[youtube] CrasjXq3ce0: Extracting video information
[youtube] {22} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {43} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {18} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {36} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {17} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {137} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {248} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {299} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {303} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {136} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {247} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {298} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {302} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {135} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {244} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {134} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {243} signature length 39.40, html5 player en_US-vfla6wgHS
[youtube] {133} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {242} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {160} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {278} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {140} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {171} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {249} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {250} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {251} signature length 41.40, html5 player en_US-vfla6wgHS
[download] Downloading video 3 of 10
[youtube] Ddqk6QZCII8: Downloading webpage
[youtube] Ddqk6QZCII8: Downloading video info webpage
[youtube] Ddqk6QZCII8: Extracting video information
[youtube] Ddqk6QZCII8: Downloading MPD manifest
[download] Downloading video 4 of 10
[youtube] AfGEZy7xk5k: Downloading webpage
[youtube] AfGEZy7xk5k: Downloading video info webpage
[youtube] AfGEZy7xk5k: Extracting video information
[youtube] AfGEZy7xk5k: Downloading MPD manifest
[download] Downloading video 5 of 10
[youtube] ae3Avjr542U: Downloading webpage
[youtube] ae3Avjr542U: Downloading video info webpage
[youtube] ae3Avjr542U: Extracting video information
[youtube] ae3Avjr542U: Downloading MPD manifest
[download] Downloading video 6 of 10
[youtube] Q6pJLXdTB7A: Downloading webpage
[youtube] Q6pJLXdTB7A: Downloading video info webpage
[youtube] Q6pJLXdTB7A: Extracting video information
[youtube] Q6pJLXdTB7A: Downloading MPD manifest
[download] Downloading video 7 of 10
[youtube] uHLF1_jtlao: Downloading webpage
[youtube] uHLF1_jtlao: Downloading video info webpage
[youtube] uHLF1_jtlao: Extracting video information
[youtube] uHLF1_jtlao: Downloading MPD manifest
[download] Downloading video 8 of 10
[youtube] tF7eymHD29k: Downloading webpage
[youtube] tF7eymHD29k: Downloading video info webpage
[youtube] tF7eymHD29k: Extracting video information
[youtube] tF7eymHD29k: Downloading MPD manifest
[download] Downloading video 9 of 10
[youtube] ZgTpZ7YlHKc: Downloading webpage
[youtube] ZgTpZ7YlHKc: Downloading video info webpage
[youtube] ZgTpZ7YlHKc: Extracting video information
[youtube] {22} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {43} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {18} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {36} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {17} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {137} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {248} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {299} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {303} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {136} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {247} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {298} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {302} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {135} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {244} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {134} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {243} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {133} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {242} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {160} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {278} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {140} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {171} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {249} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {250} signature length 41.40, html5 player en_US-vfla6wgHS
[youtube] {251} signature length 41.40, html5 player en_US-vfla6wgHS
[download] Downloading video 10 of 10
[youtube] 3xgzsbmbzZ8: Downloading webpage
[youtube] 3xgzsbmbzZ8: Downloading video info webpage
[youtube] 3xgzsbmbzZ8: Extracting video information
[youtube] 3xgzsbmbzZ8: Downloading MPD manifest
[download] Finished downloading playlist: christmas trees
@vordep
Copy link
Contributor

@vordep vordep commented Dec 21, 2016

i believe the problem relies on the fact that my pull request was not merged with youtube-dl master.
it makes sense since youtube dl master doesnt have that implementation, try to clone my fork altough its behind the youtubel-master it should not be a problem in any python version.
or you could try what dstftw said in the pull request

@linglung
Copy link
Author

@linglung linglung commented Dec 21, 2016

@vordep Awesome, finally it works.
Just FYI, it's my bad. I am misunderstanding and misinterpreting with your first comment reference before. At the first trying, actually i already tried what dstftw said to search by URL . I read and tried it so fast, i though that your classes update is same as what dstftw recommend there, that's why i said to you that it's almost perfect just missing video numbers request.

After reading your latest comment, i just realized my mistake. I figure it out of your fork. Totally it works great. checked the result, and it's correctly result as CC videos.
Thanks a million @vordep .

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', '--simulate', 'ytsearchcc10:christmas song']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2016.12.18
[debug] Python version 3.5.2 - Windows-10-10.0.14393-SP0
[debug] exe versions: ffmpeg 2.8.4, ffprobe N-82889-g54931fd
[debug] Proxy map: {}
[youtube:search:creative_commons] query "christmas song": Downloading page 1
[download] Downloading playlist: christmas song
[youtube:search:creative_commons] playlist christmas song: Collected 10 video ids (downloading 10 of them)
[download] Downloading video 1 of 10
[youtube] OunUPdb9IaU: Downloading webpage
[youtube] OunUPdb9IaU: Downloading video info webpage
[youtube] OunUPdb9IaU: Extracting video information
[youtube] OunUPdb9IaU: Downloading MPD manifest
[download] Downloading video 2 of 10
[youtube] yxdw_CwsUEo: Downloading webpage
[youtube] yxdw_CwsUEo: Downloading video info webpage
[youtube] yxdw_CwsUEo: Extracting video information
[youtube] yxdw_CwsUEo: Downloading MPD manifest
[download] Downloading video 3 of 10
[youtube] HwcSMSBPoEQ: Downloading webpage
[youtube] HwcSMSBPoEQ: Downloading video info webpage
[youtube] HwcSMSBPoEQ: Extracting video information
[youtube] HwcSMSBPoEQ: Downloading MPD manifest
[download] Downloading video 4 of 10
[youtube] OIWAXaA_V2U: Downloading webpage
[youtube] OIWAXaA_V2U: Downloading video info webpage
[youtube] OIWAXaA_V2U: Extracting video information
[youtube] OIWAXaA_V2U: Downloading MPD manifest
[download] Downloading video 5 of 10
[youtube] fX8gd9WNFE8: Downloading webpage
[youtube] fX8gd9WNFE8: Downloading video info webpage
[youtube] fX8gd9WNFE8: Extracting video information
[youtube] fX8gd9WNFE8: Downloading MPD manifest
[download] Downloading video 6 of 10
[youtube] MupHMWf5r9E: Downloading webpage
[youtube] MupHMWf5r9E: Downloading video info webpage
[youtube] MupHMWf5r9E: Extracting video information
[youtube] MupHMWf5r9E: Downloading MPD manifest
[download] Downloading video 7 of 10
[youtube] 4yRTK1dia_E: Downloading webpage
[youtube] 4yRTK1dia_E: Downloading video info webpage
[youtube] 4yRTK1dia_E: Extracting video information
[youtube] 4yRTK1dia_E: Downloading MPD manifest
[download] Downloading video 8 of 10
[youtube] lxyA3O7rQpQ: Downloading webpage
[youtube] lxyA3O7rQpQ: Downloading video info webpage
[youtube] lxyA3O7rQpQ: Extracting video information
[youtube] lxyA3O7rQpQ: Downloading MPD manifest
[download] Downloading video 9 of 10
[youtube] epgAOB4CV-Q: Downloading webpage
[youtube] epgAOB4CV-Q: Downloading video info webpage
[youtube] epgAOB4CV-Q: Extracting video information
[youtube] epgAOB4CV-Q: Downloading MPD manifest
[download] Downloading video 10 of 10
[youtube] v6rhm6DeCs0: Downloading webpage
[youtube] v6rhm6DeCs0: Downloading video info webpage
[youtube] v6rhm6DeCs0: Extracting video information
[youtube] v6rhm6DeCs0: Downloading MPD manifest
[download] Finished downloading playlist: christmas song
@luccyrobert1212
Copy link

@luccyrobert1212 luccyrobert1212 commented Feb 4, 2019

Sir when I use features option like creative commons the download speed limit decrease in kb. Any suggestion because when i clone from https://github.com/rg3/youtube-dl.git and try it will download the video in per MB but with your clone it will decrease in KB. But the features option is not available in original repo.
youtube-dl -o "/home/ubuntu/vid/%(title)s.%(ext)s" -f 136+140 -v ytsearchcc2:"Ubuntu Tutorial" --max-filesize 40m
With same command on original repo download speed is 8mb/sec and with your its 23kb/sec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.