Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Average_rating not behaving correctly in match_filter #18734

Open
tukruic opened this issue Jan 4, 2019 · 2 comments
Open

Average_rating not behaving correctly in match_filter #18734

tukruic opened this issue Jan 4, 2019 · 2 comments

Comments

@tukruic
Copy link

@tukruic tukruic commented Jan 4, 2019

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like this: [x])
  • Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2019.01.02. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2019.01.02

Before submitting an issue make sure you have:

  • At least skimmed through the README, most notably the FAQ and BUGS sections
  • [ x] Searched the bugtracker for similar issues including closed ones
  • Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

krzysztofs-macbook-pro:~ kk$ youtube-dl -v --match-filter "average_rating > 4.8" "ytsearch10:ai"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--match-filter', 'average_rating > 4.8', 'ytsearch10:ai']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.01.02
[debug] Python version 3.7.1 (CPython) - Darwin-17.7.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[youtube:search] query "ai": Downloading page 1
[download] Downloading playlist: ai
[youtube:search] playlist ai: Collected 10 video ids (downloading 10 of them)
[download] Downloading video 1 of 10
[youtube] BlPFE7R8psk: Downloading webpage
[youtube] BlPFE7R8psk: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r7---sn-u2oxu-f5fk.googlevideo.com/videoplayback?id=o-AAKqTHFlLlMVpDJKZKHxmX4fsL8igFrhreJZiGN9pwCQ&mn=sn-u2oxu-f5fk%2Csn-u2oxu-f5fk&c=WEB&ipbits=0&mm=31%2C29&initcwndbps=830000&shardbypass=yes&ip=79.184.191.139&ms=au%2Crdu&mv=m&dur=754.439&mt=1546564780&expire=1546586551&hcs=yes&requiressl=yes&lmt=1546301468088071&key=yt6&txp=5532432&pl=24&mime=video%2Fmp4&source=youtube&signature=C6FC95D35A72076D82EF898AF7A1FB99919EF002.1DBF031BA9FF5C26430E160B7D2DCFBC8195641A&fvip=8&ratebypass=yes&sparams=dur%2Cei%2Chcs%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Cshardbypass%2Csource%2Cexpire&itag=22&ei=V7UuXKu1Is_wgQfY-rz4Dg'
[download] Destination: AI in 2019-BlPFE7R8psk.mp4
[download] 100% of 193.41MiB in 01:17
[download] Downloading video 2 of 10
[youtube] ajGgd9Ld-Wc: Downloading webpage
[youtube] ajGgd9Ld-Wc: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r2---sn-u2oxu-fhpe.googlevideo.com/videoplayback?source=youtube&ratebypass=yes&mv=m&dur=889.997&mt=1546564881&lmt=1537507347119271&itag=22&ms=au%2Conr&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&key=yt6&ipbits=0&mm=31%2C26&mn=sn-u2oxu-fhpe%2Csn-5hnekn7k&ei=prUuXJz7HIXIgAf4zrfYBg&expire=1546586630&c=WEB&id=o-AMkbfujCf-4sBA-LsWMt3Nr5qeUKJQ6STGuR3wVo3t57&fvip=1&initcwndbps=930000&ip=79.184.191.139&requiressl=yes&mime=video%2Fmp4&signature=2A5AAA482669C061C91443F1B4A99C73BDF244A7.888F5B426F4FE747FC047953A4C10E01BD809A87&pl=24'
[download] Destination: How AI can save our humanity _ Kai-Fu Lee-ajGgd9Ld-Wc.mp4
[download] 100% of 46.73MiB in 00:01
[download] Downloading video 3 of 10
[youtube] Pls_q2aQzHg: Downloading webpage
[youtube] Pls_q2aQzHg: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r6---sn-u2oxu-f5fk.googlevideo.com/videoplayback?mime=video%2Fmp4&ip=79.184.191.139&key=yt6&itag=22&ipbits=0&txp=5531432&ei=rLUuXMnbLYi9gAellJCoBA&lmt=1538778780005228&signature=9043115AECBD7929121BD3E449C9727BED1C6F6E.212FE4190E4424F94FE692402525785C9BDE1BE4&id=o-AORoROtsYT5X0gH0DdVghc5F4cCo267mrGjH7piHb7Dt&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&initcwndbps=891250&mm=31%2C26&source=youtube&mn=sn-u2oxu-f5fk%2Csn-5hnedn7e&ratebypass=yes&c=WEB&expire=1546586636&fvip=6&pl=24&dur=1222.019&requiressl=yes&mt=1546564881&mv=m&ms=au%2Conr'
[download] Destination: Artificial Intelligence - Mankind's Last Invention-Pls_q2aQzHg.mp4
[download] 100% of 115.46MiB in 00:03
[download] Downloading video 4 of 10
[youtube] _0gUXxZ-ExE: Downloading webpage
[youtube] _0gUXxZ-ExE: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r2---sn-u2oxu-fhpe.googlevideo.com/videoplayback?initcwndbps=930000&ipbits=0&mt=1546564881&requiressl=yes&mm=31%2C26&ei=srUuXMKILtrB-gbvxJHQCA&fvip=2&signature=03F9E2443E325995349B5EFFB6F957E424427047.C2D2C2A8332ADFEBE0D31EB367AFB8DFA2BBFD24&c=WEB&lmt=1545068744140441&mime=video%2Fmp4&ratebypass=yes&id=o-ADhx1sB6j5CfmUTSkmmE7YK8pPbcRkZtKpOQ-jvYPE6q&expire=1546586642&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&mn=sn-u2oxu-fhpe%2Csn-5hnednlk&txp=5535432&ms=au%2Conr&source=youtube&mv=m&dur=960.818&itag=22&pl=24&ip=79.184.191.139&key=yt6'
[download] Destination: Global race for AI superiority-_0gUXxZ-ExE.mp4
[download] 100% of 93.18MiB in 00:02
[download] Downloading video 5 of 10
[youtube] YNLC0wJSHxI: Downloading webpage
[youtube] YNLC0wJSHxI: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r6---sn-u2oxu-f5fd.googlevideo.com/videoplayback?mv=m&mt=1546564973&ms=au%2Conr&source=youtube&pl=24&mn=sn-u2oxu-f5fd%2Csn-5hnekn7s&mm=31%2C26&dur=352.060&requiressl=yes&id=o-AIje6kkM29PtH3r5jm5EGnSScnRZLKqWlALeM7MdesA7&expire=1546586648&ipbits=0&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2cms%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&initcwndbps=963750&signature=BEA060042BE8301A48CA42B1B58CE3C731950A41.E099C792434487B79DFB8219EF783C166511FD32&lmt=1516092900639810&ip=79.184.191.139&pcm2cms=yes&key=yt6&itag=22&mime=video%2Fmp4&fvip=5&ratebypass=yes&c=WEB&ei=t7UuXK71PIip1gLMn5-ABg'
[download] Destination: AI Codes its Own ‘AI Child’  - Artificial Intelligence breakthrough!-YNLC0wJSHxI.mp4
[download] 100% of 39.41MiB in 00:05
[download] Downloading video 6 of 10
[youtube] uNO7QwGZKic: Downloading webpage
[youtube] uNO7QwGZKic: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r2---sn-u2oxu-f5fe6.googlevideo.com/videoplayback?requiressl=yes&ipbits=0&signature=9BC316787D02BD542513DD3F232113E9AE68AD82.071360A3CCA2949AB830318AE8D73A5436EB360E&pl=24&mime=video%2Fmp4&fvip=2&lmt=1540580178397287&ip=79.184.191.139&expire=1546586659&c=WEB&ratebypass=yes&id=o-AERbPXRYsbFSSVhQRWst2CaPNccwf1TlUDxlnVzEGXbh&mt=1546564973&mn=sn-u2oxu-f5fe6%2Csn-5hne6nsy&mm=31%2C26&txp=5531432&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&key=yt6&mv=m&dur=1966.730&source=youtube&initcwndbps=778750&itag=22&ms=au%2Conr&ei=w7UuXOrrH8_JgAe5z4qoCw'
[download] Destination: Joe Rogan Goes Balls Deep On A.I.-uNO7QwGZKic.mp4
[download] 100% of 110.21MiB in 00:03
[download] Downloading video 7 of 10
[youtube] 2eVR0i5YAv0: Downloading webpage
[youtube] 2eVR0i5YAv0: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r1---sn-u2oxu-f5fe6.googlevideo.com/videoplayback?signature=A703C97C2F2984C5AE267D24B571B61141F2B0F2.E55690B4487A7DADB2915F8D2C7186ED267338&lmt=1540167245156324&ratebypass=yes&fvip=4&ei=zLUuXPSZO42wgQebwZj4DA&requiressl=yes&ipbits=0&mt=1546564973&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&initcwndbps=778750&ip=79.184.191.139&key=yt6&dur=641.474&itag=22&pl=24&source=youtube&mv=m&ms=au%2Conr&mm=31%2C26&mn=sn-u2oxu-f5fe6%2Csn-5hne6n7s&id=o-ABH_cxqpxjqsBvjWFhprN7KcNinrz-g_LJSUnUhUR22I&c=WEB&expire=1546586669&txp=5531432&mime=video%2Fmp4'
[download] Destination: 6 Scariest Things Said by A.I. Robots-2eVR0i5YAv0.mp4
[download] 100% of 41.91MiB in 00:01
[download] Downloading video 8 of 10
[youtube] WSW-5m8lRMs: Downloading webpage
[youtube] WSW-5m8lRMs: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r7---sn-u2oxu-f5fk.googlevideo.com/videoplayback?ratebypass=yes&c=WEB&lmt=1545435539063909&expire=1546586676&mime=video%2Fmp4&source=youtube&itag=22&ei=1LUuXMveBYns1wKupIPgAg&requiressl=yes&fvip=4&dur=465.351&mt=1546564973&mm=31%2C26&signature=A670E4733F0C5DC68D1750113156E394C2EB165A.E056149B1CF737ADAE6690AB5958B25B698C1670&key=yt6&id=o-AK-5tABpsJ5dMLktksFqMYqj_B--P6g46Vi7zqV8UN0_&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&ipbits=0&initcwndbps=860000&mv=m&pl=24&txp=4535432&ms=au%2Conr&ip=79.184.191.139&mn=sn-u2oxu-f5fk%2Csn-5hnekn7l'
[download] Destination: A.I. Learns to play Flappy Bird-WSW-5m8lRMs.mp4
[download] 100% of 15.12MiB in 00:00
[download] Downloading video 9 of 10
[youtube] XvzNuw5VjBU: Downloading webpage
[youtube] XvzNuw5VjBU: Downloading video info webpage
[youtube] {22} signature length 40.41, html5 player vflSzU_20
[youtube] {43} signature length 40.41, html5 player vflSzU_20
[youtube] {18} signature length 40.41, html5 player vflSzU_20
[youtube] {36} signature length 40.41, html5 player vflSzU_20
[youtube] {17} signature length 40.41, html5 player vflSzU_20
[youtube] {313} signature length 40.41, html5 player vflSzU_20
[youtube] {271} signature length 40.41, html5 player vflSzU_20
[youtube] {137} signature length 40.41, html5 player vflSzU_20
[youtube] {248} signature length 11.66.1, html5 player vflSzU_20
[youtube] XvzNuw5VjBU: Downloading player https://www.youtube.com/yts/jsbin/player_ias-vflSzU_20/en_US/base.js
[youtube] {136} signature length 40.41, html5 player vflSzU_20
[youtube] {247} signature length 40.41, html5 player vflSzU_20
[youtube] {135} signature length 38.41, html5 player vflSzU_20
[youtube] {244} signature length 40.41, html5 player vflSzU_20
[youtube] {134} signature length 40.41, html5 player vflSzU_20
[youtube] {243} signature length 40.41, html5 player vflSzU_20
[youtube] {133} signature length 40.41, html5 player vflSzU_20
[youtube] {242} signature length 40.41, html5 player vflSzU_20
[youtube] {160} signature length 40.41, html5 player vflSzU_20
[youtube] {278} signature length 40.41, html5 player vflSzU_20
[youtube] {140} signature length 40.41, html5 player vflSzU_20
[youtube] {171} signature length 40.41, html5 player vflSzU_20
[youtube] {249} signature length 40.41, html5 player vflSzU_20
[youtube] {250} signature length 40.41, html5 player vflSzU_20
[youtube] {251} signature length 40.41, html5 player vflSzU_20
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r7---sn-u2oxu-fhpe.googlevideo.com/videoplayback?sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2cms%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&fvip=1&mime=video%2Fmp4&txp=5432432&pcm2cms=yes&source=youtube&pl=24&ratebypass=yes&initcwndbps=773750&c=WEB&mm=31%2C26&mn=sn-u2oxu-fhpe%2Csn-5hne6nsk&ipbits=0&requiressl=yes&expire=1546586680&mt=1546564973&mv=m&ei=2LUuXOnwGJGOgQe1g4rgDw&ms=au%2Conr&lmt=1545664860984440&dur=260.713&key=yt6&ip=79.184.191.139&itag=22&id=o-AE6X1GxV6x2EGw9grq-6c5vwF3UfzmSxd3Rwgl2XmAY2&signature=AE3537BE1468A22636E8FE0D5A2DC688ACAC9183.573A56DDCA0A3EADAA1AAB70EC2659160D481345'
[download] Destination: How artificial intelligence will change your world in 2019, for better or worse-XvzNuw5VjBU.mp4
[download] 100% of 37.18MiB in 00:12
[download] Downloading video 10 of 10
[youtube] Dk7h22mRYHQ: Downloading webpage
[youtube] Dk7h22mRYHQ: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r6---sn-u2oxu-f5fk.googlevideo.com/videoplayback?dur=3085.815&mv=m&source=youtube&ms=au%2Conr&itag=22&initcwndbps=860000&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&id=o-AETt-6gVSToAIxdxcSBdPywAJntuTwvv0oEaQh7C9OEy&mt=1546564973&mn=sn-u2oxu-f5fk%2Csn-5hne6nsd&mm=31%2C26&ipbits=0&pl=24&signature=6935470D8D8F0D3086FA03AE30F53D635E984818.98AAA38EF9CDB63D8F9F932D041AFEE3B2DA1105&ratebypass=yes&ip=79.184.191.139&ei=7LUuXN7LFPH6gAfchZnQCA&lmt=1529121092479258&c=WEB&expire=1546586700&key=yt6&fvip=4&requiressl=yes&mime=video%2Fmp4'
[download] Destination: The Rise of AI-Dk7h22mRYHQ.mp4
[download] 100% of 365.54MiB in 00:16
[download] Finished downloading playlist: ai

Description of your issue, suggested solution and other information

Basically the average_rating parameter appears to not be working correctly, according to:
krzysztofs-macbook-pro:~ kk$ youtube-dl -o '%(title)s.%(average_rating)s.%(ext)s' "ytsearch10:ai"
5 of these videos should not be downloaded( for example last one has average_rating of 4.7812795639)

@Lomacar
Copy link

@Lomacar Lomacar commented Mar 4, 2020

I have the same problem. In the test I am doing only one video is filtered out even though many other videos are below the rating threshold. Youtube-dl knows what the rating is but still doesn't filter correctly. I think it is only accepting integer values (flooring the rating threshold given by the user). Because the one video that it does filter out for me has a rating of 3.6. Anything at or above 4 and the filter catches it, but at 3.9 it passes through.

@Lomacar
Copy link

@Lomacar Lomacar commented Mar 4, 2020

For now a workaround could be to filter for like_count > X & dislike_count < Y. But this should be easily fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.