Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update erroneously ignores videos due to age limit #106

Closed
bnsmcx opened this issue Nov 9, 2021 · 2 comments
Closed

Update erroneously ignores videos due to age limit #106

bnsmcx opened this issue Nov 9, 2021 · 2 comments
Labels

Comments

@bnsmcx
Copy link

bnsmcx commented Nov 9, 2021

Hopefully this is user error on my part but added as much detail as I could in case it isn't.

Describe the bug
With some podcast feeds running ytcc update fails with this error even if the podcasts were published recently:

WARNING: Ignoring video 'Practical Shooting After Dark EP 166' due to age limit

To Reproduce
Steps to reproduce the behavior:

  1. Add the podcast: ytcc subscribe "test" "http://feeds.feedburner.com/PracticalPistolShow"
  2. The automatic update that happens after adding the podcast will exhibit the error I'm talking about.

Expected behavior
I'm expecting recent podcasts to correctly update and be available for download. I tried playing with the age_limit and max_backlog settings with no success.

Environment, configuration and version information

---ytcc version---
2.5.1

---youtube-dl version---
2021.06.06

---yt-dlp version---
yt-dlp not found

---Click version---
8.0.3

---SQLite version---
SQLite system library version: 3.34.1
Python module version: 2.6.0

---python version---
3.9.5 (default, May 11 2021, 08:20:37) 
[GCC 10.3.0]

---mpv version---
mpv is not installed

---config dump---
[ytcc]
download_dir = ~/Downloads
download_subdirs = false
mpv_flags = --really-quiet --ytdl --ytdl-format=bestvideo[height<=?1080]+bestaudio/best
order_by = playlists:asc, publish_date:desc
video_attrs = id, title, publish_date, duration, playlists
playlist_attrs = name, url, tags, reverse
db_path = ~/.local/share/ytcc/ytcc.db
date_format = %Y-%m-%d
max_update_fail = 5
max_update_backlog = 20
age_limit = 0

[tui]
alphabet = sdfervghnuiojkl
default_action = play_video

[theme]
prompt_download_audio = 2
prompt_download_video = 4
prompt_play_audio = 2
prompt_play_video = 4
prompt_mark_watched = 1
table_alternate_background = 245

[youtube_dl]
format = bestvideo[height<=?1080]+bestaudio/best
output_template = %(title)s.%(ext)s
ratelimit = 0
retries = 0
subtitles = off
thumbnail = true
skip_live_stream = true
merge_output_format = mkv
max_duration = 0
restrict_filenames = false

Debug log (optional)
Enable the debug logging and run buggy command again. Note, that this can print sensitive information!

ytcc --loglevel debug update
[1636429645.358416] [MainProcess/MainThread] ytcc.config.DEBUG: Trying to read config from following locations: ['/etc/ytcc/ytcc.conf', '/home/ben/.config/ytcc/ytcc.conf', '/home/ben/.ytcc.conf']
[1636429645.358777] [MainProcess/MainThread] ytcc.config.DEBUG: Config was read from following locations: ['/home/ben/.config/ytcc/ytcc.conf']
[1636429645.359718] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA foreign_keys = ON;
[1636429645.359791] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA user_version;
[1636429645.359986] [MainProcess/MainThread] asyncio.DEBUG: Using selector: EpollSelector
[1636429645.360289] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT p.id AS id, p.name AS name, p.url AS url, p.reverse AS reverse, t.name AS tag FROM playlist AS p LEFT OUTER JOIN tag AS t ON p.id = t.playlist;
[1636429645.360601] [MainProcess/MainThread] ytcc.database.DEBUG: WITH ids as ( SELECT v.id FROM video AS v JOIN content c ON v.id = c.video_id JOIN playlist p ON p.id = c.playlist_id LEFT JOIN tag AS t ON p.id = t.playlist WHERE v.publish_date > 0 AND v.publish_date < Inf AND p.name IN ('Jupiter Broadcasting') ) SELECT v.id AS id, v.title AS title, v.url AS url, v.description AS description, v.duration AS duration, v.publish_date AS publish_date, v.watch_date AS watch_date, v.thumbnail_url AS thumbnail_url, v.extractor_hash AS extractor_hash, p.name AS playlist_name, p.url AS playlist_url, p.reverse AS playlist_reverse FROM video AS v JOIN content c ON v.id = c.video_id JOIN playlist p ON p.id = c.playlist_id WHERE v.id in ids
[1636429645.510520] [MainProcess/MainThread] ytcc.updater.INFO: Checking playlist 'Jupiter Broadcasting'...
[1636429645.516529] [MainProcess/MainThread] ytcc.database.DEBUG: WITH ids as ( SELECT v.id FROM video AS v JOIN content c ON v.id = c.video_id JOIN playlist p ON p.id = c.playlist_id LEFT JOIN tag AS t ON p.id = t.playlist WHERE v.publish_date > 0 AND v.publish_date < Inf AND p.name IN ('Stoeger') ) SELECT v.id AS id, v.title AS title, v.url AS url, v.description AS description, v.duration AS duration, v.publish_date AS publish_date, v.watch_date AS watch_date, v.thumbnail_url AS thumbnail_url, v.extractor_hash AS extractor_hash, p.name AS playlist_name, p.url AS playlist_url, p.reverse AS playlist_reverse FROM video AS v JOIN content c ON v.id = c.video_id JOIN playlist p ON p.id = c.playlist_id WHERE v.id in ids
[1636429645.528224] [MainProcess/MainThread] ytcc.updater.INFO: Checking playlist 'Stoeger'...
[1636429650.402324] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 19766a045500c24b0626de440226d8320e056de177fdf77d4ddd09c7be21f680'
[1636429650.402446] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 09a139e7dbf843085b923851b2180a724cb0f3d2bbe37206ae036a93490bc5aa'
[1636429650.402499] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic f1eb24ab1bfb1e03cb191a593089b9afd18b3b143931bf4967592c1a2458e196'
[1636429650.402545] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 6b9665b938911992dd69133b00bdae0e5c6caa733501a48fb4939d27abc8b4a5'
[1636429650.402585] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 571d7193388ea3b4c1b027031b5951afca03f9dc2252ab2a8fcefd0b55c8df13'
[1636429650.402625] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 749014bfa824469763229237a0bba417f382fb32d0ad86afc0bf01c60ecec7e0'
[1636429650.402663] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic e03258c9e8e7b1730639c92ec4ca1def8dc92c2815d3860734e4a6960be230a0'
[1636429650.402701] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 23a421d1a9e7979e417fcd361771f6519384d07458961c165dc0886d6c88bdd0'
[1636429650.402740] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 5e84548b2d452d4876dc0c22da33c64bdb27c22d0c6f360042730fc83bdf37b2'
[1636429650.402776] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 24842b27bfcc9925606f7a35abd114ea99f28295a13fedb774ef7e60c4849161'
[1636429650.402814] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic a488d119b4c89aac056229354a9a96fc21017d287872a2a44ee9338ff5232d88'
[1636429650.402851] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 3edc7066ffcbbe8f0e67f2d438c966dd0db6a81d63e4f9f44ffcb642c4428642'
[1636429650.402889] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 19bdba5ea2a5c5fd224ef37337c745e028f2ed7bf4e2ec2b757bfa903a8c0d0f'
[1636429650.402926] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 13b32b67481e9d4c9929bf4c432f7644ddc8884966f3c7c9afcfe05b9f2c46f8'
[1636429650.402964] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic bad281b7e13fa7d67db3b40a52e9cbf92947fcfa7d8a85d9e307d4b46a7f9dc3'
[1636429650.403001] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic e28f179521a92b90d9a8df968e1fe4550b91039d778527cef3d710f75ae2cfff'
[1636429650.403039] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 115d1c14a9c466754c52a9ff19861ddf85a9f67780eb7530e7755d17e7d55559'
[1636429650.403078] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic dda4b1ab750da0b1c8ddf065a89738a541abd35d0e14d112f5f23cb3e8b46258'
[1636429650.403116] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT failure_count FROM extractor_meta WHERE extractor_hash = 'generic 8f4ebdecfc48f64f880be9d4615dd64fe9391ce956269006062869df64dd2113'
[1636429654.286895] [MainProcess/MainThread] ytcc.updater.WARNING: Ignoring video 'Matt's war on the USPSA BOCs carries on' due to age limit
[1636429654.287394] [MainProcess/MainThread] ytcc.database.DEBUG: BEGIN
[1636429654.287600] [MainProcess/MainThread] ytcc.database.DEBUG: INSERT OR IGNORE INTO extractor_meta VALUES ('generic 19766a045500c24b0626de440226d8320e056de177fdf77d4ddd09c7be21f680',0)
[1636429654.287963] [MainProcess/MainThread] ytcc.database.DEBUG: UPDATE extractor_meta SET failure_count = failure_count + 1 WHERE extractor_hash = 'generic 19766a045500c24b0626de440226d8320e056de177fdf77d4ddd09c7be21f680' AND failure_count < 5
[1636429654.288445] [MainProcess/MainThread] ytcc.database.DEBUG: COMMIT
@bnsmcx bnsmcx added the bug label Nov 9, 2021
@woefe
Copy link
Owner

woefe commented Nov 9, 2021

The feed has an explicit rating, which is interpreted as age limit 18 in ytcc (or actually youtube-dl / yt-dlp).
To get the missed entries of that feed you need to

  • Set age_limit appropriately in your ytcc.conf
  • Run ytcc update --max-fail 9999 once after you changed the config (This step is necessary, because ytcc ignores failed updates after 5 attempts by default)
  • The ignored videos / podcasts should now be available

@woefe woefe closed this as completed Nov 9, 2021
@bnsmcx
Copy link
Author

bnsmcx commented Nov 10, 2021

Ah, very nice, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants