v0.4.10

Project Updates

• This is a minor release to ship the latest fixes from the amazing yt-dlp team.

v0.4.9

Project updates

• This ships the latest fixes for channel extractions by the amazing yt-dlp team.
  • See below to manually trigger a refresh of your failed channels.
• Some users have reported problems for index mapping updates, particularly on bigger archives. To counteract issues like that, increase the memory limit defined in ES_JAVA_OPTS for ES so there is enough space to sync the index in bulk.
  • That is also represented in the official docker-compose.yml file, make the same changes as here.
  • If you maintain any installation help, guides, templates or similar, please update it there too.
  • Ideally make the change first before applying the update.
• This will trigger a rebuild for the video and channel indexes at first start, as always this should be mostly automatic.
  • If you encounter any problems, see here.
• This will probably be the last release for some time. We’ll make a cut here to start refactoring things in the backend, and rewrite the frontend in a modern JS framework. Check the open issues and reach out if you want to help with that process

Added

• Added per channel page size, aka deactivating shorts or streams per channel, docs, by @Boo1098
• Added retry logic for failed download tasks.

Changed

• Changed behavior when changing the TZ (Time Zone) environment variable. Existing schedules will now get changed to use the new TZ value.

Fixed

• Fixed error when adding video not part of a channel, #741
• Fixed error when a previously failed download would raise an incorrect FileNotFoundError , #752
• Fixed issue allowing retrying from cache on previously failed download.
• Fixed issue where version check task would never trigger depending on your TZ value and uptime.

Manual updates

As yt-dlp fixed all issues with channel extraction, you can run trigger a reindex for channels that have failed to index properly before, run python manage.py ta_fix_channels in your Tube Archivist container, e.g.:

docker exec -it tubearchivist python manage.py ta_fix_channels

This will scan your channels and add faulty channels to the regular reindex queue. That should return in a few seconds, you will see ✓ task queued. Then you can monitor the reindex progress on your settings page and in your docker logs.

v0.4.8

Hotfix published

There is a hotfix published for ARM64 builds where the ffmpeg binaries weren't downloaded correctly into the container. If you are affected, pull again.

Project updates

• This ships various fixes for playlist refresh limitations. Also see the docs here clarifying that functionality.
• This fixes a comment extraction problem due to a change on YT
  • See below for a manual fix for your comments
• This also partially fixes failed channel extraction, we are waiting for upstream fix, that’s tracked here
  • See below to manually trigger a refresh of your failed channels
• This ships a new schedule management system, fixing various previous limitations. Your existing schedules and notification URLs should migrate automatically at first start, monitor your startup logs and validate on your schedule settings page that everything transferred correctly.

Added

• Added delete and ignore for videos, docs #286
• Added new notification hook for add to queue task, #714
• Added thumbs clean up in Thumbnail Check task
• Added 404 handler for detail views
• Added better error handling for invalid urls
• Added a few unit tests

Changed

• Changed build to use python script to download best ffmpeg build
• Changed schedule store to use beats model, no longer requiring restart to apply schedule, docs
• Changed task notification management, docs
• Changed playlist matching logic in download post processing, docs
• Changed queue handling to use Redis as temporary store, #500
• Changed queue handling progress and position tracking to use score in ordered set
• Changed to cleaner and more concise seconds to date parser

Fixed

• Fixed comment extraction
• Fixed playlist subscription bool logic in template, #684
• Fixed playlist rescanning logic, ensuring refresh, #686
• Partial fix channel subs extraction with workaround, #701, upstream link
• Fixed adding existing playlist to queue again, #634
• Fixed playlist matching not removing deleted videos
• Fixed rescan negative total notification
• Fixed schedule cron parser, you are no longer able to enter an invalid schedule
• Fixed logout
• Fixed application startup dependency making tests unrunnable, #564

Manual updates

You don’t need to run these commands, the regular metadata refresh should fix these problems, at least for active videos. That’s just if you want to fix it now.

First command will “relink” comments with your video. Due to a failed comment extraction, that link got lost, but all your comments are still here. Run python manage.py ta_fix_comment_link in your Tube Archivist container, e.g.:

docker exec -it tubearchivist python manage.py ta_fix_comment_link

Depending on your index size and system performance, expect this to take around 15 seconds for 10k videos. You will see ✓ task completed once that’s done.

Second command will trigger a reindex for channels that have failed to extract before, run python manage.py ta_fix_channels in your Tube Archivist container, e.g.:

docker exec -it tubearchivist python manage.py ta_fix_channels

This will scan your channels and add faulty channels to the regular reindex queue. That should return in a few seconds, you will see ✓ task queued. Then you can monitor the reindex progress on your settings page and in your docker logs.

v0.4.7

Project Updates

• This will trigger a playlist migration in ElasticSearch, adding a playlist_type field. As always that happens at first start and should be automatic.
• This updates yt-dlp to the latest release. After a change on YT, comment extraction needs to be modified, that is tracked here. Please be patient.

Added

• Added user created custom playlists, by @greg321321, docs and docs
• Added better error message for cookie errors, by @Gentoli

Changed

• Changed disable video autoplay on the video detail page to avoid too many continue watching items.
• Changed first watch progress is now only sent after 10 seconds to avoid too many continue watching items.

Fixed

• Fixed failed empty subtitle return from YT, #663
• Fixed empty comment author return for some old comments, by @dot-mike
• Fixed edge cases for failed channel title building
• Fixed download progress bar for very small videos, by guarantee sending 100% progress
• Fixed download progress messages getting called multiple times in DOM
• Fixed failed legacy channel migration trigger by removing ta_migpath from startup, #687

v0.4.6

Project Updates

• This is a small maintenance release fixing some bugs
• This has a workaround for a memory usage problem in Celery, the task scheduler

Fixed

• Fixed a memory leak in celery by reducing the worker life time, by @PhuriousGeorge
• Fixed an issue where deleting a channel wouldn't clean up subtitles correctly
• Fixed reindexing item that has since been deleted locally after queue start
• Fixed throttling video redirect when reindexing by validating IDs and raising error on mismatch

v0.4.5

Project Updates

• This is a small release, mainly shipping small but crucial fixes
• Getting this out, then we can take our time for the next additions

Changed

• Changed temporary queue building interface, use redis sorted set instead of lists

Fixed

• Fixed version check comparison over multiple version diff
• Fixed an issue where version_check schedule builder will reset unexpectedly, #611, by @lamusmaser
• Fixed upstream channel about page parser, #614
• Fixed upstream like_count extraction for videos
• Fixed mobile scaling issue in chrome based browsers, #616
• Fixed auto delete day parsing incompatible data types, #622
• Fixed author_is_uploader extraction in comments after yt-dlp API change
• Fixed incompatibility between celery and redis library versions by downgrading redis

Hotfix

If you have updated within approximately the first 24h under certain circumstances you may have gotten an error during the version check at startup. Pull the image again, if you are affected.

v0.4.4

Project Updates

• This fixes various mapping problems, triggering a rebuild of the ta_video and ta_playlist index
  • As always, this should be mostly automatic, if not see here.
• This fixes an unfortunate problem where the periodic refresh task wouldn't add outdated videos

Added

• Added additional sort by duration and file size, #559, by @spechter
• Added fallback handling for missing keys in manual import, #581, by @lamusmaser
• Added additional tiles to the dashboard, changed dashboard layout
• [API] Added video stats endpoint
• [API] Added channel stats endpoint
• [API] Added playlist stats endpoint
• [API] Added download stats endpoint

Changed

• [API] Removed previous /api/stats/primary/ endpoint in favor of more detailed dedicated endpoints
• Changed footer text alignment, by @derekslenk

Fixed

• Fixed query not matching outdated documents for reindex
• Fixed reindex outdated pagination error
• Fixed comment extraction for members only videos not passing cookie correctly, #596
• Fixed auto delete watched on new blank index without watched videos, #592
• Fixed mapping problem for some playlists, #605
• Fixed various spelling errors on settings pages, by @pairofcrocs

v0.4.3

Project Updates

• This is the second step in moving away from redis persistence, refactoring the environment variable parsing
• This ships a workaround for the new channel about popup parsing failure

Added

• Added read only user functionality, docs, #529 by @stratus-ss
• Added store for volume change in session
• Added CSS styling functionality, docs, by @extome9000
• Added daily download media size to dashboard
• [API] Added zip file backup endpoints
• [API] Added user config endpoints

Changed

• Changed env var handling, Refactored Environment settings, by @anonamouslyginger
• Changed dashboard tile building, by @MerlinScheurer
• Changed daily download stats to be timezone aware
• Touch up the icons, fix coloring for theme, by @pairofcrocs

Fixed

• Fixed channel about page parsing, #587
• Fixed direct video download button, #566 by @OmarLaham
• Fixed off by one message in filesystem rescan

v0.4.2

Project update

• First step towards moving away from redis plugins and redis persistence. At application startup this will migrate your user configurations from Redis to ES.

Added

• Added search score to full text search results, docs
• Added mark as unwatched button for playlists and channels, docs and docs, #495 by @froseph
• Added env vars for wider range of ES configurations, docs, #462 by @ajgon
  • ES_SNAPSHOT_DIR: File path for where the snapshots will get stored
  • ES_DISABLE_VERIFY_SSL: Disable SSL verification for ES
• Added health check endpoint, example in compose, by @ajgon
• Added forward authentication to support auth proxies, docs, #403 by @ajgon
• Added years to duration strings for aggregations

Changed

• Changed biggest channel stats to three different sorted tables, by @MerlinScheurer
• Changed and consolidated duration parser to produce human readable string
• Changed and completed search processor and serializer to use API throughout
• Changed user configuration to store in ES, first step moving away from Redis plugins, by @anonamouslyginger
• Changed ES wrapper for better dynamic args builder
• Changed json backup to split into multiple smaller files, #406
• Old wiki is now removed, in favor of docs

Fixed

• Fixed URLs to settings docs, #545 by @MerlinScheurer
• Fixed yt-dlp playlist extraction
• Fixed various yt-dlp errors with newest release, #522

v0.4.1

Project updates

• Due to an unfortunate misunderstanding between epoch seconds and milliseconds, this will trigger an index rebuild
  • Depending on your archive size, this can take anywhere from a few seconds to a few minutes to complete.
  • Be patient, follow the logs and let it do its thing.
• Tube Archivist Companion, the browser extension also got an update to v0.2.0 with lots of great additions. See the release notes there.

Added

• Added archive statistic dashboard
• [API] Added playlist delete endpoint
• [API] Added channel ID search endpoint
• [API] Added archive aggregation endpoints
• Added f hotkey for fullscreen to video player, by @dmynerd78

Changed

• Changed settings pages, are now split up for easier navigation, by @anonamouslyginger
• Changed redis config save to trigger bgsave
• Changed to ignore eaDir files from synology, #523
• Changed to reset autostart videos when stopping queue, #520

Fixed

• Fixed final thumbnail error handling
• Fixed validation error when subscribing to video in playlist, #299
• Fixed download folder list not using ingore_filelist
• [API] Fixed sponsorblock endpoint not returning 404, #526
• Fixed wrong index of epoch seconds/milliseconds in ES