0.19.0

[MohamedBassem]

Welcome to the 0.19.0 release of Hoarder! This release is packed with a lot of new features! The long awaited REST APIs, RSS subscriptions, AI summarization, Image OCR, archiving videos, importing from Omnivore, and more!

If you're enjoying Hoarder, you can consider supporting it here ☕️.

New Feature

• RSS Subscriptions 🧪: You can now subscribe to RSS feeds and Hoarder will automatically pull them every hour.
  • You can use this to auto hoard stuff from your favourite blogs, subreddits, or youtube channels.
  • This feature is still experimental and haven't been tested on a a wide variety of feeds. Please file issues whenever you face any problems using it.
• The long awaited REST API is here. You can view the API docs here.
  • You can view the OpenAPI spec here in case you want to generate clients for it. I'm planning to offer an official Typescript SDK some time in the future.
• AI Summarization: Hoarder now has the ability to generate summaries for the links that you hoard! Check the screenshots section to see it in action.
  • Currently, summarization is a manual action. In the future, we'll add support for auto summarization.
• Video downloads: For ultimate hoarding, Hoarder now has the ability to download videos from the links that you hoard (e.g. youtube) thanks to @kamtschatka.
  • Given that videos can be huge, this is an opt-in feature. You can enable it with CRAWLER_VIDEO_DOWNLOAD=true. Find more configuration options here.
  • Hoarder uses ytdl under the hood, and you can find their list of supported files here.
• Omnivore Imports: Omnivore is shutting down. You can now import your bookmarks from Omnivore into Hoarder. Check the documentation here.
• User Management: Admins can now reset user passwords, change their roles and even create new users thanks to @kamtschatka.
• Image OCR: Hoarder will now attempt to read and index text in the images that you hoard! Check the configuration variables here.
• Others:
  • @kamtschatka Added authentication failure logging in case you want to configure fail2ban against hoarder logs.

UX Improvements

• We have a new redesigned settings page to better support the new options we're adding. Check it out in the screenshot section.
• You can use Cmd/Ctrl + K to quickly focus on Hoarder's searchbar.
• We reverted the mobile apps to again view webpages inline, but now added buttons for quick actions underneath it.

Fixes

• Single new lines in notes were getting lost in markdown. @kamtschatka fixed that.
• Pocket switched their export format to CSVs instead of their old format. Hoarder now supports importing the new format.
• Tags that had numbers or dashes in them were getting skipped, this is now fixed. Thanks @MakeSomeGood for the bug report and investigation!
• The All Tags page was getting very slow when there's a lot of tags. This turned out to be a small misconfiguration in the database that we fixed.
• Custom prompts had a max length of 100, which wasn't enough to some people. The new limit is 500.

Screenshots

AI Summarization

Screen.Recording.2024-11-09.at.4.12.35.PM.mov

RSS Feeds & New Settings Page

Admin User Management

Upgrading

To upgrade:

• If you're using HOARDER_VERSION=release, run docker compose pull && docker compose up -d.
• If you're pinning it to a specific version, upgrade the version and then run docker compose pull && docker compose up -d.

All Commits

• fix: Increase max prompt length to 500 chars instead of 100 - @MohamedBassem in 8ce272d
• docs: Add documentation about how to import from pocket and omnivore - @MohamedBassem in 2b6eb7c
• docs: Update CLI importing docs to better handle large collections (Update 10-import.md #630) - @bjornmonnens in e57937e
• fix: Remove old downloaded video when it gets refreshed - @MohamedBassem in a24755d
• fix: Only update bookmark tagging/crawling status when worker is out of retries - @MohamedBassem in f8bed57
• fix: Feed refreshes were not getting re-enqueued for failed jobs - @MohamedBassem in 10070c1
• ui: Mark the RSS subscriptions as an experimental feature - @MohamedBassem in 9ca49b2
• fix: Fix slow AllTags page because of misconfigured tagId index. Fixes [Perf] Move away from the APIs that return all tags in one request #528 - @MohamedBassem in 8f44c81
• fix: Pass arguments to monolith and yt-dlp as array for better escaping - @MohamedBassem in cbaa033
• feature: Add support for importing bookmarks from Omnivore. Fixes Import from omnivore #602 - @MohamedBassem in 7ec5746
• fix: Adopt pocket's new export format. Fixes Import from Pocket: Pocket now mails you zipped CSV files #570 - @MohamedBassem in 7042f26
• fix: Fix bug in tag normalization regex. Fixes Some tags not being added after inference with Open AI endpoint #595 - @MohamedBassem in 306567b
• feature: Schedule RSS feed refreshes every hour - @MohamedBassem in d34b538
• feature(web): Add the ability to view the bookmarks of a particular rss feed - @MohamedBassem in fa8286a
• feature: Add support for subscribing to RSS feeds. Fixes Do you plan to allow this to consume RSS feeds? #202 - @MohamedBassem in cf1a251
• fix(mobile): Fix bookmarkview page on android not showing bookmark actions. Fixes [Android] WebView Does Not Fill Entire Screen and Lacks Controls  #603 - @MohamedBassem in 2efc7c8
• fix: Fix type in the separate container env variable - @MohamedBassem in 4c94b37
• feature: Archive videos using yt-dlp. Fixes Allow downloading more content from a webpage and index it #215 (Allow downloading more content from a webpage and index it #215 #525) - @kamtschatka in 4a13c36
• docs: Add a FAQ page (Faq #590) - @kamtschatka in 3b7451f
• deps: Extract the queue implementation into its own repos - @MohamedBassem in a746e9a
• ui: Redesign the settings page and move it to its own layout - @MohamedBassem in eb7da99
• fix: Index the summary in search - @MohamedBassem in 801ba36
• fix: Some visual fixes to the AI summary - @MohamedBassem in e06a5ea
• fix: Fix search input escape listener to only work when the input is in focus - @MohamedBassem in 9d6d70d
• fix: Fix build by regenerating the openai spec - @MohamedBassem in c0b0691
• feature: Add a summarize with AI button for links - @MohamedBassem in 731d2df
• refactor: Move inference to the shared package - @MohamedBassem in 3e727f7
• docker: Add a flag to detect apps running in legacy container setup - @MohamedBassem in db45aaf
• release(mobile): Bump mobile version to 1.6.5 - @MohamedBassem in bb344e5
• feature(mobile): Use inline WebView for expanding bookmarks - @MohamedBassem in 2ce42a8
• feature: Add OCR support for images. Fixes [Feature Request] OCR search text in images #296 - @MohamedBassem in 019b5d2
• ci: Add a build step that ensures that open api spec doesn't diverge - @MohamedBassem in f793646
• ci: Fix build by adding corepack enable to CI - @MohamedBassem in 5b20d97
• docs: Add link to discord to navbar - @MohamedBassem in 09d29e4
• api: Add tag ids and support variable baseURL - @MohamedBassem in b476e91
• docs: Generate OpenAPI docs - @MohamedBassem in 6ffa51d
• chore: Define hoarder's rest API in zod format - @MohamedBassem in 3c1ec3a
• fix: Improve field names in the tag APIs - @MohamedBassem in 4086c37
• feature(api): Add REST APIs to update bookmarks, tags and lists - @MohamedBassem in e89a386
• feature(api): Add REST APIs to manipulate lists and tags for bookmarks - @MohamedBassem in 20e5225
• feature: Add APIs to create new lists and bookmarks - @MohamedBassem in 10dcf2e
• fix: Better handling for body JSON parsing errors - @MohamedBassem in f5fd3c4
• feature: Add DELETE REST APIs for bookmarks, lists and tags - @MohamedBassem in 719e25d
• feature: Add GET REST APIs for bookmarks, lists and tags - @MohamedBassem in 62395ec
• featue: Add infra for REST APIs and implement GET /bookmarks - @MohamedBassem in fb297ea
• fix(workers): Pin execa to avoid ERR_PACKAGE_PATH_NOT_EXPORTED error - @MohamedBassem in a822ff2
• feature(web): Add keyboard shortcut to focus on search bar. Fixes [Feature Request] Keyboard Shortcut & Search Improvements #449 ([Feature Request] Keyboard Shortcut & Search Improvements #449 #554) - @kamtschatka in cbc268e
• feature: Allow reseting user password, change their roles and create new users. Fixes Request: Add Invite-Only Registration Option for Admins #495 (User management #567) - @kamtschatka in 9a56e58
• feature: Log authentication failures to support fail2ban. Fixes [Feature Request] Log failed login attempts for fail2ban implementation  #477 ([Feature Request] Log failed login attempts for fail2ban implementation #477 #569) - @kamtschatka in 0debc6b
• chore: Adding issue templates (Adding issue templates #546) - @Inventhrice in e55362e
• fix(web): Single linebreaks are lost when viewing textnote card. Fixes Single linebreaks are lost when viewing textnote card #549 (Single linebreaks are lost when viewing textnote card #549 #551) - @kamtschatka in 6977ba7
• release(mobile): Bump mobile version to 1.6.4 - @MohamedBassem in 4454d63
• fix(mobile): Improve connection testing accuracy - @MohamedBassem in 90953fd

---

This discussion was created from the release 0.19.0.