Skip to content

v0.12.0

Choose a tag to compare

@jakublibik jakublibik released this 07 Jul 18:06
· 75 commits to master since this release

Added

  • Copy article action on the article ··· menu (desktop) and the bottom action bar (mobile). It copies the title, source and body to the clipboard as both rich HTML and plain text, so it pastes with formatting and images into rich editors and as clean text everywhere else. Relative image and link URLs are rewritten to absolute so they still resolve after pasting.
  • The Stats backlog cards (labeled and starred) now link straight into the matching reader view, so you can jump from a count to the actual articles.
  • Admin → Feeds: a "Rate limits" view (shown only when some exist) lists the fetch pace Readfine has learned per host, with the host, its spacing, and how and when that was learned, each with a Clear action. Errored feeds in the admin table also show their predicted next fetch, and a feed with no per-feed interval override shows the effective default instead of a blank dash.
  • Admin → Feeds: a "By host / A-Z" toggle that groups the feed list by fetch host, so all of a site's feeds (say, every Reddit feed) sit under one host header with a count instead of a flat alphabetical list. Hosts sort alphabetically, single-feed hosts fall into an "Other" bucket, and any host holding an errored feed floats to the top so problems stay visible. Within a group, and in the flat list, feeds now sort by status first (errored, disabled, paused, active) and then by name. The choice is remembered per browser.

Changed

  • A single HTTP 403 no longer disables a feed. Reddit and YouTube return 403 as a transient anti-bot or rate-adjacent block (datacenter IP, generic user-agent) far more often than as a permanent denial, so 403 now backs off through the error tier like 408/429 and 5xx and only disables after several consecutive failures. Genuinely permanent 4xx (400, 401, 404, 410) still disable immediately.
  • The fetcher reads rate-limit headers (Retry-After, RateLimit-*, X-RateLimit-*) on both successful and 429 responses and applies a per-host cooldown. Once a host reports its budget is spent (for example Reddit's x-ratelimit-remaining: 0), other feeds on that host wait out the reset instead of hammering it with more 429s. The waiting happens inside the fetch round, up to a budget that keeps the round short enough not to miss the next slot; anything over that defers to the next round. Feeds on other hosts still fetch in parallel.
  • Manually refreshing a feed (the sidebar ↻ and the admin "force fetch") now respects a known rate-limit window instead of firing straight into another 429. While the host is cooling down it shows "Rate-limited, try again in …" (seconds or minutes, from the server's reset headers). A bare 403 anti-bot block is treated differently: only the background scheduler paces itself on those, since a manual retry often succeeds.
  • Readfine learns a sustainable fetch pace per host and spaces its requests accordingly, rather than only reacting once a host reports its budget already spent. It reads the pace from a host's rate-limit headers on successful responses and tightens it when the host keeps answering 429. The learned pace only ever tightens, so it never oscillates, and is capped so a feed can't stall forever. This keeps aggressive hosts like Reddit, where a burst of same-host feeds fetched back to back would trip a 403/429, from being throttled. The pace is stored and survives restarts and deploys, so a host isn't re-probed into a rate limit on every restart, and manual refreshes respect it too.
  • Feeds are fetched at whichever of the four 15-minute ticks (:00/:15/:30/:45) first follows their interval, instead of being pinned to the top of the hour. This spreads load across the hour on each feed's own phase rather than piling every hourly feed onto :00, and it improves freshness: an hourly feed first fetched a few minutes past the hour used to wait until the next :00 (up to ~2 h between fetches) and now refreshes about an hour later as intended. Feeds that miss a tick (a host cooldown, a transient error, a restart mid-round) recover at the next tick instead of waiting a full interval.
  • The per-feed refresh button (↻) now reloads the article list when you're viewing that feed, so newly fetched articles appear right away instead of only after re-selecting it. Refreshing a feed you're not viewing still just updates its unread badge.
  • The fetch-interval selector spells out the server default next to the "Default" option (for example "Default (60 min)") on the subscribe, scrape-setup and feed-edit forms, and wraps better on narrow screens.
  • Adding a feed now shows specific messages for rate-limiting (429, including when to retry) and temporary server errors (5xx) instead of a bare "HTTP error {status}".
  • The Feeds, Filters and Labels settings pages and the admin Users page show an item count next to the heading, kept current as items are added or removed without a reload.
  • Labels and filters sort case-insensitively everywhere now: settings lists, label pickers and chips. Previously the database collation put all uppercase names before any lowercase one, so a new lowercase label or filter got stuck at the end of the list.
  • The filter list shows a "priority N" badge on filters whose priority isn't the default, making it clear why a filter sorts and runs ahead of alphabetical order.
  • Briefings sent to extra recipients now put the account owner in To: and the additional recipients in Bcc, so co-subscribers no longer see each other's addresses. The modal also notes that delivery can lag the scheduled time by up to 15 minutes (the scheduler tick).
  • The admin "force fetch" button shows a spinner and blocks double-clicks while the synchronous fetch runs, instead of looking like it did nothing for several seconds.
  • Settings → AI cost estimates now cover the current Anthropic, OpenAI and Google model families. A configured model that isn't in the built-in price list is estimated from a typical model for its provider (shown with a "~" and a note under the table) instead of showing as unknown or zero.
  • The Trend column in the AI cost table tracks estimated cost rather than raw operation count, and the Fast/Quality/Total rows show a trend too (previously blank), so the arrows reflect what actually moves your spend, such as longer articles costing more at the same number of runs.

Fixed

  • Collapsing and expanding the sidebar is instant now. It used to refetch the whole sidebar from the server on every toggle, so the old layout lingered, briefly squished into the new width, until the request returned. Both the collapsed rail and the full sidebar are rendered up front and the toggle just switches between them in the browser, with no round-trip. On the mobile "collapsible" sidebar, opening the overlay no longer reflows the article-list text either, because the rail is a fixed strip now and the list keeps a constant width.
  • Toast notifications (a feed's error when you open it, or a manual refresh result) no longer render at roughly half-width on mobile. They stretch edge to edge with a small gutter on narrow screens and stay centred with a sensible max-width on wider ones.
  • Filters that share a priority run in a fixed order now, exactly the order the Settings → Filters list shows (priority, then name). Their order used to be left to the database, so a "stop on match" filter could behave differently between fetches.
  • The green "Feed added successfully" banner no longer reappears when you refresh the Feeds settings page after subscribing.
  • Articles that carry a label stay visible in their label view even after their feed is deleted or unsubscribed. The view used to inner-join the feed and hide them, leaving the sidebar badge counting an apparently empty category.
  • The article-list loading overlay matches the neutral dark-mode background instead of a blue-tinted grey, and is delayed slightly so quick cached loads don't flash a spinner.
  • The AI cost table's total is no longer quietly understated when a model slot uses a model missing from the price list. That slot used to be added as $0, making the total look complete while dropping part of the cost.

Security

  • Filter regex conditions run under a per-match timeout now, closing a denial-of-service hole. The old create-time heuristic could be bypassed by a catastrophic-backtracking pattern (for example ([a-z]+)*), and because matching ran synchronously on the event loop during fetch, filter tests and retroactive apply, and CPython's re neither times out nor releases the GIL, a single crafted filter could freeze the whole app for every user. Evaluation now uses the regex module with a hard timeout; a timed-out pattern counts as no match, and normal filter behaviour is unchanged.