Skip to content

Releases: justin-stanley/feather-reader

v0.4.8

Choose a tag to compare

@justin-stanley justin-stanley released this 08 Oct 13:28
50e34f9

FeatherReader 0.4.8 hardens how it reads atproto repos:

  • A PDS that cuts a subscription list short can no longer drop a reader's subscriptions.
  • A publication read that fails is now filed under what the PDS actually sent, not as "unreachable".
  • A small publication that shares a repo with big ones is read in full.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • No schema change and no new settings. Upgrading from 0.4.7 is a deploy, and rolling back to 0.4.7 is a redeploy. Both were rehearsed on a fork of a production volume: 0.4.8 booted with db: ok and served its pages, and 0.4.7 then booted on the result, also with db: ok.
  • A subscription listing that repeats its cursor is refused, and the reader sees their last-known list, as for any listing failure.
  • A refresh that would drop 3 or more of a reader's feeds reads the list twice. It is applied only if both reads agree. Otherwise the reader sees their last-known list with an alert, and nothing is removed.
  • The /stats "why they are failing" counts shift for publications. Listing failures that were counted as fetch now count as parse, status or body, according to what the PDS sent.
  • Publications a repo's shared read starved are re-read alone, within the existing read deadline.

Security

  • A truncated subscription listing could remove a reader's access to feeds (#278, closes #203). That listing replaces the reader's whole sub_ref authorization set.
    • Repeated cursor: the three subscription walks now refuse a repeated cursor on a non-empty page.
    • Large drop: a page with a cursor and no records is how a real PDS ends a list, so the walk can't treat it as an error. Instead, a refresh that would drop 3 or more feeds must read the same list twice before it is applied.
    • Failed second read: the reader is told the real cause.

Fixed

  • Publication listing failures were all filed as Fetch (#279, closes #227). They are now typed and mapped:
    • an empty body or no records field is Parse;
    • a 2xx error envelope is Status;
    • too many pages, bytes or records, or an oversized body, is Body.
  • Big siblings could starve a quiet publication in the same repo (#281, closes #229). It was latent at measured scale.
    • When: a publication the shared walk cut short because it ran out of bytes is re-read alone after the group's reads are stored. The page limit, a repeated cursor and the combined cap don't trigger a re-read.
    • Order: starved publications are re-read first.
    • Bounds: only one budget is held at a time, and re-reads stay within the existing read deadline. A failed re-read keeps the group's outcome.

Known

  • A page carrying a malformed record still charges its whole wire size to a repo's shared read budget, siblings' documents included.
  • A few refusals are still filed as Fetch (#280).

v0.4.7

Choose a tag to compare

@justin-stanley justin-stanley released this 07 Oct 16:16
035cc9c

FeatherReader 0.4.7 limits what a hostile feed body can do:

  • Article bodies are rendered through a type that can only be built by sanitizing. The reader no longer uses |safe.
  • An article body the sanitizer is slow on can no longer stall the poller.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • No schema change and no new settings. Upgrading from 0.4.6 is a deploy, and rolling back to 0.4.6 is a redeploy. Both were rehearsed on a fork of a production volume: 0.4.7 booted with db: ok and served its pages, and 0.4.6 then booted on the result, also with db: ok.
  • Rendering an article re-sanitizes its stored body, with an in-memory cache of 256 bodies / 8 MiB. Ordinary bodies take microseconds and render the same as before. A short note and a link to the original are shown instead of the body when:
    • the body is over 2 MiB;
    • it can't get one of the 2 render permits within 2 s;
    • it was already found slow to clean and has left the cache.
  • A feed whose body takes over 5 s to sanitize:
    • keeps any bodies already stored;
    • stores new entries without a body;
    • doesn't save its ETag / Last-Modified;
    • backs off as a Body failure.
  • /stats gains a row when polls are deferred because no sanitize permit was free. It shows that slow bodies' sanitizes are holding all four permits.
  • Shutdown waits at most 5 s for blocking work, so a sanitize the poller gave up on can't hold a deploy past Fly's kill_timeout.

Security

  • Stored article bodies no longer render with |safe (#273, closes #151).
    • The type: SanitizedHtml has a private field, and its only constructor runs ingest's own sanitizer. It implements askama's HtmlSafe. Two compile_fail doctests show a raw string can't become one.
    • Re-sanitized at render: a body written straight into the database, bypassing ingest, renders inert.
    • Bounded cost: re-cleaning sits behind a size cap, 2 permits, single flight and a SHA-256-keyed cache.
    • Slow bodies: a body whose clean used over 500 ms of the thread's own CPU time is cleaned at most once per process.
  • A feed body could stall the poller for minutes (#274, closes #226).
    • Off the runtime: ingest sanitizes on the blocking pool, under 4 permits and a 5 s timeout. The permit wait is bounded too.
    • Per feed: each feed has at most one sanitize running, counted from its start. A feed whose sanitize timed out is refused until that sanitize finishes.
    • Deferred polls: a poll that can't get a permit is deferred, not blamed on its feed.

Known

  • Four hostile feeds, for example four URL variants of one feed, can still hold all four sanitize permits until their sanitizes finish. Every other poll is deferred meanwhile. This now shows on /stats and in the logs, but it isn't prevented yet (#275).

v0.4.6

Choose a tag to compare

@justin-stanley justin-stanley released this 06 Oct 17:58
aaa18bd

FeatherReader 0.4.6 stops renames from losing data:

  • Renaming a subscription can no longer erase another atproto client's edit made at the same moment.
  • Renaming a folder now changes only its name instead of rebuilding the record.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • No schema change and no new settings. Upgrading from 0.4.5 is a deploy, and rolling back to 0.4.5 is a redeploy. Both were rehearsed on a fork of a production volume: 0.4.6 booted with db: ok and served its pages, and 0.4.5 then booted on the result, also with db: ok.
  • Rename writes now carry swapRecord. A rename that races another client's edit is retried once against the fresh record, and the other client's change is kept. If it still conflicts, the reader sees "changed elsewhere … Reload and try again" instead of a false success. A folder rename that fails now shows an error. It used to redirect as if it had worked.
  • The manage page posts the values it showed (seen_url, seen_title, seen_folder, seen_name), so a change made elsewhere after the page loaded is detected too. A form posted without them still works.

Fixed

  • A subscription rename could erase another client's concurrent edit (#267, closes #149).
    • Every write path: putRecord takes swapRecord on the Rust OAuth client, the app-password client and the sidecar.
    • The rename: it writes against the CID it read. On InvalidSwap it re-reads and merges per field against what the reader first saw:
      • a field the reader didn't touch keeps the fresh value;
      • a field only the reader changed takes the reader's value;
      • a field both sides changed to different values is reported as a conflict.
    • Repeated saves: a double-submitted Save reports success and writes once.
    • Cache ordering: the local cache is updated only after the PDS write lands.
  • Renaming a folder overwrote the folder record (#269, closes #268). A rename rebuilt the record, which reset position, replaced createdAt with the rename time and dropped any field another client had added. Now it changes only name, keeps unknown fields through a catch-all on Folder, writes with compare-and-swap, uses the same merge-and-retry, and never re-creates a folder that was deleted elsewhere.

Known

  • Subscription has no catch-all for unknown fields either, so a subscription rename still drops fields the struct doesn't name. The fields it does name are kept. This is a follow-up.

v0.4.5

Choose a tag to compare

@justin-stanley justin-stanley released this 06 Oct 15:24
215b557

FeatherReader 0.4.5 lets an operator teardown revoke the rust backend's sessions before anything is deleted (#257). Production runs that backend. Before this release, a teardown wiped those sessions' tokens without revoking them, so they stayed valid at each user's PDS until they expired. Fixing that exposed races in how a token refresh and a sign-out write the session row, and those are fixed here too.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • No schema change, no new settings. Upgrading from 0.4.4 is a deploy, and rolling back to 0.4.4 is a redeploy. Both were rehearsed on a fork of a production volume: 0.4.5 booted with db: ok and served its pages, then 0.4.4 booted on the result, also with db: ok.
  • Session writes are now conditional. Login still writes unconditionally, and users see no difference.
    • A token refresh updates the session row only if the row still holds the tokens the refresh started from. It never re-creates a row that a sign-out deleted.
    • A sign-out deletes the row only if it hasn't changed since it was read.
    • A refresh that loses such a race revokes the tokens it could not store. If that revocation fails, a warn line says so.
  • --revoke-all-sessions signs every user out. It is an operator tool for teardown, not routine maintenance. See deploy/teardown.md.

Security

  • Teardown revokes the rust backend's sessions (#264, closes #257).
    • The new command. featherreader --revoke-all-sessions signs every stored session out at its own PDS, using RFC 7009 revocation through the same path as /logout.
    • Order matters. deploy/teardown.sh runs the sidecar revoke first, then a Rust revoke while the app is still serving, then stops the services, then runs a post-stop sweep, then wipes. The main pass has to run while the app is up: each PDS authenticates the revocation against the client metadata and JWKS the app serves, and caches them for only 10 minutes.
    • Pre-flight checks. Before anything is deleted, the command checks that it is the production client, that the encryption key decrypts the stored sessions, and that the signing key matches the JWKS the app serves. If any check fails it exits 2 and deletes nothing.
    • Partial failure. It exits 3 and prints a summary line, which teardown.sh requires before it will wipe.
    • Races closed. A refresh racing a sign-out can no longer resurrect a session or orphan a live token. A session whose issuer changes in the middle of a sign-out has its token revoked at its own issuer, never sent to another one.
  • The work went through seven review rounds. It is covered by tests against a fake authorization server over TLS, a shell test of teardown.sh (86 assertions), and 66 deliberate-break checks, all caught.

Docs

  • Every doc and number is up to date with the code (#256). Four audits checked every Markdown doc against the code, and a final sweep covered numbers and line references. The open risks the audits found are tracked as issues #258–#263.

v0.4.4

Choose a tag to compare

@justin-stanley justin-stanley released this 05 Oct 03:40
54373cc

FeatherReader 0.4.4 moves the feed parser to feed-rs 3.0. Entry ids and permalinks stay exactly as they were, RSS bylines now show real names, and a parser crash is contained as a parse failure. It also closes the reserved address ranges the SSRF guard missed.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • No schema change and no new settings. Upgrading from 0.4.3 is a deploy, and rolling back to 0.4.3 is a redeploy. Both were rehearsed on a fork of a production volume: 0.4.4 booted with db: ok, and 0.4.3 then booted on the result, also with db: ok.
  • Bylines change on each feed's next poll. RSS items that showed "author", and Atom authors that showed "unknown", get the real name, or no byline when the feed gives only an email address. Entry ids, titles, links, dates and content do not change. A side-by-side run of 2.4 and 3.0 over 27 live feeds (774 entries) found no difference in any of them.
  • A feed whose author field crashes feed-rs 3.0 now fails as a parse error and backs off, until the offending item leaves the feed. The trigger is a multi-byte character directly next to an email address. 2.4 parsed these feeds, and none of the 27 sampled feeds has one.

Changed

  • feed-rs 2.4 → 3.0 (#249):
    • Same ids and links: comments links are no longer picked as an entry's link or used for its id, and generated ids match 2.4's.
    • No more repeats: an item with no guid and no link now gets a stable id, instead of a random one that made it reappear on every poll.
    • Pinned dedup hashes: they use SipHash-1-3 explicitly, so a Rust toolchain upgrade can't change them, and the values already stored are unchanged.
    • More dates parsed: some formats 2.4 rejected are now read. The two-day future bound still applies.
    • Known regression in 3.0 itself, invalid feeds only: unescaped markup inside a title or description keeps only the text before the first tag.

Security

  • The SSRF guard refuses the ranges it missed (#254): 240.0.0.0/4, the IPv4 documentation ranges, fec0::/10, 100::/64 and 2001:db8::/32. It refuses them inside every IPv6 form it already unwraps too.
  • A local docker build no longer sends the app's OAuth signing key or .claude/ to the Docker daemon (#252). Shipped images were never affected.

Dependencies

tokio-rustls 0.26.6 (#248); the sidecar's npm group, including @atproto/api 0.22 (#250); CI action pins (#251).

v0.4.3

Choose a tag to compare

@justin-stanley justin-stanley released this 05 Oct 00:40
ed2e5ab

FeatherReader 0.4.3 fixes two problems on the path that writes to your PDS. They apply on any PDS, and were found while evaluating vlpds. The README is also rewritten for 0.4.x.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • No schema change and no new settings. Upgrading from 0.4.2 is a deploy, and rolling back to 0.4.2 is a redeploy. Both were rehearsed on a fork of a production volume: 0.4.3 booted with db: ok, and 0.4.2 then booted on the result, also with db: ok.
  • What to expect after deploying:
    • A reader whose read-state sync was already stuck recovers on the next flush, after one readState listing.
    • An OPML import of more than 200 feeds now succeeds, or reports how many feeds were saved.
  • deploy/upgrade-from is now 0.4.2, so the upgrade-boot gate tests from it.

Fixed

  • applyWrites is sent in calls the PDS accepts (#242, closes #240).
    • The limits: the reference PDS refuses more than 200 writes in one call, and older releases refuse request bodies over 150 KiB.
    • What broke before: an OPML import put every feed in one call, up to 500, and a read-state flush put every dirty feed in one call. Both could fail outright.
    • Now: writes go out in order, in calls of at most 200 writes and 128 KiB, stopping at the first failure. A partial failure reports how many writes landed.
  • A read-state flush that disagreed with the PDS no longer fails forever (#243, closes #241).
    • The cause: the flusher chose between create and update from a local flag. If a record existed when the flag said it didn't, every flush for that reader failed. That happened after a lost success response or a fresh local database. The reverse case, a record deleted elsewhere, failed the same way.
    • Now: it lists the reader's readState records once, corrects the flags, and retries once. It never loops.
    • Split flushes: if a split flush fails partway, the writes that did land are recorded, so nothing after them gets stuck.

Docs

  • The README describes 0.4.x (#244, #245). It covers standard.site publications, the real OAuth backend in production, the invite-only beta, the release pipeline, and four architecture diagrams. The diagrams are shown as light/dark images so they display in the GitHub mobile app and on crates.io.

Known

  • #246: a fresh or restored local database overwrites the reader's existing readState records on their PDS instead of merging them. This predates 0.4.3. #243's recovery path reaches this overwrite where it used to get stuck.

v0.4.2

Choose a tag to compare

@justin-stanley justin-stanley released this 04 Oct 22:08
0b1c967

FeatherReader 0.4.2 adds a public feature page for standard.site and a short list of recent releases. A feather-reader.com link posted to Bluesky or elsewhere now unfurls with a description and an image.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • No schema change and no new settings. Upgrading from 0.4.1 is a deploy, and rolling back to 0.4.1 is a redeploy. Both were rehearsed on a fork of a production volume. 0.4.2 booted with db: ok and served its link cards and share image. Then 0.4.1 booted on the result, also with db: ok.
  • Link cards build absolute URLs from FEATHERREADER_PUBLIC_URL, the setting OAuth already uses. When it's unset, the cards point at the development default. Set it to your public origin.

Added

  • A feature page at /standard-site (#237). It explains:

    • what a standard.site publication is;
    • what FeatherReader shows from one: title, date, link and a plain-text summary;
    • how to subscribe;
    • the limits.

    The how-to-subscribe part appears only when FEATHERREADER_STANDARD_SITE is on. The landing page, /about and the footer link to the page.

  • A "latest releases" section on /standard-site and the landing page (#237). It links each release's GitHub page and changelog entry. The list lives in one place, web::RELEASES, and a test requires its newest entry to match the crate version. A release can't ship without adding itself.

  • Link cards on every page (#238):

    • Each page: a meta description, a canonical URL, Open Graph tags and a summary_large_image Twitter card.
    • Public pages: their own title and description.
    • Signed-in pages: the generic site card, plus noindex. No handle or DID appears in the page head.
    • The share image: static/social-card.png, 1200×630, 69 KB, generated from a committed SVG by scripts/social-card.sh.

v0.4.1

Choose a tag to compare

@justin-stanley justin-stanley released this 04 Oct 20:57
955c607

FeatherReader 0.4.1 is a follow-up to 0.4.0. The public pages now explain standard.site publications. The subscribe form can now submit the DID form of a publication URI, which browsers refused in 0.4.0.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • No schema change and no new settings. Upgrading from 0.4.0 is a deploy, and rolling back to 0.4.0 is a redeploy. Both were rehearsed on a fork of a production volume: 0.4.1 booted with db: ok, then 0.4.0 booted on the result, also with db: ok.
  • What changes depends on FEATHERREADER_STANDARD_SITE:
    • Off: the subscribe input is the same as in 0.4.0. The landing and about pages describe publications and say this instance isn't accepting new publication subscriptions.
    • On: the pages and the subscribe form explain how to subscribe, and the input accepts at://did:….

Added

  • The website says what 0.4.0 does (#234). The landing page and /about describe standard.site publications alongside RSS feeds:

    • what a publication is;
    • what FeatherReader shows from it: title, date, link and a plain-text summary;
    • that the subscription is the same portable record as an RSS one;
    • how to subscribe.

    The subscribe form on /manage shows both accepted spellings: at://did:plc:…/site.standard.publication/… and the handle form.

Fixed

  • The DID form of a publication URI could not be submitted from a browser (#234). The subscribe input was type="url", and browsers' URL parser rejects at://did:plc:…/…: the colons in the DID are read as a port. With the flag on, the input is now type="text". A pattern still requires the address to start with http(s):// or at://, in any case and with surrounding whitespace allowed. Without it, example.com/blog would reach the server and come back as "Couldn't find a feed".

CI

  • The crate publish is now started by release-image, not by a workflow_run trigger (#233). crates.io trusted publishing refuses workflow_run, so v0.4.0's automatic crate publish failed and was finished by hand. The order is unchanged: if the upgrade-boot gate fails, no image is pushed and no crate is published.

v0.4.0

Choose a tag to compare

@justin-stanley justin-stanley released this 04 Oct 19:44
7fd5d04

FeatherReader 0.4.0 adds standard.site support. It reads standard.site publications (site.standard.publication and site.standard.document records in their authors' atproto repos) as subscriptions, alongside RSS. Publication subscriptions already stored start delivering as soon as this version runs.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • No schema change. Upgrading from 0.3.10 is a deploy, and rolling back to 0.3.10 is safe. Both were rehearsed on a fork of a production volume: 0.4.0 upgraded it and answered /health with db: ok, then 0.3.10 booted on the result, also with db: ok.
  • Two data fixes run at every start. Both do nothing once applied:
    • feeds.kind is re-derived from the URL, as since 0.3.9, now with a third kind, unsupported.
    • An entry stored with a published date more than two days in the future has that date cleared (#213).
  • FEATHERREADER_STANDARD_SITE controls what may be stored, not what is read. Stored publications are polled with the flag off too. With it on, a publication can also be pasted into the subscribe form.
  • New setting: FEATHERREADER_PUBLICATION_READ_DEADLINE_SECS, default 30, at least 1. It is the longest one publication read may take. It is kept under Fly's 45 s kill_timeout, so a read in flight at shutdown can finish.
  • Stricter setting: FEATHERREADER_POLL_INTERVAL=0 is now refused at startup.
  • A new background loop, the publication poller, starts 75 s after boot. FEATHERREADER_STARTUP_DELAY_SECS shortens that delay, as it does for the other loops.
  • No fly.toml changes.

Added

  • Publications are polled (#225). They have their own loop, separate from the RSS poller, so a slow publication never holds up RSS. Before each read the loop checks for shutdown and the DB-size watermark. Each row is read at most once per pass.
  • Each publisher's repo is read once per pass, however many of its publications are due (#228). That's up to 16 per read. Each publication has its own document cap, so a busy one can't crowd a quiet sibling out.
  • Subscribe from the form (#230), when the flag is on. Paste at://did:plc:…/site.standard.publication/… or the handle form at://alice.example.com/site.standard.publication/…. The handle is resolved to a DID before the subscription is stored. The first poll runs at once, so articles appear straight away.
  • A new feed kind, unsupported (#225), for any at:// row that is not a well-formed publication. Such rows are never polled, and /admin/metrics counts them as unpollable.

Security

  • Every stored field has a size bound (#224, closes #205). The bounds were set from production's real entries:

    Field Bound
    Title 5,000 bytes
    Author 1,000 bytes
    URL 8,192 bytes
    Body 2 MiB
    Entry id 2,048 bytes (a longer id becomes a stable hash)

    Over-long values are truncated, never refused. A body is sanitised first and bounded after.

  • One malformed record no longer costs a whole page (#224, closes #177).

    • In a reader's own repo, the listing is refused rather than silently dropping a subscription, and the reading and manage pages show an alert.
    • In a publisher's repo, the record is skipped, counted, and charged to the walk's byte budget.
  • A publication read has an overall deadline (#225), so a slowly paging repo can't hold up the publication loop.

Changed

  • Publication failures are filed under what happened (#225). On /stats, a deleted record, a tombstoned DID, RepoNotFound or RepoDeactivated now shows as status or parse. fetch is kept for answers that never arrived.

Fixed

  • A future-dated RSS item stayed first in the reading list for good, and the retention sweeps could never delete it (#188). Such dates are now discarded, and rows already stored with one are re-dated at startup.
  • An undated entry sorted to the bottom of every list while being treated as newest everywhere else (#187). Lists now order on COALESCE(published, fetched_at).

CI

  • An upgrade-boot gate runs before anything publishes. It boots the previous release's image to create and seed a database, then boots the candidate against that database, then boots the previous image again to prove rollback. In release-image.yml the gate runs before the push, and the image it tested is the image that's pushed. release-crate.yml now waits for release-image to succeed.

API changes (breaking, for users of the feather-reader crate)

  • atproto::AtProtoError::DidResolution gains a cause: atproto::DidResolutionCause field. The new type is marked #[non_exhaustive].
  • atproto::ListRecordsResponse gains malformed and wire_bytes, and atproto::RecordWalk gains malformed. Code that builds either one with a struct literal must set them.
  • New public items:
    • atproto::MalformedRecords
    • standard_site::fetch_repo, standard_site::NotAPublication
    • feed::poll_feed_by_kind, feed::poll_publication_group
    • store::due_feeds_of_kind, store::stagger_unscheduled

Known, not fixed here

  • #226: the HTML sanitiser (ammonia) takes quadratic time on some inputs, and it runs on the poller's task. This predates 0.4.0.
  • #227: some publication reads that got an answer but a bad one are still filed as fetch.
  • #229: publications from one repo share one byte budget per read. Big siblings could in principle starve a quiet one, though that can't happen at the sizes measured.

v0.3.10

Choose a tag to compare

@justin-stanley justin-stanley released this 03 Oct 21:34
3061097

FeatherReader 0.3.10 replaces 0.3.9, which is yanked: it crash-looped on startup against any existing database (no such column: kind). 0.3.10 is everything 0.3.9 contained, with that fixed and with upgrade tests that would have caught it. Upgrade from 0.3.8 or earlier straight to 0.3.10.

The release is mostly hardening. It closes an SSRF gap in the IPv6 address guard, puts a byte bound on every response body in the atproto layer, and makes record walks fail closed instead of returning a short list as if it were complete. It also lands the storage and retention half of standard.site publications. Nothing polls a publication yet, so readers see no change there.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • One additive schema change, applied automatically at startup. #184 adds a feeds.kind column (TEXT NOT NULL DEFAULT 'rss') and an idx_feeds_kind index, and every row's kind is re-derived from its URL on each start.
  • Rolling back to 0.3.8 is safe. 0.3.8 never reads the column, and its default keeps 0.3.8's inserts valid. The next 0.3.10 start corrects any rows added while rolled back. This was rehearsed on a fork of a production volume: 0.3.10 upgraded it, then 0.3.8 booted on the result with db: ok.
  • What to expect on first boot: if you have at:// subscriptions, one feeds.kind re-derived log line, with to_unpollable counting them. Without any, nothing is logged. PRAGMA table_info(feeds) showing kind is the real check.
  • New optional setting: FEATHERREADER_PUBLICATION_RETENTION_DAYS, default 3650. It bounds standard.site publication entries only.
  • One retention behaviour change. The sweeper treats retention as fully disabled only when all three retention settings are 0. An instance running RETENTION_DAYS=0 RETENTION_HARD_DAYS=0 now starts a daily sweep. On an RSS-only instance that sweep deletes nothing.
  • No fly.toml changes.
  • Do not deploy the 0.3.9 image (sha256:6a3c399660a45b75f6556ac93a6c12ec33cd57a4c72de25876458d5fbf379545). It stays on ghcr.io because the registry has no yank.

Fixed in 0.3.10

  • 0.3.9 failed to start against any existing database (#219). The base schema created idx_feeds_kind before the migration that adds the kind column. On an existing database the table already exists, so the column was missing and startup failed before migrations ran. The index is now created right after the column. The crashed boot wrote nothing, so a database that hit it is still a clean 0.3.8 one and upgrades normally.
  • Upgrade tests from released schemas (#219). Every test used to start from an empty database, which is why none could see this. Two new tests start from schemas the v0.3.8 and v0.2.0 binaries actually created. Each requires the upgraded database to match a fresh one exactly: every column and every index. A separate review built all 19 tags from v0.2.0 to v0.3.9, and 0.3.10 upgraded every one.

Security (from 0.3.9)

  • IPv6 addresses that embed a forbidden IPv4 address are refused (#210). The SSRF guard unwrapped only the IPv4-mapped and IPv4-compatible forms. NAT64, 6to4, IPv4-translated, Teredo and ISATAP addresses got through. For example, 64:ff9b::a9fe:a9fe and 2002:a9fe:a9fe:: both reach the cloud metadata service.
  • Every response body in the atproto layer is capped (#192). Five were read without any byte limit. These were the sidecar's /internal/repo proxy, the DID document fetch (whose host a did:web chooses), resolveHandle, and two sidecar session reads.
  • A listRecords body is bounded before it is parsed (#201), and each page is parsed once rather than twice (#194). One crafted 8 MiB response previously retained about 824 MB on a 512 MB machine.
  • Record walks are bounded in retained bytes across all four walks (#193), not only in record count.
  • A record walk that runs out of pages now refuses (#200). It used to return a truncated list as a success, which could wipe a reader's subscription projection down to the partial list.
  • The error-envelope guard catches a non-string error (#194). For example, {"error":404,"records":[]} no longer reads as a healthy empty page.
  • Sidecar dependency advisories:

Changed (from 0.3.9)

  • standard.site publications are retained by count, not by age (#206). Measured against three real publications, the 14-day age window stored zero entries. Long-form publishing isn't news-paced.
  • A feed records what kind it is (#184), so SQL and the fetcher can't disagree about whether a row is pollable.
  • cargo doc is a CI gate (#214), and the 45 warnings behind it are fixed.

Fixed (from 0.3.9)

  • An at:// URI is recognised regardless of the case of its scheme (#183). Feed-ceiling usage now shows on /admin/metrics.
  • An oversized retention setting no longer silently kills the sweeper. A value too large to be a date panicked the sweeper task. That pass is now disabled, with a warning naming the setting.
  • Publication entry dates are stable (#186). An undated document is dated from its record key's TID, and a date in the future is discarded.
  • feeds.kind is re-derived in both directions on every start (#189). Taking a feed out of the poller clears its orphaned backoff. An unreadable feeds row is skipped with a warning instead of blocking startup.
  • A certificate test no longer fails on slow first connections (#199). It now retries a timeout instead of treating latency as a verdict about the certificate.

Added (from 0.3.9)

  • standard_site::store_publication (#202), the storage half of publication support, with the poll semantics earlier review rounds found.

Known, not fixed here

  • The record-walk memory budget applies per walk, and walks nest, so the real process ceiling is a multiple of one walk's budget.
  • An undated entry sorts last in the reading list while being safe from eviction (#187, pre-existing).
  • Follow-ups from the release review: #217 (the SSRF guard misses a few reserved ranges) and #218 (.dockerignore hardening).

Artifacts

  • crates.io: feather-reader 0.3.10
  • Container: ghcr.io/justin-stanley/feather-reader:0.3.10 = ghcr.io/justin-stanley/feather-reader@sha256:897db46629a95f6dad6c655282fe274701172ab2add43c6df44af5a898894a90, with SLSA build provenance. Verify with gh attestation verify oci://ghcr.io/justin-stanley/feather-reader@sha256:897db46629a95f6dad6c655282fe274701172ab2add43c6df44af5a898894a90 -R justin-stanley/feather-reader, and deploy by digest.

Full diff: v0.3.8...v0.3.10