v0.4.0

FeatherReader 0.4.0 adds standard.site support. It reads standard.site publications (site.standard.publication and site.standard.document records in their authors' atproto repos) as subscriptions, alongside RSS. Publication subscriptions already stored start delivering as soon as this version runs.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

• No schema change. Upgrading from 0.3.10 is a deploy, and rolling back to 0.3.10 is safe. Both were rehearsed on a fork of a production volume: 0.4.0 upgraded it and answered /health with db: ok, then 0.3.10 booted on the result, also with db: ok.
• Two data fixes run at every start. Both do nothing once applied:
  • feeds.kind is re-derived from the URL, as since 0.3.9, now with a third kind, unsupported.
  • An entry stored with a published date more than two days in the future has that date cleared (#213).
• FEATHERREADER_STANDARD_SITE controls what may be stored, not what is read. Stored publications are polled with the flag off too. With it on, a publication can also be pasted into the subscribe form.
• New setting: FEATHERREADER_PUBLICATION_READ_DEADLINE_SECS, default 30, at least 1. It is the longest one publication read may take. It is kept under Fly's 45 s kill_timeout, so a read in flight at shutdown can finish.
• Stricter setting: FEATHERREADER_POLL_INTERVAL=0 is now refused at startup.
• A new background loop, the publication poller, starts 75 s after boot. FEATHERREADER_STARTUP_DELAY_SECS shortens that delay, as it does for the other loops.
• No fly.toml changes.

Added

• Publications are polled (#225). They have their own loop, separate from the RSS poller, so a slow publication never holds up RSS. Before each read the loop checks for shutdown and the DB-size watermark. Each row is read at most once per pass.
• Each publisher's repo is read once per pass, however many of its publications are due (#228). That's up to 16 per read. Each publication has its own document cap, so a busy one can't crowd a quiet sibling out.
• Subscribe from the form (#230), when the flag is on. Paste at://did:plc:…/site.standard.publication/… or the handle form at://alice.example.com/site.standard.publication/…. The handle is resolved to a DID before the subscription is stored. The first poll runs at once, so articles appear straight away.
• A new feed kind, unsupported (#225), for any at:// row that is not a well-formed publication. Such rows are never polled, and /admin/metrics counts them as unpollable.

Security

• Every stored field has a size bound (#224, closes #205). The bounds were set from production's real entries:

  Field	Bound
  Title	5,000 bytes
  Author	1,000 bytes
  URL	8,192 bytes
  Body	2 MiB
  Entry id	2,048 bytes (a longer id becomes a stable hash)

  Over-long values are truncated, never refused. A body is sanitised first and bounded after.

• One malformed record no longer costs a whole page (#224, closes #177).

  • In a reader's own repo, the listing is refused rather than silently dropping a subscription, and the reading and manage pages show an alert.
  • In a publisher's repo, the record is skipped, counted, and charged to the walk's byte budget.

• A publication read has an overall deadline (#225), so a slowly paging repo can't hold up the publication loop.

Changed

• Publication failures are filed under what happened (#225). On /stats, a deleted record, a tombstoned DID, RepoNotFound or RepoDeactivated now shows as status or parse. fetch is kept for answers that never arrived.

Fixed

• A future-dated RSS item stayed first in the reading list for good, and the retention sweeps could never delete it (#188). Such dates are now discarded, and rows already stored with one are re-dated at startup.
• An undated entry sorted to the bottom of every list while being treated as newest everywhere else (#187). Lists now order on COALESCE(published, fetched_at).

CI

• An upgrade-boot gate runs before anything publishes. It boots the previous release's image to create and seed a database, then boots the candidate against that database, then boots the previous image again to prove rollback. In release-image.yml the gate runs before the push, and the image it tested is the image that's pushed. release-crate.yml now waits for release-image to succeed.

API changes (breaking, for users of the feather-reader crate)

• atproto::AtProtoError::DidResolution gains a cause: atproto::DidResolutionCause field. The new type is marked #[non_exhaustive].
• atproto::ListRecordsResponse gains malformed and wire_bytes, and atproto::RecordWalk gains malformed. Code that builds either one with a struct literal must set them.
• New public items:
  • atproto::MalformedRecords
  • standard_site::fetch_repo, standard_site::NotAPublication
  • feed::poll_feed_by_kind, feed::poll_publication_group
  • store::due_feeds_of_kind, store::stagger_unscheduled

Known, not fixed here

• #226: the HTML sanitiser (ammonia) takes quadratic time on some inputs, and it runs on the poller's task. This predates 0.4.0.
• #227: some publication reads that got an answer but a bad one are still filed as fetch.
• #229: publications from one repo share one byte budget per read. Big siblings could in principle starve a quiet one, though that can't happen at the sizes measured.