Skip to content

v0.3.10

Choose a tag to compare

View all tags
@justin-stanley justin-stanley released this 03 Oct 21:34
· 49 commits to main since this release
v0.3.10
3061097
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

FeatherReader 0.3.10 replaces 0.3.9, which is yanked: it crash-looped on startup against any existing database (no such column: kind). 0.3.10 is everything 0.3.9 contained, with that fixed and with upgrade tests that would have caught it. Upgrade from 0.3.8 or earlier straight to 0.3.10.

The release is mostly hardening. It closes an SSRF gap in the IPv6 address guard, puts a byte bound on every response body in the atproto layer, and makes record walks fail closed instead of returning a short list as if it were complete. It also lands the storage and retention half of standard.site publications. Nothing polls a publication yet, so readers see no change there.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

  • One additive schema change, applied automatically at startup. #184 adds a feeds.kind column (TEXT NOT NULL DEFAULT 'rss') and an idx_feeds_kind index, and every row's kind is re-derived from its URL on each start.
  • Rolling back to 0.3.8 is safe. 0.3.8 never reads the column, and its default keeps 0.3.8's inserts valid. The next 0.3.10 start corrects any rows added while rolled back. This was rehearsed on a fork of a production volume: 0.3.10 upgraded it, then 0.3.8 booted on the result with db: ok.
  • What to expect on first boot: if you have at:// subscriptions, one feeds.kind re-derived log line, with to_unpollable counting them. Without any, nothing is logged. PRAGMA table_info(feeds) showing kind is the real check.
  • New optional setting: FEATHERREADER_PUBLICATION_RETENTION_DAYS, default 3650. It bounds standard.site publication entries only.
  • One retention behaviour change. The sweeper treats retention as fully disabled only when all three retention settings are 0. An instance running RETENTION_DAYS=0 RETENTION_HARD_DAYS=0 now starts a daily sweep. On an RSS-only instance that sweep deletes nothing.
  • No fly.toml changes.
  • Do not deploy the 0.3.9 image (sha256:6a3c399660a45b75f6556ac93a6c12ec33cd57a4c72de25876458d5fbf379545). It stays on ghcr.io because the registry has no yank.

Fixed in 0.3.10

  • 0.3.9 failed to start against any existing database (#219). The base schema created idx_feeds_kind before the migration that adds the kind column. On an existing database the table already exists, so the column was missing and startup failed before migrations ran. The index is now created right after the column. The crashed boot wrote nothing, so a database that hit it is still a clean 0.3.8 one and upgrades normally.
  • Upgrade tests from released schemas (#219). Every test used to start from an empty database, which is why none could see this. Two new tests start from schemas the v0.3.8 and v0.2.0 binaries actually created. Each requires the upgraded database to match a fresh one exactly: every column and every index. A separate review built all 19 tags from v0.2.0 to v0.3.9, and 0.3.10 upgraded every one.

Security (from 0.3.9)

  • IPv6 addresses that embed a forbidden IPv4 address are refused (#210). The SSRF guard unwrapped only the IPv4-mapped and IPv4-compatible forms. NAT64, 6to4, IPv4-translated, Teredo and ISATAP addresses got through. For example, 64:ff9b::a9fe:a9fe and 2002:a9fe:a9fe:: both reach the cloud metadata service.
  • Every response body in the atproto layer is capped (#192). Five were read without any byte limit. These were the sidecar's /internal/repo proxy, the DID document fetch (whose host a did:web chooses), resolveHandle, and two sidecar session reads.
  • A listRecords body is bounded before it is parsed (#201), and each page is parsed once rather than twice (#194). One crafted 8 MiB response previously retained about 824 MB on a 512 MB machine.
  • Record walks are bounded in retained bytes across all four walks (#193), not only in record count.
  • A record walk that runs out of pages now refuses (#200). It used to return a truncated list as a success, which could wipe a reader's subscription projection down to the partial list.
  • The error-envelope guard catches a non-string error (#194). For example, {"error":404,"records":[]} no longer reads as a healthy empty page.
  • Sidecar dependency advisories:

Changed (from 0.3.9)

  • standard.site publications are retained by count, not by age (#206). Measured against three real publications, the 14-day age window stored zero entries. Long-form publishing isn't news-paced.
  • A feed records what kind it is (#184), so SQL and the fetcher can't disagree about whether a row is pollable.
  • cargo doc is a CI gate (#214), and the 45 warnings behind it are fixed.

Fixed (from 0.3.9)

  • An at:// URI is recognised regardless of the case of its scheme (#183). Feed-ceiling usage now shows on /admin/metrics.
  • An oversized retention setting no longer silently kills the sweeper. A value too large to be a date panicked the sweeper task. That pass is now disabled, with a warning naming the setting.
  • Publication entry dates are stable (#186). An undated document is dated from its record key's TID, and a date in the future is discarded.
  • feeds.kind is re-derived in both directions on every start (#189). Taking a feed out of the poller clears its orphaned backoff. An unreadable feeds row is skipped with a warning instead of blocking startup.
  • A certificate test no longer fails on slow first connections (#199). It now retries a timeout instead of treating latency as a verdict about the certificate.

Added (from 0.3.9)

  • standard_site::store_publication (#202), the storage half of publication support, with the poll semantics earlier review rounds found.

Known, not fixed here

  • The record-walk memory budget applies per walk, and walks nest, so the real process ceiling is a multiple of one walk's budget.
  • An undated entry sorts last in the reading list while being safe from eviction (#187, pre-existing).
  • Follow-ups from the release review: #217 (the SSRF guard misses a few reserved ranges) and #218 (.dockerignore hardening).

Artifacts

  • crates.io: feather-reader 0.3.10
  • Container: ghcr.io/justin-stanley/feather-reader:0.3.10 = ghcr.io/justin-stanley/feather-reader@sha256:897db46629a95f6dad6c655282fe274701172ab2add43c6df44af5a898894a90, with SLSA build provenance. Verify with gh attestation verify oci://ghcr.io/justin-stanley/feather-reader@sha256:897db46629a95f6dad6c655282fe274701172ab2add43c6df44af5a898894a90 -R justin-stanley/feather-reader, and deploy by digest.

Full diff: v0.3.8...v0.3.10

Assets 2
Loading