v0.4.7

FeatherReader 0.4.7 limits what a hostile feed body can do:

• Article bodies are rendered through a type that can only be built by sanitizing. The reader no longer uses |safe.
• An article body the sanitizer is slow on can no longer stall the poller.

Full engineering detail is in CHANGELOG.md.

Upgrade notes

• No schema change and no new settings. Upgrading from 0.4.6 is a deploy, and rolling back to 0.4.6 is a redeploy. Both were rehearsed on a fork of a production volume: 0.4.7 booted with db: ok and served its pages, and 0.4.6 then booted on the result, also with db: ok.
• Rendering an article re-sanitizes its stored body, with an in-memory cache of 256 bodies / 8 MiB. Ordinary bodies take microseconds and render the same as before. A short note and a link to the original are shown instead of the body when:
  • the body is over 2 MiB;
  • it can't get one of the 2 render permits within 2 s;
  • it was already found slow to clean and has left the cache.
• A feed whose body takes over 5 s to sanitize:
  • keeps any bodies already stored;
  • stores new entries without a body;
  • doesn't save its ETag / Last-Modified;
  • backs off as a Body failure.
• /stats gains a row when polls are deferred because no sanitize permit was free. It shows that slow bodies' sanitizes are holding all four permits.
• Shutdown waits at most 5 s for blocking work, so a sanitize the poller gave up on can't hold a deploy past Fly's kill_timeout.

Security

• Stored article bodies no longer render with |safe (#273, closes #151).
  • The type: SanitizedHtml has a private field, and its only constructor runs ingest's own sanitizer. It implements askama's HtmlSafe. Two compile_fail doctests show a raw string can't become one.
  • Re-sanitized at render: a body written straight into the database, bypassing ingest, renders inert.
  • Bounded cost: re-cleaning sits behind a size cap, 2 permits, single flight and a SHA-256-keyed cache.
  • Slow bodies: a body whose clean used over 500 ms of the thread's own CPU time is cleaned at most once per process.
• A feed body could stall the poller for minutes (#274, closes #226).
  • Off the runtime: ingest sanitizes on the blocking pool, under 4 permits and a 5 s timeout. The permit wait is bounded too.
  • Per feed: each feed has at most one sanitize running, counted from its start. A feed whose sanitize timed out is refused until that sanitize finishes.
  • Deferred polls: a poll that can't get a permit is deferred, not blamed on its feed.

Known

• Four hostile feeds, for example four URL variants of one feed, can still hold all four sanitize permits until their sanitizes finish. Every other poll is deferred meanwhile. This now shows on /stats and in the logs, but it isn't prevented yet (#275).