Skip to content

v0.9.4

Choose a tag to compare

@gordonmurray gordonmurray released this 20 Jul 14:58
· 37 commits to main since this release
083c913

Fixed

A query filter calling a function whose result can change between two identical requests no longer serves a stale cached result.

The exact result cache keys on the predicate's text. That is correct for a stable predicate like section = 'warnings' and wrong for one like _ingested_at < now(): the text is byte-identical on every request, so the cache kept replaying the first result set while the cutoff it described moved on, until the namespace was next written to. On a read-heavy namespace that could persist for a long time.

Such a predicate now bypasses the result cache in both directions, neither read nor written, so it always reflects the backend. Stable predicates, the common case, keep the cached fast path.

The decision comes from planning the predicate with the same planner and function registry LanceDB uses to run it, then walking the resulting expression tree, rather than matching on the predicate text. A column named now still caches normally, and the parenthesis-free CURRENT_TIMESTAMP spelling is caught even though it is not a function call until the planner rewrites it.

Worth noting for anyone reasoning about this themselves: the affected set includes now(), current_timestamp, and current_date, which the planner classifies as stable rather than volatile. Stable means fixed within a single query but free to differ between queries, which for a cache spanning queries is no safer than random().

Closes #89.

Docker: ghcr.io/gordonmurray/firnflow:0.9.4

Full changelog: v0.9.3...v0.9.4