[0.8.7] — 2026-05-19
Fixed
-
Parsed-DataFrame cache below the byte cache — 12x speedup on warm
F-table reads (Bug 7, 2026-05-19 latency diagnostic)._get_data_impl
previously calledpd.read_csvon the cached CSV bytes for every
invocation. The byte cache hit in <10 ms but parsing the 300 KB F1
CSV with ragged-row handling + numeric coercion took 5+ seconds on a
shared CPU. Net effect:rba.F1.latest()p50 = 5.2 s warm, which
set the floor for the ausdata-api/v1/real-cash-ratep50 (4.75 s
warm) and bumped p99 to 6.5 s.Added an LRU-bounded
(csv_filename, md5(body)) -> (header, DataFrame)
cache layer between the byte cache and the parser. Bounded at 10
entries (~300 MB worst case at 30 MB/parsed-frame). Cache is
module-global (shared across worker threads) because every thread
parses the same upstream bytes to the same DataFrame — sharing the
parse is a strict win on the gateway hot-path. Fresh upstream
publications miss cleanly via the md5 key.Also routes
describe_table()through the shared cache so describing
a recently-fetched table is ~10 ms instead of seconds.New
reset_df_cache_for_tests()helper for clean test isolation
(mirrors asic-mcp's parquet-cache reset pattern). New
tests/test_df_cache_hits_on_same_csv.pypins the cache behaviour:
N identical-bytes calls trigger exactly 1 parse; differing bytes
re-parse; LRU caps at_DF_CACHE_MAX_ENTRIES; reset clears state.Expected impact at ausdata-api:
/v1/real-cash-ratep50 drops from
4.75 s -> <500 ms; p99 from 6.5 s -> <1.5 s.