Skip to content

v0.8.7

Latest

Choose a tag to compare

@Bigred97 Bigred97 released this 20 May 21:58
· 14 commits to main since this release

[0.8.7] — 2026-05-19

Fixed

  • Parsed-DataFrame cache below the byte cache — 12x speedup on warm
    F-table reads
    (Bug 7, 2026-05-19 latency diagnostic). _get_data_impl
    previously called pd.read_csv on the cached CSV bytes for every
    invocation. The byte cache hit in <10 ms but parsing the 300 KB F1
    CSV with ragged-row handling + numeric coercion took 5+ seconds on a
    shared CPU. Net effect: rba.F1.latest() p50 = 5.2 s warm, which
    set the floor for the ausdata-api /v1/real-cash-rate p50 (4.75 s
    warm) and bumped p99 to 6.5 s.

    Added an LRU-bounded (csv_filename, md5(body)) -> (header, DataFrame)
    cache layer between the byte cache and the parser. Bounded at 10
    entries (~300 MB worst case at 30 MB/parsed-frame). Cache is
    module-global (shared across worker threads) because every thread
    parses the same upstream bytes to the same DataFrame — sharing the
    parse is a strict win on the gateway hot-path. Fresh upstream
    publications miss cleanly via the md5 key.

    Also routes describe_table() through the shared cache so describing
    a recently-fetched table is ~10 ms instead of seconds.

    New reset_df_cache_for_tests() helper for clean test isolation
    (mirrors asic-mcp's parquet-cache reset pattern). New
    tests/test_df_cache_hits_on_same_csv.py pins the cache behaviour:
    N identical-bytes calls trigger exactly 1 parse; differing bytes
    re-parse; LRU caps at _DF_CACHE_MAX_ENTRIES; reset clears state.

    Expected impact at ausdata-api: /v1/real-cash-rate p50 drops from
    4.75 s -> <500 ms; p99 from 6.5 s -> <1.5 s.