Skip to content

perf(ui): /api/state still ~5s on remote store — serialize-to-parallel the remaining per-collection round-trips (follow-up to #28) #31

Description

@AusafG5

Follow-up to #28 (now fixed and shipped in 0.4.3 — thank you). The list_history_refs() change killed the fat-history-doc hotspot (~4.7s → ~227ms) and the shared MongoClient cache removed the per-request TLS/SRV handshake. /api/state on a remote store dropped from ~20–37s to ~5–8s.

The residual ~5s is no longer a payload problem — it's serial round-trips × RTT on a cross-region link. Timed the discrete ops the status path still runs (warm, cached client, remote store):

op time
list_history_refs() (the #28 fix) ~227ms
distinct(id_field, live_when) collection A ~1210ms
distinct(id_field, live_when) collection B ~920ms
distinct(id_field, live_when) collection C ~1000ms
cfgit_heads.find({env}) ~2230ms
cfgit_history.find({env}, {doc:0}) ~2130ms

Each is fast server-side (low-hundreds of docs) — the wall-clock is one full RTT per call, run sequentially, so it sums. On a low-latency/local store this is invisible; on a managed remote cluster it's the whole ~5s.

Suggestions (any subset)

  1. Parallelize the independent reads. The per-collection list_record_ids distincts + get_heads + list_history_refs have no data dependency on each other — issue them concurrently (thread pool, or asyncio+motor) and gather. ~5s → roughly max(single op) ≈ ~1s.
  2. Fold the per-collection distinct calls into fewer round-trips where the store allows (e.g. one aggregation returning (collection, id) grouped, instead of one distinct per tracked collection).
  3. Optional: a single status aggregation. For the Mongo adapter, most of status() (live ids + heads join) could be expressed as one server-side $lookup/$group pipeline, collapsing several client round-trips into one.

Net effect would take the remote-store UI from "noticeably laggy" to "instant." Local stores already are.

(Profiled on 0.4.3. Same standalone timing script from #28 available.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions