get_analytics ignores bucket_ms; wide windows time out and force studio to cap the range picker

[dawsontoth]

Filed from studio-side investigation (HarperFast/studio#1576 and its follow-ups). Reported by a customer (IBM, it-mil-1.stage2.ibm-nonprod, harper-pro 5.1.22) and reproduced on our own anvils.acme-inc.stage cluster.

Summary

get_analytics accepts a bucket_ms parameter but appears to ignore it — rows come back at raw emission cadence regardless. On any window wider than about a day this makes the response large enough that studio's Status tab times out, and studio has no client-side remedy because the rows arrive before it can bucket anything.

What studio sends

Studio sizes bucket_ms to the selected window so a chart lands in a sane number of points:

range	bucket_ms sent	points wanted
1h	60 000	60
6h	120 000	180
24h	600 000	144
7d	3 600 000	168
30d	14 400 000	180

What comes back

Rows at the raw emission cadence, with period: 0. On the reporting instance that cadence was 90 s (despite analytics.aggregatePeriod: 60 being configured — possibly a separate bug worth its own look).

Verified against real get_analytics output over a 24 h window, both nodes, for mqtt-connections:

i5w-us-west-1.stage2...   rows=960  min=283  max=294  period=0
vk5-it-mil-1.stage2...    rows=228  min=1    max=5    period=0

960 = 86400/90 exactly — one row per emission period, no aggregation applied, and bucket_ms of 300 000 had no effect on the row count or spacing.

Consequences

1. Wide windows time out. On anvils.acme-inc.stage (2 nodes, 5 databases) a 7 d Storage tab reliably produced AxiosError: timeout of 60000ms exceeded in the browser console. 30 d does not return in any usable time. Row math for database-size alone at 30 d:

30d / 90s = 28,800 periods x 2 nodes x 5 databases = ~288,000 rows

and the Storage tab requests several metrics in parallel. A cluster with a realistic table count would be far worse — table-size is per (node, database, table).

2. Silent truncation. Studio caps responses at 50 000 rows and tail-keeps, so when a wide window does return, the chart shows only the most recent slice — roughly 5 days of a 30-day window — with no indication. That is a studio bug we are fixing, but it only exists because the payload is unbounded.

3. period: 0 breaks bucket inference. With no aggregation period stamped, studio falls back to a guessed bucket size (60 s), which mismatched the actual 90 s cadence and caused a rendering artifact serious enough to look like a production incident — the customer's Connections chart showed repeated dives to ~0 that never happened (HarperFast/studio#1576). Stamping a real period on gauge rows would remove the guess entirely.

Asks

1. Honor bucket_ms, or document that it is unsupported so clients can stop sending it and gate their UI accordingly.
2. If server-side downsampling is not planned, is there a supported way to get aggregated analytics over a wide window? Right now the only option is to transfer every raw row.
3. Stamp a real period on emitted rows (at minimum on gauge metrics like mqtt-connections / ws-connections / database-size), rather than 0.
4. Clarify whether the latency is scan cost, serialization, or transfer. This decides whether honoring bucket_ms makes wide windows fast or merely smaller — we cannot tell from the client, and it determines how much studio can re-enable.
5. Unrelated but noticed: analytics.aggregatePeriod: 60 was configured on an instance whose rows landed 90 s apart. Possibly worth checking independently.

Studio-side follow-up

Until bucket_ms is honored, studio is capability-gating its range picker: it projects the row count a preset would return from the observed row density and disables presets that would blow the cap, so 7 d / 30 d are unselectable on builds that ignore the hint rather than hanging. We would rather re-enable them.

Related: HarperFast/studio#1576, HarperFast/studio#1587, HarperFast/studio#1588.