Request timing analytics on the site Overview, with a link to the captured queries#797
Merged
Conversation
…alytics view The watcher already reads the nginx access feed to flag slow routes; it now also writes each request to a SQLite store in the data dir, buffered between save ticks and flushed in one batch, with rows past a seven day retention pruned hourly. The store is pure Go so the CGO-free build stays intact, and the existing in-memory windows keep driving the doctor and slow-route notifications unchanged. lerd-ui reads the same store over a WAL connection and serves a per-site analytics view for a chosen window, the response-time distribution, per-route p50 and p95, a status-class breakdown, per-minute throughput, and the most recent requests. This is the data behind an upcoming request-timing panel.
The Overview gains a Request timing section driven by the durable store: a range picker, KPIs for the typical and p95 response times, request count and error rate, a response-time distribution, a throughput chart, the slowest routes with click to profile, and a routes and recent-requests panel. The first request after a site has sat idle past the idle-suspend timeout is a cold start whose wake cost would skew the numbers, so it is recorded with a cold flag and kept out of every timing figure, the site and per-route percentiles and the distribution, while still counting toward the request total and showing in the recent list with a marker. Older stores gain the cold column through a lightweight migration.
…timing panel The slowest-routes list ranks and reads each route by its recent p95, the p95 over the route's most recent samples, so a route that was slow but has since been fixed drops down or off as newer faster requests arrive, even while its old slow samples still sit in the window's overall p95. The recent-requests list shows each call's clock time rather than a relative age. With the fuller analytics view now on the Overview, the compact request-timing panel is removed from the Debug tab, which is left with only the debug-bridge lenses, and the slow-route doctor hint and the sites documentation point at the Overview instead.
…put chart Static assets, and requests nginx serves without the app, judged by a static-asset file extension or a zero request time (manifest.json, robots.txt, service workers), are kept out of the request-timing view both at ingest and for anything already stored, so a page's dozens of asset requests no longer drown out its app routes. The per-route p50 was serialized away when it was exactly zero, which rendered as NaN in the routes table; it is always sent now. The throughput chart gains a labelled y axis and clock-time x labels, with points placed by their real time, so its numbers are readable.
When debug capture is on, each route in the timing view gains an Inspect queries button that opens the query evidence recorded against that exact route, matched on the same route key the timing groups by. A new endpoint gathers the captured query events for the route and returns each request's query count and total query time alongside its N+1 and slow-query findings, with the offending SQL and its caller, worst by query time first. Evidence exists only for requests hit while capture was on, so a route not yet exercised with the debugger opens empty with a prompt to reload it. This reuses the same analysis the optimize report and the N+1 notifier already run, so the timing view and the debugger agree on which queries belong to which route.
…filter The timing view's Inspect queries button no longer opens a modal that re-rendered the queries; it links to the Debug tab's Queries lens, the one place that renders captured queries, seeded with the route's path so the lens scopes to it. The per-route endpoint and the modal are removed, so there's a single source of truth for query rendering. The Debug lenses, Queries, Dumps, and the kind lenses, now share one search filter within a site's Debug view, so a search set in one lens, or seeded by the Inspect link, carries over when you switch lenses. The search matches the request path as well as the SQL, file, worker, and branch, and the filter box gains a clear button distinct from the Clear that wipes captured data.
geodro
changed the title
…ng windows The durable store already drops static assets and the zero-time requests nginx serves directly; the in-memory aggregator that feeds the doctor's typical response time, the slow-route notifications, and the MCP route_timing action now drops them the same way, so those stay consistent with the analytics view and a page's asset requests don't drag the baseline down.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a request-timing analytics view to the site Overview, driven by the always-on nginx access feed so it needs no debugger to use, and links a slow route to the queries behind it.
Every request nginx serves is written to a small SQLite store in the data directory, buffered off the access feed and flushed in batches, and pruned to a seven-day window. The driver is pure Go so the CGO-free build is unaffected, and the existing in-memory windows keep driving the doctor and slow-route notifications unchanged. lerd-ui reads the store back over a WAL connection to build the view for a chosen range, fifteen minutes up to seven days.
The Overview section shows the typical and p95 response times, the request count and error rate, a response-time distribution, a throughput chart, the slowest routes, and a table of every route with its p50 and p95, plus a recent-requests tab with each call's time, method, path, status, and duration. The slowest-routes list ranks each route by its recent p95, so a route that has been fixed drops down or off as newer faster requests arrive even while its old slow samples still sit in the window. Requests nginx serves without the app, static assets by extension and anything with a zero request time such as manifest.json, are left out so a page's asset requests don't drown its app routes. The first request after a site has sat idle past the idle-suspend timeout is a cold start whose wake cost would skew the numbers, so it is kept out of every timing figure while still counting toward the totals and marked in the recent list.
When debug capture is on, each route carries an Inspect queries button that jumps to the Debug tab's Queries lens filtered to that route, the one place that renders captured queries, rather than duplicating them in a modal. The Debug lenses now share a single search filter within a site's Debug view, and the search matches the request path so filtering by a route works.
Latency throughout is nginx's per-request time in milliseconds. With the fuller view on the Overview, the older compact request-timing panel is removed from the Debug tab.
Closes #796