v1.4.0 - Rendering off the request path
Track C: the work that decides whether ScopeMaker holds up under more than one
user at a time. Every change here was measured before and after.
Rendering leaves the request path
Render cache. Every export is keyed on a fingerprint of the document's
actual content — scope fields, project, bid package, every section and item.
An unchanged scope is served from stored bytes and renders nothing.
The obvious implementation, hashing updated_at, is wrong here. The edit routes
set updated_by_id to mark a scope dirty, and assigning the same user id is
not a change — so SQLAlchemy issues no UPDATE and onupdate never fires. One
person editing two items in a row would have kept the old fingerprint and been
served a document missing their edit.
Render queue. With RENDER_ASYNC=1 a render that is needed goes to a
worker (flask run-worker) and the request returns a small polling page instead
of holding a gunicorn worker for a second or two.
The queue is a database table, not Redis. This deployment already runs
PostgreSQL, and mandating a broker in order to download a PDF is a poor trade.
Workers claim jobs with a conditional UPDATE rather than FOR UPDATE SKIP LOCKED, so the same code runs on SQLite and PostgreSQL and several workers can
share one queue:
docker compose up -d --scale worker=3A worker that dies mid-render leaves its job running; the next one requeues it
after ten minutes and gives up after three attempts. Async is off by default,
because turning it on without a worker means exports never complete.
The editor's N+1
Rendering a scope walked item.children per item — one query each. The tree is
now built from the already-loaded flat collection with set_committed_value,
which also stops SQLAlchemy re-fetching it.
Measured on 13 scopes / 843 items:
| Page | Before | After |
|---|---|---|
| editor | 167 ms, 73 queries | 110 ms, 11 queries |
| DOCX export (repeat) | 341 ms, 20 queries | 13 ms, 7 queries |
Optimistic locking
Scopes carry a row_version. Two people editing the same document now get an
explicit conflict rather than one silently overwriting the other.
Metrics
/metrics serves Prometheus text exposition — request counts and latency,
render timings by format, export cache hit rate, live queue depth. No
prometheus_client dependency; the text format is simple enough to emit
directly and most self-hosters will never scrape it.
The endpoint returns 404 until METRICS_TOKEN is set, then requires the
token. Requests are labelled by Flask endpoint, never by path: paths contain
scope ids, and one time series per document is how a Prometheus instance falls
over.
scripts/load_test.py builds a throwaway dataset and reports latency
percentiles alongside the query count per page — the number that actually
transfers from a laptop to production.
Fixed
- Deleting a parent item promoted its children to the top level instead of
deleting them, quietly corrupting the outline. render_nownever stampedstarted_aton the synchronous path, so those jobs
reported no duration.
Upgrading
docker compose pull && docker compose up -dOne migration (a0e2627d3a55) adds the render_jobs table and
scopes.row_version. Nothing else changes; async rendering and metrics are both
opt-in.
Full changelog: https://github.com/ibuilder/scopemaker/blob/main/CHANGELOG.md