Skip to content

v1.4.0 - Rendering off the request path

Choose a tag to compare

@ibuilder ibuilder released this 07 Aug 18:20
· 18 commits to main since this release

Track C: the work that decides whether ScopeMaker holds up under more than one
user at a time. Every change here was measured before and after.

Rendering leaves the request path

Render cache. Every export is keyed on a fingerprint of the document's
actual content — scope fields, project, bid package, every section and item.
An unchanged scope is served from stored bytes and renders nothing.

The obvious implementation, hashing updated_at, is wrong here. The edit routes
set updated_by_id to mark a scope dirty, and assigning the same user id is
not a change — so SQLAlchemy issues no UPDATE and onupdate never fires. One
person editing two items in a row would have kept the old fingerprint and been
served a document missing their edit.

Render queue. With RENDER_ASYNC=1 a render that is needed goes to a
worker (flask run-worker) and the request returns a small polling page instead
of holding a gunicorn worker for a second or two.

The queue is a database table, not Redis. This deployment already runs
PostgreSQL, and mandating a broker in order to download a PDF is a poor trade.
Workers claim jobs with a conditional UPDATE rather than FOR UPDATE SKIP LOCKED, so the same code runs on SQLite and PostgreSQL and several workers can
share one queue:

docker compose up -d --scale worker=3

A worker that dies mid-render leaves its job running; the next one requeues it
after ten minutes and gives up after three attempts. Async is off by default,
because turning it on without a worker means exports never complete.

The editor's N+1

Rendering a scope walked item.children per item — one query each. The tree is
now built from the already-loaded flat collection with set_committed_value,
which also stops SQLAlchemy re-fetching it.

Measured on 13 scopes / 843 items:

Page Before After
editor 167 ms, 73 queries 110 ms, 11 queries
DOCX export (repeat) 341 ms, 20 queries 13 ms, 7 queries

Optimistic locking

Scopes carry a row_version. Two people editing the same document now get an
explicit conflict rather than one silently overwriting the other.

Metrics

/metrics serves Prometheus text exposition — request counts and latency,
render timings by format, export cache hit rate, live queue depth. No
prometheus_client dependency; the text format is simple enough to emit
directly and most self-hosters will never scrape it.

The endpoint returns 404 until METRICS_TOKEN is set, then requires the
token. Requests are labelled by Flask endpoint, never by path: paths contain
scope ids, and one time series per document is how a Prometheus instance falls
over.

scripts/load_test.py builds a throwaway dataset and reports latency
percentiles alongside the query count per page — the number that actually
transfers from a laptop to production.

Fixed

  • Deleting a parent item promoted its children to the top level instead of
    deleting them, quietly corrupting the outline.
  • render_now never stamped started_at on the synchronous path, so those jobs
    reported no duration.

Upgrading

docker compose pull && docker compose up -d

One migration (a0e2627d3a55) adds the render_jobs table and
scopes.row_version. Nothing else changes; async rendering and metrics are both
opt-in.

Full changelog: https://github.com/ibuilder/scopemaker/blob/main/CHANGELOG.md