Skip to content

Performance and Caching

mavrovde edited this page Nov 13, 2026 · 3 revisions

Performance and Caching

Answers must be fast and cheap within a small memory and connection budget. This page lists the design targets and the mechanisms that meet them. All figures here are design targets to be asserted by tests, not measurements.

Design targets

  • Hard ceiling: 2 seconds for every request, enforced in the service and in the database.
  • Most endpoints: p95 under 200 ms.
  • Repeated requests: answered with 304 Not Modified or from cache.
  • No sequential scan of time-series data by any endpoint, proven by query-plan tests.

Latency budgets per feature (design targets)

Feature p95 / p99
Places search, describe a point 120 / 300 ms
Conditions near a point, along a motorway, roadworks ahead 200 / 500 ms
Fuel, charging, parking near a point 200 / 500 ms
Fuel, charging, parking along a corridor 400 / 900 ms
Departures, disruptions, accessibility 250 / 600 ms
Weather risk 200 / 500 ms
Route briefing (composed) 800 / 1 500 ms
One page of history 400 / 1 000 ms
Any 304 10 / 30 ms

The budgets live in a file in the repository and are asserted by the benchmark leg on fixture data. Production behaviour is checked after each release against the same budgets.

HTTP caching from source freshness

  • Each source family has a freshness version that the platform maintains cheaply (a small table, never an aggregate over time-series data). The API reads it lazily, only while traffic exists.
  • ETag = a hash of the feature, the normalised query, the allowed source families, the versions of the families read and the language. A match is a 304 before admission, so it costs no database permit. It still counts against the quota.
  • Cache-Control: private, max-age per source family; a composed feature takes the minimum. Defaults (configurable):
Family max-age
Road conditions 60 s
Fuel prices 300 s (follows the source's terms)
Charging, parking 60 s
Departures 30 s
Weather 300 s
Places, driving rules 1 day
  • Vary: Authorization, Accept, Accept-Language, Accept-Encoding; Last-Modified is the newest "as of" time.

In-process result cache

  • Keyed by the same input as the ETag; holds encoded response bytes; bounded by a configurable byte budget; entries expire with the family's max-age.
  • Single-flight: many identical concurrent requests cause exactly one database read.
  • Because the allowed-families hash is part of the key, two keys with different licence permissions can never share a cached body.
  • An optional micro-cache at the edge is possible later, keyed on the URL and the key's plan.

Query design

  • Responses that are lists or geometry are assembled in the database (json_build_object, json_agg, ST_AsGeoJSON), which avoids decoding and re-encoding large shapes.
  • Fields the domain reasons over (lamps, severities, prices) are mapped into domain types; only pure geometry passes through as raw JSON.
  • Every view the API reads has an index that matches its access path (spatial indexes for "near" and "along"; B-tree indexes for road and time).
  • History requires a bounded time window, and the query keeps time-partition pruning. Keyset pagination over (time, row key); never OFFSET.
  • An EXPLAIN test runs every history query against a database with several time partitions, in both generic and custom plan modes, and asserts that no sequential scan touches a time partition and that only the partitions inside the window are read.
  • Prepared statements are cached per connection.

Payload

  • gzip and brotli above a minimum size; zstd optional.
  • fields= to select a subset of fields.
  • geometry=none|simplified|full.
  • Page size default 50, bounded by plan and configuration.

Connection pooling

  • A small application pool for reads and a smaller one for the access store, both well inside the database-side role limits.
  • No external pooler while there is one instance. PgBouncer is the planned option for a later multi-instance stage.

Memory (design estimates)

Candidate Expected steady resident memory Container limit
Go roughly 25–45 MB 128 MiB, with the Go memory limit set below it
Rust roughly 10–30 MB 64–96 MiB
TypeScript roughly 60–120 MB 256 MiB, with V8's old-space limit set below it

These are estimates from the design (runtime, cache budget, key cache, connections), to be confirmed by the benchmark. See Benchmark Plan.

Home

About

Design

Data Model

Technical specification

Testing

Engineering

Technology choice

Reference

Clone this wiki locally