Skip to content

Benchmark Plan

mavrovde edited this page Nov 1, 2026 · 2 revisions

Benchmark Plan

The stack decision among Go, Rust and TypeScript is made on evidence. This page describes the benchmark that provides it. It has not been run yet; results will be published here and committed to the repository when it has. The comparison it tests is on Stack Comparison.

Principles

  • Never against production. A throwaway PostGIS database with production-shaped fixtures: one time partition of road situations, plus the charging and place views.
  • Equal constraints. Each candidate runs in a container limited to one CPU and 256 MiB of memory, with a database pool of four connections; the TypeScript candidate runs as one Node process. A second, squeeze run repeats the load steps at 128 MiB for all three and records whether each one survives and how its latency changes.
  • Separate load generator. k6 in its own container.
  • Repeatable. One script runs everything and writes a dated JSON result file; anyone can rerun it.
  • Same skeleton depth. Each candidate implements the same two features through its own store adapter, deadline executor and admission controller, as described on its architecture page.

The two queries

  1. Charging stations within a radius, nearest first (spatial index), returned as GeoJSON.
  2. Situations on one motorway in a six-hour window, on time-partitioned data, with a cursor.

Checks and thresholds

# Question Pass threshold What it decides
1 Cancellation: a stub that sleeps for five seconds must answer within 2 s + 50 ms, and the database backend must be idle within 200 ms, on the timeout path and the client-disconnect path; 1 000 iterations without ever reusing a poisoned connection 100 % Go's native cancel vs Rust's cancel guard vs TypeScript's CancelRequest helper; any leak eliminates the candidate
2 Latency: p50, p95 and p99 for both queries at three load steps, from light to well above the expected early load p95 within the budgets on [[Performance and Caching Performance-and-Caching]]; 0 % errors
3 Overload: a small steady critical stream plus a bulk flood far above capacity, with the pool reduced to two critical p95 within budget; bulk answered with 429 and Retry-After; nothing past 2 s the admission controller in each language, and whether Node's single loop protects the critical tier
4 Footprint: resident memory idle and under the highest load step; cold start; image size; survival in the squeeze run under load: Go ≤ 64 MB, Rust ≤ 48 MB, TypeScript ≤ 160 MB; start under 1 s whether Go's collector is comfortable under 128 MiB, and whether TypeScript fits the shared-host budget
5 Round trips: Go's single batched round trip vs the explicit-transaction forms (Rust: transaction; TypeScript: three round trips, since pg does not pipeline) ≤ 0.3 ms difference at p50 the design of each query-execution helper
6 Caching: 304 rate on repeated polls; single-flight under 50 concurrent identical requests 304 ≥ 95 %; exactly one database query the caching design
7 Agent productivity: one AI agent implements a third feature (refuel near a point) end to end in each skeleton: adapter, mapping, use case, DTO, OpenAPI, tests wall time, compile-and-fix iterations, review findings the effort estimates and the fluency risk
8 CI time to green, cold and warm cache recorded operating cost

All thresholds are design targets chosen before the run. The footprint targets differ per stack on purpose: each is the estimate on its architecture page with a margin, and all three are well inside the shared-host budget on Technology Evaluation.

Decision rule

  1. A candidate that fails check 1 or check 3 is out.
  2. Among Go and Rust, the preferred candidates: pick Go unless it fails check 2 or 4 where Rust passes, or check 7 shows Rust's agent velocity within 15 % of Go's.
  3. TypeScript is picked only if the owner widens the choice beyond the preferred candidates, it passes checks 1 to 4, and check 7 shows it at least 15 % faster than the Go or Rust candidate otherwise chosen.

See Stack Comparison for the reasoning and the sensitivity of the ranking.

Baseline for context

PostgREST may be run beside the three candidates as a no-code baseline for the two queries. It shows how much of the latency belongs to the database rather than to any language; it does not take part in the decision.

After the decision

The benchmark scripts stay in the repository. The latency budgets become a CI leg that asserts them on fixtures, and the overload scenario becomes the acceptance test for priority admission. See Testing and Quality.

Home

About

Design

Data Model

Technical specification

Testing

Engineering

Technology choice

Reference

Clone this wiki locally