Skip to content

Performance and Scalability

Jump to bottom
Frank Yglesias Bertheau edited this page Jul 7, 2026 · 1 revision
<h1>ᚾ&nbsp;NornGate — Performance &amp; Scalability</h1>

NornGate is a blocking runtime: gates sit in front of execution on purpose, so latency is a cost the platform owns rather than hides. This page states the numbers, where they come from, how the system scales, and what happens under load.

All figures below are internal benchmark reference numbers / SLO targets, not contractual SLAs. Validate against your own workload.

ᛋ Performance as a deliberate cost

The value of NornGate is the blocking gate — so the design question is not "how do we avoid latency" but "how do we keep the latency of a full five-gate check small and predictable." This is the deliberate inverse of a never-block-inference design (Kaman), where controls fire-and-forget off the hot path. NornGate blocks; it therefore publishes and defends the latency rather than externalizing it.

Reference numbers:

  • ~150ms — full pipeline p95 (automated path)
  • ~12ms — average per-gate latency
  • <1ms — Wasm filter overhead per gate
  • ~50K req/s — per cluster

ᛖ Latency — where the time goes

The p95 is dominated by the sandbox; every other gate is cheap. Illustrative budget for an automated-path request:

Stage p95 contribution
G0 Ingress (Wasm) ~1ms
G1 Policy (cached decision) ~5ms
G2 Approval (automated consent) ~2ms
G3 Sandbox (warm microVM pool) ~120ms
G4 Arbitration ~10ms
Mesh + mTLS + routing overhead ~12ms
Total (automated path) ~150ms

References. Firecracker (microVM boot / warm pools), Redis (rate limiting), Kafka (partition scaling), Envoy/Istio HPA + KEDA (autoscaling), never-block-inference contrast — Kaman. Naming doctrine: Prose Edda / Poetic Edda, per Norse Cosmology & Platform Design.

ᚾ NornGate — Performance & Scalability

NornGate is a blocking runtime: gates sit in front of execution on purpose, so latency is a cost the platform owns rather than hides. This page states the numbers, where they come from, how the system scales, and what happens under load.

All figures below are internal benchmark reference numbers / SLO targets, not contractual SLAs. Validate against your own workload.

ᛋ Performance as a deliberate cost

The value of NornGate is the blocking gate — so the design question is not "how do we avoid latency" but "how do we keep the latency of a full five-gate check small and predictable." This is the deliberate inverse of a never-block-inference design (Kaman), where controls fire-and-forget off the hot path. NornGate blocks; it therefore publishes and defends the latency rather than externalizing it.

Reference numbers:

  • ~150ms — full pipeline p95 (automated path)
  • ~12ms — average per-gate latency
  • <1ms — Wasm filter overhead per gate
  • ~50K req/s — per cluster

ᛖ Latency — where the time goes

The p95 is dominated by the sandbox; every other gate is cheap. Illustrative budget for an automated-path request:

Stage p95 contribution
G0 Ingress (Wasm) ~1ms
G1 Policy (cached decision) ~5ms
G2 Approval (automated consent) ~2ms
G3 Sandbox (warm microVM pool) ~120ms
G4 Arbitration ~10ms
Mesh + mTLS + routing overhead ~12ms
Total (automated path) ~150ms

Hot path vs human path. The ~150ms covers requests that clear automated approval. When G2 requires a human, the request enters an async pending state (up to the 30s timeout, then 408) and is not counted in the hot-path p95 — a human decision and a millisecond gate are different regimes. Keeping human transits under ~5% (auto-approval for low-risk work, via the approval budget) is what preserves the number.

Timeouts are deny ceilings, not latencies. The per-gate timeouts (G0 5s, G1 10s, G2 30s, G3 60s, G4 10s) bound failure, not expected latency; a gate that exceeds its budget denies. Per-realm route timeouts (Muspelheim 300s) are separate again — upstream limits for long jobs, not gate cost. See [The Spine](The-Spine).

ᛒ Scaling model

Each component scales on a different axis, so there is no single throughput wall.

Component Scaling axis Mechanism
G0 / G1 (stateless gates) Replicas HPA on CPU / RPS
G2 human approval Async — not latency-bound Queue; off the hot path
G3 Sandbox Warm-pool depth Pre-warmed Firecracker microVM pool (KEDA on queue depth)
G4 Arbitration Conflict domains Shard by resource key
Urd ledger Partitions Kafka partition count
Whole cluster Horizontal Multi-cluster per realm / per tenant (~50K req/s each)

The spine is distributed, not a single proxy. "One control path" is a logical invariant enforced by an Envoy sidecar on every workload plus an HA istiod control plane — there is no single proxy all traffic funnels through, so no physical SPOF. Its failure semantics are covered under bottlenecks and degraded mode, below.

ᛁ Caching and cold-start

Determinism is what makes caching correct. Because a verdict is a pure function f(request, policy_snapshot, state_snapshot, config) ([Deterministic Execution](Deterministic-Execution)), identical inputs must produce an identical verdict — so a decision can be cached keyed on the snapshot IDs and reused safely until policy or state changes. The cache is not a heuristic shortcut; it is exact.

  • Policy-decision cache — keyed on (request, policy_version, state_snapshot); invalidated when the policy bundle or relevant state changes.
  • mTLS session resumption — drops handshake overhead from ~5–10ms to <1ms after the first connection.
  • Edge caching (Alfheim) — read-only responses cached at the edge (50–100ms saved).
  • Sandbox cold-start — a cold Firecracker microVM boots in ~125ms; a warm pool serves requests from an already-booted VM, removing that boot cost from the hot path.

ᚺ Bottlenecks and degraded mode

Known bottlenecks and their mitigations:

Component Main bottleneck Mitigation
Ingress Rate-limit store Redis clustering
Policy Evaluation latency Cached (deterministic) decisions
Approval Human throughput Auto-approval + approval budget
Sandbox VM startup Warm microVM pools
Arbitration Distributed locks Sharded conflict domains
Audit ledger Partition throughput Kafka partition scaling

Backpressure over collapse. Load is shed deliberately: G0 rate limits return 429, and Skuld forecasting watches queue depth so saturation is throttled at the edge rather than propagated inward.

Degraded mode is graded by risk, not binary. Fail-closed does not mean "deny everything on any hiccup." When a central dependency is impaired:

  • Read-only / pre-authorized / cached-decision actions continue under the last valid snapshot.
  • Anything requiring fresh G1/G2 evaluation denies (safe failure).
  • The mode change is itself written to Urd.

So a central-service outage safely stops new consequential actions fleet-wide while low-risk traffic continues — availability preserved by risk class, never by opening the gates ([Security & Fail-closed](Security-and-Fail-Closed)).

ᛏ Service-level objectives

Component SLO targets with alert thresholds:

Component Latency (p95) Availability Alert
Data plane (Envoy) <5ms 99.99% >10ms for 5m
Control plane (istiod) <100ms (config push) 99.95% >1s for 5m
mTLS handshake <10ms 99.99% >50ms for 5m
Distributed tracing (export) <5ms 99.9% >20ms for 5m
Routing (VirtualService) <1ms 99.99% >5ms for 5m

SLO, not SLA. These are internal reliability targets that drive alerting, not contractual guarantees. Treat the latency and throughput figures as a starting envelope and re-measure on your hardware, model mix, and sandbox workload — the sandbox stage is the variable that moves the p95 most.

ᚱ Next steps

Iconography

Section glyphs are Elder Futhark runes (Unicode Runic block, U+16A0–U+16FF) — semantic, not emoji. Full set on [Home](Home#iconography):

Rune Name Gloss Marks
ᚾ Nauðiz need, constraint the platform mark
ᛋ Sowilō the sun — energy, throughput performance as a cost
ᛖ Ehwaz the journey, the road travelled latency
ᛒ Berkanan growth, increase scaling model
ᛁ Isa ice, the fixed and repeatable caching & cold-start
ᚺ Hagalaz hail, the load storm weathered bottlenecks & degraded mode
ᛏ Tiwaz order, measured guarantees service-level objectives
ᚱ Raidō the ride, the road next steps

References. Firecracker (microVM boot / warm pools), Redis (rate limiting), Kafka (partition scaling), Envoy/Istio HPA + KEDA (autoscaling), never-block-inference contrast — Kaman. Naming doctrine: Prose Edda / Poetic Edda, per [Norse Cosmology & Platform Design](Norse-Cosmology-and-Platform-Design).

Welcome

Getting Started

Core Concepts

Guides

Reference

Silence means no.

Clone this wiki locally