Skip to content

NTNT v0.6.0 — Reviewed database changes and native live updates

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 31 Oct 00:44
v0.6.0
e8031af
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

NTNT v0.6.0 — Reviewed database changes and native live updates

NTNT 0.6.0 adds a SQL-first workflow for managing PostgreSQL and SQLite databases,
plus native Server-Sent Events for live browser updates. You can inspect an existing
database, review a target-bound plan, execute it with checks, and see what committed
when a deployment fails. Live events travel through your application's ordinary HTTP
routes and middleware, without adding a separate messaging service to a single-process app.

The release also adds opt-in HTTP rate limiting, trusted client-IP resolution,
realtime worker initialization before readiness, and more consistent imports and
function bindings.

Upgrading from 0.5.6: this is the next published version. The 0.5.7 version was
used during development and was not released separately; its work is included here.

Database changes you can review

Your schema remains SQL. Numbered .sql files are the authored history; optional
TOML sidecars add named requires and ensures queries. The new ntnt db commands
use that history to compare the live catalog with the expected applied prefix,
replay a candidate in isolated scratch, and produce a plan for an explicitly selected
database and target.

In a configured project:

ntnt db status --database app --target staging --json
ntnt db plan --database app --target staging --out plan.json --json
# An operator/deployment step supplies separate approval for this exact plan.
ntnt db migrate --database app --target staging --plan plan.json --json
ntnt db verify --database app --target staging --json

Use --json for versioned, machine-readable results rather than scraping progress
messages; the human CLI renders the same underlying evidence.

The plan file and its digest do not grant authority. Execution requires the exact
reviewed digest from a separately protected, declared secrets provider, then
revalidates the target and base state under migration ownership.

For supported transactional operations, each migration commits its SQL, required
checks, resulting-catalog verification and success ledger entry together. A failure
later in a batch preserves earlier commits and reports that prefix. If a COMMIT
acknowledgement is lost, the runner reconnects and inspects the ledger/catalog; it
never treats an uncertain outcome as permission to blindly rerun SQL.

Several distinctions are deliberate:

  • Pending migrations are not drift. Verification compares against the covered
    history, not whatever happens to be newest in the repository.
  • Edited or missing history, wrong targets, stale plans and incomplete inspection
    cannot silently become a clean result.
  • Empty replay proves a schema can be constructed, not that production rows satisfy
    a check. Reports count only assertions actually attempted.
  • SQLite rebuilds remain explicitly authored SQL. A guarded rebuild mode checks
    table counts, named indexes/triggers/views and foreign keys; it does not invent
    a data mapping or prove its business meaning.

Bring an existing database into the workflow

plan --adopt-through <migration-id> compares an existing schema with a reviewed,
replayable source prefix. Named adoption_checks run under the same owned execution
path before adoption commits. The runner records an observed checkpoint, not
fictional applied rows, and does not replay the old migration scripts on the target.
Recognized PostgreSQL v1 history can be imported with its original source checksums.

The customer/order rehearsal
runs old and new NTNT application readers through populated adoption and an upgrade,
checking exact retained values and compatibility for that fixture. It uses disposable
data and the same executor as the CLI.

See Database Migrations
for configuration, SQL/check coverage, approval and recovery. The initial adapters
cover PostgreSQL 16–18 and bundled SQLite 3.45-series. PostgreSQL migration transport
currently requires explicit sslmode=disable: use a trusted Unix socket or separately
secured local transport. Provision distinct, compatible scratch yourself.

Native live updates over SSE

A named broadcast bus can be shared by HTTP workers and in-process tasks. An SSE
route subscribes through the normal request/middleware path, so authorization stays
in the application route rather than a second server.

import { broadcast, publish, schedule } from "std/concurrent"
import { sse } from "std/http/server"

let ticks = broadcast("ticks")
get("/events", fn(req) { return sse(ticks) })

schedule(1000, fn() {
    unwrap(publish(ticks, map { "at": str(now()) }, map { "event": "tick" }))
})

listen(8080)

An open stream does not reserve an NTNT interpreter or an OS thread. Configurable
startup caps bound streams, topics, encoded event size and retained/in-flight
payload bytes. When a client cannot keep up, the runtime disconnects it rather than
silently skipping events. Stream exhaustion returns HTTP 503; publication limits
return Err.

Apps with a live setup-time bus or an explicit realtime listen map now wait for
all configured HTTP workers to initialize before binding or announcing readiness.
Initial setup errors fail startup with source-path diagnostics and a shared deadline.
Use "realtime": map {} to opt into this behavior when buses are created only during
requests. Buffered-only apps retain their existing listener-first startup.

SSE is process-local and nondurable. It does not provide replay, cross-process fan-out,
WebSockets or immediate revocation of an already-authorized stream. publish success
is not delivery acknowledgement; clients should reconcile authoritative state after
reconnect. Use ntnt run for streaming: the synchronous ntnt test server returns 501.

Try the live dashboard example.

Traffic controls at the request boundary

NTNT_RATE_LIMIT=on enables automatic rate limiting before the request body is read
or the handler runs. Defaults are 30 writes/minute and 600 dynamic reads/minute per
client. Route-specific limits replace the global budget for that route; explicit
check_limit(key, rate) calls handle account-, email- or other application-defined
keys even when automatic limiting is off.

The new req.client_ip trusts forwarding headers only when the TCP peer belongs
to configured NTNT_TRUSTED_PROXIES. Configure actual proxy peers or a dedicated
proxy network before enabling limits. Use client_ip for IP-based limits and audit
logs; the older req.ip field keeps its existing forwarding-header behavior.

Limits are off by default, process-local and reset on restart. They are not a
distributed quota service or volumetric DDoS protection. Automatic limiting is
not applied by the synchronous test/Intent server. The
rate-limit example
and agent guide
cover route policies, exemptions, proxy configuration and explicit-key tests.

More predictable application code and HTTP behavior

  • Imports retain their origin. Explicit ./ and ../ imports resolve relative
    to the file that declares them, including inside helpers, callbacks, tasks and
    jobs. A different worker or caller CWD no longer changes their meaning.
  • Function bindings determine calls. Context-dependent builtins now use the
    same value-resolution path for aliases, module-qualified calls and higher-order
    use. User functions and imports can shadow them consistently; the linter reports
    imports that hide route registration functions.
  • Shutdown has a bound. Async HTTP graceful drain uses the configured
    request-timeout window, with a one-second minimum, instead of waiting indefinitely
    for connections, then continues shutdown handling.
  • Secondary HTTP workers can enqueue jobs from request handlers. The enqueue
    path no longer silently returns Unit there; bootstrap suppression remains.
  • Static mounts can have an explicit extension allowlist.
    serve_static(prefix, dir, map { "extensions": ["css", "png"] }) limits that
    public tree. With the option enabled, unlisted and extensionless files, dotfiles
    and disallowed resolved symlink targets return 404 on both servers. Without the
    option, existing behavior is unchanged.
  • Markdown dependency security: pulldown-cmark is updated to 0.13.5 for
    RUSTSEC-2026-0414, with a patched
    minimum for future dependency resolution. NTNT did not enable the affected
    wikilink option, and Markdown rendering options are unchanged.

Upgrade checklist

  1. Run lint and application tests against the candidate. Audit code that deliberately
    relied on caller/CWD-relative ./ imports. Bare imports remain CWD-relative,
    absolute imports remain valid, and ordinary template/filesystem resource paths
    are unchanged.
  2. Check imported names that collide with route functions. For example, use
    get as kv_get when importing the KV getter rather than hiding route get.
    The import_shadows_route diagnostic points at these cases.
  3. Keep deployment policy explicit. The new database workflow does not run
    automatically at startup or replace an existing application-owned runner.
    Adopt only a reviewed matching baseline, retire conflicting old migration
    writers, and keep backup/restore and application cutover plans separate.
  4. Configure trusted proxies before enabling automatic rate limiting. For realtime
    apps, size startup caps for the worker setup as well as connected clients, handle
    publish errors, and implement reconnect reconciliation.
  5. Rust embedders constructing Statement::Import must supply source_file;
    None retains interpreter-base/CWD behavior. Parser::new remains source-less;
    use Parser::with_source_file when an origin is known. Serialized AST JSON is
    unchanged.

The database foundation is complete within its published transactional and
inspection coverage. Concurrent/nontransactional DDL, schema-aware query analysis,
general legacy-history import and automatic schema repair remain outside this
release. WebSockets remain separate follow-up work.

Release archives retain Linux x64, Linux ARMv7 hard-float, macOS Apple Silicon and
Windows x64 support, with checksum verification. See
release builds and installation
for platform requirements and pinned installation. When upgrading from versions
older than 0.5.6, also read their intervening upgrade notes.

Rate-limit measurement notes and known performance limits

These are the implementation measurements recorded for the limiter, not fresh
0.6.0 release benchmarks or promises for every deployment. The unmet targets are
retained here so the favorable measurements do not hide contention/flood limits.

Release builds, same 16-core host, wrk 2 threads / 64 connections, 5 × 6 s runs, median; baseline is main before #294 (faca493).

Scenario Result Budget
Limiter off vs baseline, hello route 134.3k vs 135.2k rps (−0.7%); p99 0.96–1.5 ms both ≤1% — met
Limiter on, under limit, hello 134.4k rps (−0.6%) ≤2% — met
Limiter on, distinct client IPs 117.1k vs 120.1k off (−2.5%) ≤3% — met
Key spray, MAX_KEYS=100000 +10 MB RSS, legitimate client served ≈+16 MB — met
Key spray, MAX_KEYS=1000000 (~3.5M requests, distinct IPs) +44 MB RSS, 140k rps, legitimate client served bounded, no collapse — met
One check, uncontended 39 ns <100 ns — met
16 threads, distinct keys 280–360 ns per-thread <500 ns — met
16 threads, one key 3.2 µs per-thread (200 ns aggregate) <500 ns — not met
Flood: rejected vs served cost 24.8 µs vs 53.8 µs CPU per request (2.2× cheaper) ≥3× — not met
Legitimate client p99 during a flood 266 → 483 µs (6-conn flood); 292 µs → 1.0 ms (64-conn) within 5% — not met

Notes on the unmet rows:

  • One key, 16 threads: all checks for one key share one shard lock, so they serialize. One key is one visitor; in practice this is a single client flooding, which is the case the limiter rejects.
  • Rejection cost: a 429 costs the same as an unmatched-route 404 (24.6 µs), the HTTP stack's floor on this host. The limiter adds no measurable cost on top; a 3× ratio would require a cheaper connection/response path, not a cheaper limiter.
  • Legitimate p99 under flood: load generator and server share one host, so the flood competes for CPU. Volumetric floods remain the edge's job (see the deployment guide).

Reproduce the micro numbers with cargo test --release --lib rate_limit::tests::check_timing -- --ignored --nocapture.

Full changelog: v0.5.6...v0.6.0

What's Changed

Full Changelog: v0.5.6...v0.6.0

Contributors

larimonious
Assets 12
Loading