Skip to content

NTNT v0.6.1 — Native WebSockets and runtime hardening

Choose a tag to compare

@github-actions github-actions released this 15 Nov 20:11
· 102 commits to main since this release
e2f7747

NTNT v0.6.1 — Native WebSockets and runtime hardening

NTNT 0.6.1 adds native WebSocket endpoints to the async HTTP server, collects
unreachable closure/environment cycles, and expands the SQL-first database
migration workflow. It also fixes error-scope restoration, integer-overflow
panics, worker wakeups, static-file routing and WebSocket refusal framing.

This announcement describes the changes since the last published release,
v0.6.0. Native SSE, the initial database workflow and HTTP rate limiting were
already available there; they are not new features of 0.6.1.

WebSockets through ordinary routes

Register an endpoint during application setup, then return websocket() from a
GET handler. The existing middleware and application authorization run before
any upgrade. Callback functions stay in the existing HTTP workers; an idle
connection holds native async state, not a dedicated worker or OS thread.

A minimal public, local echo endpoint:

import { ws_endpoint, websocket, ws_send } from "std/http/server"

fn echo(conn, message, state) {
    ws_send(conn, message["data"])?
    return map { "messages": state["messages"] + 1 }
}

let echo_endpoint = ws_endpoint("echo", echo, map {
    "origins": ["http://localhost:8080"]
})

get("/ws", fn(req) {
    return websocket(echo_endpoint, map { "state": map { "messages": 0 } })
})

listen(8080)

Run with ntnt run, then connect to ws://localhost:8080/ws from that origin.
This echo is intentionally public. Production applications must authorize their
routes and configure their actual public origin, not copy the localhost policy.
The bundled WebSocket demo also shows a signed-in broadcast feed.

  • Explicit connection state: one callback runs at a time per connection.
    Callbacks receive and return JSON-compatible state, bounded to 16 KiB. Text and
    binary messages, on_open, and best-effort on_close are supported.
  • Shared broadcast source: websocket(endpoint, map { "source": bus })
    forwards later publications as JSON text, sharing encoded event data with SSE.
    ws_send success means queued, not delivered. A lagging socket closes with 1013;
    an oversized broadcast closes that socket with 1009 without failing publication
    to other eligible subscribers.
  • Bounded work: send queues, receive backlogs and callback admission are
    bounded. With several workers serving, callback admission normally leaves one
    available for HTTP and follows live worker membership. Already-running callbacks
    cannot be preempted: after a worker fails they may occupy every survivor until
    one finishes or the worker restarts. A single long callback can delay HTTP in
    the one-worker case.
  • Finite lifetime: sockets close with 1001 after max_lifetime_ms (default
    five minutes; configurable from one second to 24 hours). Reconnecting runs
    middleware and authentication again. Handshake authentication is not immediate
    revocation; sensitive commands must recheck current authorization.
  • Exact Origin policy: use endpoint origins, otherwise SITE_URL's origin,
    otherwise loopback origins in development. Production startup fails without a
    trusted origin configuration. CORS is not WebSocket authorization.

This is bounded, process-local, nondurable delivery. There is no replay,
cross-process broker, room protocol, automatic message retry or per-embedded-app
realtime isolation. Clients must reconnect and reconcile authoritative state.
WebSockets use HTTP/1.1 upgrade, not HTTP/2 extended CONNECT or HTTP/3 WebSockets.
Use the real async ntnt run server for protocol tests; the synchronous
ntnt test server returns 501 for streaming/upgrades.

Source identity and development reload

Realtime applications now compare the exact source bytes loaded by each HTTP
worker at the corresponding setup boundary where it first reaches listen or a
server block. A mismatch fails initial startup before bind/readiness. A restarted
worker that loads different source stays out of service until it matches the
committed startup source again. Endpoint applications use this realtime path too.

This is not complete program attestation. Endpoint registration checks native
configuration, but full callback/captured-value identity, imports first taken
after listen, and source provenance inherited by task interpreters are not all
covered. The runtime build is shared by workers in one process; the source
snapshot is not a release-binary fingerprint.

Development reload closes WebSockets with 1012 and rejects stale connection
state/callback work. It does not atomically reload every worker: only worker 0
hot-reloads. With NTNT_WORKERS>1, other workers can still serve old HTTP code and
refuse current WebSocket work until restart. Default single-worker development
and production immutable-code deployments avoid this mixed-worker reload path.
Coordinated development reload remains separate work (#361).

Runtime and HTTP fixes

  • Unreachable closure/environment cycles are collected, including imported helper
    functions retained by completed scheduled callbacks and local closures in
    persistent interpreters. Escaped callbacks retain their bindings while reachable.
    This is memory reclamation, not a new deterministic native-resource close API.
  • Match guards/bodies, struct-invariant errors and failing let ... otherwise
    blocks restore the caller's environment before propagating errors. Contract
    diagnostics still use the struct's field values.
  • Integer division and remainder report runtime errors for zero divisors or
    signed overflow instead of panicking the interpreter or terminating HTTP service.
  • Imports referenced only by otherwise recovery blocks count as used, including
    aliases, destructuring and nested recovery blocks.
  • HTTP workers no longer lose wakeups while switching between HTTP work,
    WebSocket callbacks and returned callback slots. The shared wait path uses fresh
    flume async receives, not flume::Selector.
  • Editing a route file in place retains the full reload prelude, including its
    imports and response helpers.
  • Static mounts respect URL path boundaries; /assets does not hide a sibling
    such as /assets-img. Async .mjs files use
    application/javascript; charset=utf-8, matching .js behavior.
  • Closing WebSockets release broadcast subscriptions before the close handshake
    and application hook, rather than retaining unread events during slow teardown.
  • Finite native WebSocket refusals set the UTF-8 byte Content-Length and stay
    uncompressed. This preserves complete refusal responses through the tested
    proxy path without changing Origin policy, successful upgrades, SSE or ordinary
    HTTP compression.

Database migrations and PostgreSQL auth startup

The ntnt db workflow introduced in 0.6.0 now handles more real PostgreSQL schemas
and reports narrower, observable lock behavior:

  • public can be the managed schema. Replay/execution accepts complete migration
    scripts including DO, functions, triggers, policies, grants, seed rows and
    deferred constraints. Unsafe transaction/session escapes, cross-database work
    and CASCADE drops remain rejected; unsupported analysis is reported as unknown,
    not silently treated as understood.
  • Catalog comparison covers views, sequences, SQL/PLpgSQL routines, triggers,
    RLS policies, declarative partitions, expression/partial indexes and managed
    extensions. Replay respects the target's row-security capability.
  • Migration ownership uses an advisory lock rather than blanket table locks.
    Statements take their required locks under a one-second lock timeout; named
    transition-check tables use SHARE. Lock timeout/deadlock retries are bounded,
    and scripts have a 90-second statement limit. Plans predict lock modes and flag
    potentially blocking DDL; execution reports held relation locks and retries.
  • A single named CREATE INDEX CONCURRENTLY can run outside a transaction. An
    interrupted attempt blocks later work with DB_ATTEMPT_UNRESOLVED until an
    operator reconciles the same plan as completed or rolled back. Cleanup does not
    authorize blind replay of uncertain execution.
  • The unnecessary final table-lock pass is removed, avoiding a reported failure
    after an otherwise committed migration.
  • Already-current PostgreSQL auth schemas can start under a runtime DML role
    without issuing redundant DDL. Startup validates required schema shapes and
    bounds lock waits. Missing/outdated schema still needs owner-authorized
    bootstrap/migration; the runtime does not grant itself privileges.

Retained implementation evidence, not a fresh release benchmark: the migration
work was exercised on Jarri staging with 33 migrations and about 11 GB of data.
In that recorded small-migration rehearsal, worst concurrent write stall fell
from 937 ms to 164 ms. Those observations are specific to that workload, not a
latency guarantee. PostgreSQL TLS transport remains separate work: use the
supported trusted local/secured transport described in the migration guide.

Compatibility and upgrade checklist

  1. Run application lint/tests with the candidate. Treat new runtime errors as
    recoverable errors, not process-panic hooks. Re-review/regenerate migration
    plans with the new binary and supply normal separate approval for the exact
    target-bound plan; do not change applied migration history.
  2. Deploy production source immutably and restart workers together. Do not edit
    files underneath a running realtime generation or rely on multi-worker hot
    reload as a coordinated deployment.
  3. Configure WebSocket origins and application authorization explicitly. Implement
    reconnect/refetch behavior and handle lifetime, reload and overload closes.
  4. Bootstrap/upgrade PostgreSQL auth schema with the owner before restarting with
    a restricted runtime role. Keep runtime and migration authority separate.
  5. Rust embedders: Value::Function::closure now uses the Rc-compatible
    Closure wrapper; convert an environment Rc with .into(). Capture/shared
    binding semantics remain. For file-backed source attestation use
    SourceFile::read(...).parse() and eval_source; source-less AST execution
    remains supported but cannot be attested by assigning a filename.

pulldown-cmark 0.13.6 fixes
RUSTSEC-2026-0433, a quadratic
nested-image-wikilink parser path. NTNT does not enable wikilinks, so its Markdown
configuration did not expose that path. Rebuild the binary to obtain updated
native dependencies. Contributors/builders use the pinned Rust 1.99.1 toolchain.

Linux x64, Linux ARMv7 hard-float, macOS Apple Silicon and Windows x64 archives
retain their normal checksum and packaging gates. This release does not add a
KV close/reconnect API, coordinated multi-worker development reload, stateful
Intent journeys or general migration repair.

The version-pinned references below become available with the v0.6.1 tag:

Full changelog: v0.6.0...v0.6.1

What's Changed

  • feat(db): run ntnt db on real PostgreSQL applications; narrower locks (0.6.1) by @larimonious in #321
  • feat(db): predict migration locks and warn about blocking changes by @larimonious in #326
  • feat(db): concurrent index builds outside the migration transaction by @larimonious in #327
  • fix(runtime): reclaim lexical environment cycles automatically by @larimonious in #323
  • feat(db): explicit recovery for interrupted concurrent index builds by @larimonious in #328
  • feat(runtime): track exact loaded-source provenance (DD-041) by @larimonious in #319
  • chore(lint): apply clippy's automatic fixes by @larimonious in #329
  • chore(lint): fix remaining clippy warnings and enforce zero in CI by @larimonious in #330
  • chore(rust): pin Rust 1.99.1 for every build by @larimonious in #331
  • chore(deps): cargo update within existing version ranges by @larimonious in #332
  • feat(runtime): realtime workers must load the same source (DD-041) by @larimonious in #333
  • DD-041 Slice B: WebSocket upgrade and per-connection owner by @larimonious in #334
  • ci: accept RUSTSEC-2026-0459 until the rusqlite 0.41 upgrade by @larimonious in #337
  • DD-041 Slice B: WebSocket endpoints with worker callbacks by @larimonious in #335
  • deps: rusqlite 0.41 (SQLite 3.54) for RUSTSEC-2026-0459 by @larimonious in #340
  • test(db): fix flaky lock-wait in PostgreSQL catalog tests by @larimonious in #342
  • fix(auth): allow DML-only PostgreSQL auth cold-start by @larimonious in #341
  • DD-041 Slice B: forward broadcasts to WebSockets by @larimonious in #339
  • test(db): fix flaky lost-commit reconcile check by @larimonious in #343
  • docs(db): quick start, current status and specific next actions (#245) by @larimonious in #352
  • fix(hot-reload): keep the prelude when a route file is reloaded (#357) by @larimonious in #358
  • DD-041 Slice B: WebSocket lifetime limit and close on development reload by @larimonious in #349
  • DD-041 Slice B: WebSocket callback slots follow live workers by @larimonious in #350
  • examples: runnable WebSocket demo (echo + authorized feed) by @larimonious in #351
  • fix(runtime): workers never miss a request after losing a wakeup race by @larimonious in #360
  • fix(http): isolate static mounts and bound WebSocket broadcasts by @larimonious in #362
  • fix: handle integer division errors and otherwise import usage by @larimonious in #364
  • fix(runtime): restore temporary scopes before error recovery by @larimonious in #365
  • fix: serve .mjs with JavaScript MIME in async server by @larimonious in #367
  • fix(http): explicitly frame finite WebSocket refusals by @larimonious in #369
  • docs: prepare 0.6.1 release and reconcile realtime status by @larimonious in #370

Full Changelog: v0.6.0...v0.6.1