clink v0.3.0

Ninety-nine commits of engine, benchmark and correctness work since v0.2.0.
No public header was removed or renamed, and snapshot and savepoint
encodings are unchanged - the GROUP BY accumulator's byte layout was audited
for this release. Operators that previously lost state at a restore now
persist it, so a v0.2.0 snapshot still restores and simply carries none of
that newly persisted state. One behavioural note: the keyed-shuffle routing
fix below means a savepoint restored across the upgrade can move keys between
subtasks, the same class of movement as a rescale.

Full notes: CHANGELOG.md.

The benchmark suite is published, and every query is gated

The full 17-query nexmark suite now runs on a five-node cluster against
canonical data, each query correctness-gated against an independent oracle at
parallelism 4 and every window kind gated cross-engine. Headline, measured:
clink processes an event for 1.9x to 5.3x less CPU (median 2.45x) than a
JVM stream processor producing identical output - method, caveats and raw
per-run data at Benchmarks,
priced out at Cost and footprint.
Two earlier q18/q19 figures were found unsound, withdrawn and re-measured.

Making that gate honest found real defects, now fixed

The row and columnar carriers of a keyed shuffle disagreed about which
subtask a key belongs to (the row side read the key through a double; 74.5%
of FNV-folded keys misrouted at parallelism 4, silently splitting group
state). A data batch larger than the send-credit window was silently
dropped; a failed send vanished instead of failing the task; an
unpartitioned stateful operator was fanned out to produce N answers. All
fixed, each with a test that failed against the original code.

A restore now preserves what was open

An open window survives a restore (tumbling, hopping, session and cumulate -
previously every open window was silently lost), and so do top-N retained
rows, LAST_N state, the OVER aggregate's sync path, and the upsert and
netting sinks' compaction view. Window arithmetic is floored rather than
truncated, event-time reads are exact, and window arguments fail at bind
time.

Also in this release

• Performance: projection pushdown reaches the columnar JSON bridge
  (45% off the shuffle split, 27% off decode); per-group aggregate state
  fell by a third (AggState 264 -> 104 bytes); the windowed fire stopped
  scanning every group per watermark (4.4x at 200k groups); the Kafka
  source fetches in batches; parallel pipeline instances co-locate, taking
  forward edges off the network.
• SQL: WHERE accepts expression operands; a MATCH_RECOGNIZE DEFINE
  predicate with an expression operand (price < PREV(price) * 0.997) now
  matches instead of silently never firing; declared FLOAT and DECIMAL are
  honoured at columnar JSON decode; doubles are no longer truncated to six
  significant digits.
• Embedded: the engine configures logging on first open and honours
  CLINK_LOG_LEVEL, so clink run and pyclink stop printing registry
  chatter that could not be turned off.
• Packaging: installed binaries carry an rpath to the pinned toolchain -
  a host cmake --install previously produced a binary macOS refused to
  load; CLINK_ISA_BASELINE and opt-in CLINK_WITH_JEMALLOC build knobs.

The three consumer-facing fixes (rpath, MATCH_RECOGNIZE expression
operands, embedded logging) were found by
market-pulse, a downstream
consumer showcase built against this release, on its first day running
against an installed clink.

Artifacts

pyclink wheel (macOS 14+, Apple silicon)

pip install pyclink-0.3.0-py3-none-macosx_14_0_arm64.whl

Self-contained: it bundles libclink with a static Arrow and vendors no
dylibs at all. The v0.2.0 caveats carry over unchanged: no object-store
filesystems (s3://, gs://, abfs://) and no HTTPS in the HTTP
subsystem; a source build keeps both.

sha256: 3743f309fc34940e5f311a7258447499d6b9c6889173d86ec5a344e393ebfcd8

Runtime image (linux/amd64)

ghcr.io/orhaugh/clink-runtime:0.3.0 (also :latest, :main,
:sha-<short>), used by docker-compose, the Helm chart and the Kubernetes
operator.