clink v0.2.0
Forty-four commits of engine, build and CI work since v0.1.0. No API or format
breaks: state written by 0.1.0 restores unchanged, and every new behaviour is
either on by default with a documented opt-out or off by default.
Full notes: CHANGELOG.md.
Distribution is the headline
clink now ships artifacts rather than only source. The runtime image is published,
and the Helm chart, the Kubernetes operator and its samples default to it - before
this release they referenced an image tag that existed only on the author's
machine, so the Kubernetes path was undeployable by anyone else. Prebuilt pyclink
wheels build on a release tag again. Documentation publishes to
orhaugh.github.io/clink.
Columnar execution now covers both ends of a pipeline
Columnar JSON decode is the default for Kafka tables (columnar_decode='false'
opts out), with an adaptive damper so systematically unfaithful data pays ~1.6%
instead of 2x, and the keyed shuffle splits columnar batches on the cluster path
with zero row decode. Operators can now also emit born-columnar output: an
append-only INNER join and the windowed fire build typed Arrow columns directly
instead of a name-keyed row per emission, enabled by the planner only where a
consumer can ingest columnar.
Gated nexmark q12: sustained slope 1.08M -> 1.83M rec/s (+69%), CPU 113s ->
54s. Born-columnar emission adds -11% CPU on a join under a filter.
Also in this release
- Performance: scratch keys and transparent state probes removed the
per-record key-string build from the window, session, aggregate (-31% on the
q12 row path) and equi-join (-30%) operators; a pre-sorted bulk join-output
build took the join to -54% cumulative. - State: opt-in ForSt backend (
CLINK_WITH_FORST=ON) withforst://,
changelog+forst://,s3+forst://ands3sst+forst://for live remote data
files, plus a deferred-read mode that puts hot-path state in the engine. - Correctness: an input is closed only when closed AND drained, fixing an
end-of-stream data-loss race; a worker registration is installed before it is
acked.
Artifacts
pyclink wheel (macOS 14+, Apple silicon)
pip install pyclink-0.2.0-py3-none-macosx_14_0_arm64.whlSelf-contained: it bundles libclink with a static Arrow and vendors no
dylibs at all, so it depends on nothing but macOS system libraries. Verified by
installing into a clean venv and running a windowed GROUP BY through the bundled
library.
sha256: 6a94099b06cb0a2e0e9db745138fb5f2c97f58a6c26ba3598219e0d6c5b05b8c
Two deliberate reductions, both of which are what let the wheel keep a macOS
14 floor rather than requiring the build host's much newer version:
- No object-store filesystems.
s3://,gs://andabfs://paths do not
work from the wheel. - No HTTPS in the HTTP subsystem. It serves and fetches plain HTTP only.
Local files, SQL, the embedded engine and everything else are unaffected. A build
from source keeps both capabilities - they are on by default there.
Linux wheels are not published: the pinned toolchain image is Debian trixie, whose
glibc floor would make them installable only on very new distributions.
Runtime image (linux/amd64)
docker pull ghcr.io/orhaugh/clink-runtime:0.2.0
helm install my-clink deploy/helm/clink # defaults to this imageAlso tagged :latest and :sha-9929808d6655. Anonymous pull verified.