clink v0.10.0

clink v0.10.0 makes an embedded run survive a crash. Run the same script
again after clink run or a pyclink Engine is killed, and it continues
from its last checkpoint instead of starting over.

pip install pyclink

An embedded run resumes after a crash. With --checkpoint-dir set, a
rerun after a kill, a crash, a failure or Ctrl-C restores the job from its
own checkpoints. Before, the rerun started from nothing. It re-read its
sources from the beginning and re-published everything its exactly-once
sinks had already committed. In the integration test, clink run reads
Kafka and writes windows to a transactional Kafka sink. It is killed with
SIGKILL mid-stream and started again. Every window is committed once. With
resume switched off, the same test commits 16 of the 40 windows twice.

The restore point is chosen as a cluster's HA recovery chooses it,
including in-doubt resolution for sinks whose commits need confirmation.
New checkpoints number above every id already on disk.

• A job that reached the end of its input cleanly starts over on a rerun.
  A bounded load or a full-refresh materialized view runs again, rather
  than resuming at its end and publishing nothing.
• A run that died before completing a checkpoint of its own also starts
  over. It is never handed the state of the run before it.
• A checkpoint directory written by a different job graph is refused, with
  both fingerprints named. Job ids restart at 1 in every process, so the
  directory records which graph wrote it.
• --fresh starts from empty state. pyclink has Engine(fresh=True) and
  the C ABI has clink_engine_options.fresh.

File sinks keep their earlier output across a restore. The plain file
sink and the partitioned file sinks truncated their files whenever they
opened, including on a restore. After a cluster failover, the file held
only the rows written since the restore point. They now append on a
restore. Rows after the restore point are written again as the sources
replay them, which is these sinks' at-least-once contract. An overwrite
sink keeps its staging file. delivery_guarantee='exactly_once' remains
the exactly-once file path.

Known limitation. The plain Parquet sink cannot append, and a killed run
leaves its file without a footer. After any restore, cluster failover
included, it holds only the rows written since. Where a restore must keep
earlier Parquet output, use the exactly-once Parquet sink.

Compatibility.

• CheckpointConfig and the coordinator's deploy path are on the plugin
  ABI surface, so the plugin ABI fingerprint changes. Rebuild job modules
  against this release.
• clink_engine_options gains fresh at its end. The change is
  append-only, and struct_size covers callers built against v0.9.x.
• A checkpoint directory written by an earlier release records no graph
  fingerprint, so resuming from it is refused. Start such a job with
  --fresh, or use a new directory.
• Cluster submits neither resume nor write the new records. On a restore,
  a cluster's file sinks now append to their output, as described above.

The exactly-once specification's trace validation now covers an embedded
kill and resume. The recorded run is under formal/traces, and CI checks it
against the model on every push.

Documentation: https://orhaugh.github.io/clink/

Artefacts

• pip install pyclink: manylinux_2_28_x86_64, manylinux_2_28_aarch64 and
  macosx_14_0_arm64 wheels on https://pypi.org/p/pyclink.
• ghcr.io/orhaugh/clink-runtime:0.10.0: runtime image for linux/amd64 and
  linux/arm64.
• clink-v0.10.0-linux-x86_64-ubuntu24.04.tar.gz: relocatable SDK (CLI,
  headers, libraries), with its checksum published alongside.