Skip to content

clink v0.1.0

Choose a tag to compare

@orhaugh orhaugh released this 21 Jul 22:24
· 635 commits to main since this release

Initial public release: the engine as described in the README.

Naming, settled for 1.0: the cluster roles are the coordinator (control
plane) and workers (subtask hosts), clink_node --role=coordinator|worker;
the fluent API entry point is clink::api::Pipeline; the in-process test
cluster is clink::test::TestCluster. Domain vocabulary (watermarks, windows,
checkpoints, savepoints, keyed state, key groups, slots) is unchanged.

In brief:

  • Typed operator DAG and fluent API on a local runtime and a distributed
    Coordinator/Worker runtime (TLS/mTLS, HA, HTTP API + dashboard).
  • Event time end to end: watermarks, tumbling/sliding/session windows,
    interval joins, CEP.
  • Keyed and broadcast state over in-memory, file-backed, RocksDB, and
    changelog backends; rescale, schema evolution, savepoints.
  • Exactly-once checkpointing with true 2PC sinks (file, Kafka, Parquet,
    S3, Postgres) and an effectively-once upsert family.
  • Arrow-native columnar wire format and columnar operator fast paths.
  • SQL frontend: embedded (clink run, libclink C ABI, pyclink, Flight
    SQL) or submitted to a cluster.
  • Deterministic incident replay: flight recorder, state-diff,
    replay --verify, frozen regression bundles.
  • State as data: snapshots are Arrow IPC; export to Parquet/Iceberg;
    queryable live state.
  • Connector suite across messaging, object storage, table formats,
    databases, and HTTP endpoints (see docs/connectors/).
  • Public testing framework (clink::test), Kubernetes Helm chart and
    operator, reproducible benchmark harnesses.