Skip to content

Release v0.1.8

Choose a tag to compare

View all tags
@thanhtham010891 thanhtham010891 released this 28 May 08:01
· 10 commits to main since this release
v0.1.8
fe93dbc

Documentation

New: Runtime Guarantees

Added guides/runtime-guarantees.md — the
single source of truth for what the runtime promises. Covers:

  • source-order delivery in linear and buffered modes
  • sink fail-closed semantics and when each policy applies
  • the full checkpoint advancement table (success, drop, DLQ, fail-closed,
    log-and-continue)
  • DLQ routing and DLQ failure policy
  • DLQ replay acknowledgement
  • cancellation and lifecycle ordering
  • intentional non-guarantees: at-least-once delivery, no two-phase commit,
    untrusted-config caveat, public-edge hardening, cross-source ordering

The page is now the authoritative contract. architecture.md and
failure-handling.md cross-link into it instead of restating guarantees in
prose.

New: Recovery Support Matrix

Added guides/recovery-matrix.md — the
per-source resume contract. Covers:

  • which built-in sources are checkpointable and what their resume key looks
    like
  • the three conditions a source must satisfy to be treated as checkpointable
  • per-source notes for JsonLinesSource, CsvSource, ParquetSource,
    HTTPSource, and IterableSource
  • what resume does and does not promise
  • checkpoint failure handling for both the load and save paths
  • a minimal recipe for adding resume support to a custom source

sources.md and guides/checkpointing.md now cross-link into the matrix
instead of carrying their own short tables.

Tests

Preservation: runtime guarantees

Added tests/preservation/test_runtime_guarantees.py (12 tests). Each test
maps to one declared guarantee in runtime-guarantees.md:

  • linear-mode source-order commit
  • buffered-mode source-order commit when the buffered stage resolves futures
    in reverse order
  • sink fail-closed propagates without DLQ
  • checkpoint advances on successful write
  • checkpoint advances when middleware DLQs the record
  • DLQ routing carries stage="middleware" and middleware name
  • middleware failure without DLQ continues the pipeline and counts the error
  • sink failure with DLQ advances the checkpoint
  • is_checkpoint_capable requires the explicit supports_checkpoint flag
  • DLQFailurePolicy.RAISE propagates DLQ write errors
  • DLQFailurePolicy.LOG_ONLY (default) swallows DLQ write errors
  • SinkFailurePolicy.LOG_AND_CONTINUE advances the checkpoint over a failed
    write

Preservation: recovery edge cases

Added tests/preservation/test_recovery_edge_cases.py (10 tests):

  • corrupted checkpoint payload (missing fields) raises a typed TypeError —
    protects the 0.1.7 fix
  • non-dict checkpoint payload raises a typed TypeError
  • prepare_resume failure aborts the run under FAIL_CLOSED
  • prepare_resume failure under LOG_AND_CONTINUE starts from scratch
  • checkpoint load() failure aborts the run under FAIL_CLOSED
  • checkpoint load() failure under LOG_AND_CONTINUE starts from scratch
  • checkpoint save() failure under LOG_AND_CONTINUE does not retry-storm —
    protects the 0.1.7 mark_saved fix
  • non-checkpointable source paired with a checkpoint store runs without
    checkpointing instead of raising
  • two-run resume restores progress from the saved position
  • SQLiteCheckpointStore survives close/reopen — basis for cross-process
    resume
Assets 2
Loading