0.1.0

FoundationDB CDC to Kafka Bridge (v0.1.0 Foundational)

Foundational release of Kafka-FDB-Streamline, an end-to-end Change Data Capture (CDC) bridge linking FoundationDB 8.0+ native mutation streams to Apache Kafka topics with strict ordering and durability guarantees.

What's Included

1. Native Protobuf Schema Pipeline (fdbkafka.cdc.v1)

• Standardized Protobuf schema (mutations.proto) supporting single key mutations, clear ranges, high-throughput batches (FDBMutationBatch), and commit boundary audit records (VersionEnd).
• Automated schema compilation (buf generate), linting, and drift verification in CI.

2. Binary Serialization & Slicing (src.serialization)

• Pure, stateless serializer/deserializer handling all 16 FoundationDB native mutation operation codes.
• Deterministic batch packing bounded by max_record_bytes without splitting atomic mutations.
• Unpacks wire records into frozen, typed dataclasses.

3. Native CDC Consumer Engine (src.cdc)

• FoundationDB API 800 client wrapper with directory subspace prefix range alignment ([prefix, strinc(prefix))).
• Bounded polling with strict single-unacknowledged-reply lifecycle gating (awaiting_ack).
• Cluster read-version synchronization barrier for safe resume and cumulative retention watermark advancement.

4. High-Durability Kafka Publisher (src.kafka)

• confluent-kafka producer wrapper enforcing strict delivery invariants: acks=all, enable.idempotence=true, max.in.flight.requests.per.connection=5, enable.gapless.guarantee=true, and message.timeout.ms=0.
• Dual modes: Idempotent (with sticky callback error tracking) and Transactional (atomic multi-record commits for read_committed consumers).
• Queue-purging graceful close behavior to prevent thread blocking during shutdowns.

5. Bridge Coordinator & Self-Healing Daemon (src.bridge)

• BridgePipeline: Single-step coordinator enforcing stable single-partition key routing per CDC reply.
• Atomic, crash-durable cursor persistence with parent directory fsync barriers.
• Autonomous daemon loop with exponential backoff and automatic pipeline rebuild from durable checkpoints.

---

Pre-Release Verification & Stress Testing

• Test Suite: 639 passing tests (unit, property-based Hypothesis, golden bytes, and live 10,000-mutation integration verification).

---

Installation

pip install git+https://github.com/Ronitsabhaya75/Kafka-FDB-Streamline.git@v0.1.0

Full Changelog: v0.1.0...v0.1.0