Skip to content

v2.3.0-beta

Choose a tag to compare

View all tags
@maxpert maxpert released this 13 Dec 04:17
· 748 commits to master since this release
v2.3.0
8255c4e

Highlights

This release introduces the CDC Publisher system for streaming database changes to external systems, along with significant MySQL compatibility improvements and bug fixes.

New Features

CDC Publisher System

Stream database changes to Kafka and NATS in real-time using the Debezium event format.

  • Kafka sink - Publish CDC events to Apache Kafka topics
  • NATS sink - Publish CDC events to NATS JetStream subjects
  • Debezium-compatible format - Works with Kafka Connect, Apache Flink, Spark Streaming, and other stream processors
  • Multi-sink support - Configure multiple sinks with independent cursors
  • Glob filtering - Filter by table/database patterns (e.g., order_*, *_log)
  • At-least-once delivery - Automatic retry with exponential backoff
[publisher]
enabled = true

[[publisher.sinks]]
name = "kafka-main"
type = "kafka"
format = "debezium"
brokers = ["localhost:9092"]
topic_prefix = "marmot.cdc"
filter_tables = ["*"]

Compact 53-bit ID Generation

New auto_id_mode = "compact" option generates JavaScript-safe 53-bit IDs instead of 64-bit snowflake IDs.

  [mysql]
  auto_id_mode = "compact"  # or "snowflake" (default)

MySQL Compatibility Improvements

  • LAST_INSERT_ID() function support
  • FOUND_ROWS() function support
  • INSERT ... ON DUPLICATE KEY UPDATE transpilation
  • UPDATE/DELETE ... JOIN transpilation
  • MySQL handshake response parser
  • WordPress compatibility functions

Bug Fixes

  • CDC entry loss fix - Fixed issue where CDC entries were lost during EphemeralHookSession Commit/Rollback
  • CDC TableName extraction - Fixed DRY violation and hierarchical QueryContext refactor
  • UNIQUE KEY handling - Keep inline instead of incorrectly extracting to CREATE INDEX
  • StatementTypeToOpType mapping - Fixed enum mapping bug with added type safety
  • Build fix - Fixed compilation without sqlite_preupdate_hook tag
  • CDC lock cleanup - Added reverse index optimization (O(n) → O(m))
  • Multi-row replication - Fixed CDC pipeline for multi-row operations

Performance Improvements

  • Reverse index for CDC locks reduces cleanup from O(n) to O(m)
  • Optimized UpdateWithJoinRule in transpilation

Documentation

Examples

  • CDC Demo - Real-time TODO app demonstrating CDC with NATS JetStream
  • WordPress Demo - WordPress running on Marmot with non-standard ports

Breaking Changes

None

Upgrade Notes

No migration required. CDC Publisher is disabled by default.

Full Changelog

v2.1.6...v2.2.0

Assets 12
Loading