Skip to content

v2.0.2 Beta

Choose a tag to compare

View all tags
@maxpert maxpert released this 25 Nov 16:45
· 748 commits to master since this release
v2.0.2
f744d1a

Changelog

Marmot v2 is a ground-up rewrite replacing the NATS/Raft-based architecture with a simpler, self-contained gossip-based system.

Breaking Changes

  • MySQL Protocol Required: Applications must connect via MySQL wire protocol (default port 3307) instead of direct SQLite file access
  • Configuration Overhaul: NATS settings removed, new gRPC/gossip/MVCC settings added
  • Data Directory: Place SQLite .db files in data_dir - they are imported on first startup

Removed

  • NATS JetStream dependency
  • Trigger-based CDC (global_change_log_script.tmpl, table_change_log_script.tmpl)
  • stream/ and logstream/ packages
  • Consistent hashing and sharding (replication_factor, virtual_nodes)
  • db_path config - replaced by data_dir with auto-import

New Features

MySQL Protocol Server

  • Connect with any MySQL client (mysql CLI, DBeaver, MySQL Workbench, JDBC)
  • USE database; support
  • SHOW DATABASES, SHOW TABLES, DESCRIBE table
  • INFORMATION_SCHEMA queries for GUI compatibility

DDL Replication

  • CREATE TABLE, ALTER TABLE, DROP TABLE now replicate across cluster
  • CREATE DATABASE, DROP DATABASE support
  • Cluster-wide DDL locking (30s lease) prevents conflicts
  • Automatic idempotency rewriting (CREATE TABLE IF NOT EXISTS)

CDC-Based Replication

  • Binary row-level changes instead of SQL statement replay
  • Deterministic application regardless of SQL dialect
  • Row key extraction for conflict detection

MVCC Transactions

  • Snapshot isolation per transaction
  • Write intent tracking for conflict detection
  • Hybrid Logical Clock (HLC) for distributed ordering
  • Last-Write-Wins conflict resolution with HLC + Node ID

Anti-Entropy System

  • Automatic background healing for lagging nodes
  • Delta sync for recent lag (< 10K transactions or < 1 hour)
  • Snapshot transfer for larger gaps
  • Gap detection prevents data loss during garbage collection
  • Per-database replication state tracking

Gossip Protocol (SWIM-style)

  • No external message broker required
  • Failure detection with SUSPECT → DEAD transitions
  • Configurable timeouts (suspect_timeout_ms, dead_timeout_ms)

Consistency Levels

  • LOCAL_ONE: Local node only (fastest)
  • ONE: Any single replica
  • QUORUM: Majority of nodes (default)
  • ALL: All alive nodes (strongest)

Prometheus Metrics

  • Served on gRPC port at /metrics endpoint
  • No separate metrics port needed

Configuration Changes

Removed settings:

  • nats_urls, stream_prefix, subject_prefix
  • db_path (replaced by data_dir)
  • replication_factor, virtual_nodes

New settings:

[cluster]
grpc_bind_address = "0.0.0.0"
grpc_advertise_address = ""
grpc_port = 8080
seed_nodes = []
gossip_interval_ms = 1000
gossip_fanout = 3
suspect_timeout_ms = 5000
dead_timeout_ms = 10000

[mysql]
enabled = true
bind_address = "127.0.0.1"
port = 3307
max_connections = 1000

[mvcc]
gc_interval_seconds = 30
gc_retention_hours = 2
heartbeat_timeout_seconds = 10
version_retention_count = 10
conflict_window_seconds = 10

[replication]
default_write_consistency = "QUORUM"
default_read_consistency = "LOCAL_ONE"
enable_anti_entropy = true
anti_entropy_interval_seconds = 60
delta_sync_threshold_transactions = 10000
delta_sync_threshold_seconds = 3600
gc_min_retention_hours = 2
gc_max_retention_hours = 24

[coordinator]
prepare_timeout_ms = 2000
commit_timeout_ms = 2000
abort_timeout_ms = 2000

[ddl]
lock_lease_seconds = 30
enable_idempotent = true

Migration from v1

  1. Update connection strings from direct SQLite to MySQL protocol:

    # v1: Direct SQLite
sqlite3 /path/to/database.db

# v2: MySQL protocol
mysql -h 127.0.0.1 -P 3307 -u root database

  2. Move existing .db files to data_dir - they are auto-imported on startup

  3. Update configuration file (see new settings above)

  4. Ensure schemas are synchronized across nodes before first cluster start

v1.x (Legacy - NATS/Raft Based)

The v1.x series used NATS JetStream for message transport and trigger-based CDC for change capture.

Features

  • Leaderless replication via NATS JetStream
  • Trigger-based change capture
  • Eventual consistency with LWW conflict resolution
  • Multiple snapshot backends (NATS, S3, WebDAV, SFTP)
  • Direct SQLite file access (sidecar pattern)

Limitations

  • No DDL replication (manual schema sync required)
  • No multi-database support
  • Required external NATS cluster or embedded NATS
  • No explicit consistency level control
Assets 12
Loading