Skip to content

ReplicaDB 1.0.0

Choose a tag to compare

@github-actions github-actions released this 06 Sep 19:35
· 103 commits to master since this release

ReplicaDB 1.0.0

The biggest release in ReplicaDB's history

v1.0.0 is ReplicaDB's first stable release and the largest functional expansion in the project's history. ReplicaDB keeps its existing standalone CLI, while adding a complete managed replication platform with a server, REST API, web frontend, scheduler, distributed workers, durable PostgreSQL state, and operational observability.

This release supports two complementary workflows:

  • Standalone CLI for direct transfers, automation, and scripts.
  • ReplicaDB Server for persistent jobs, users, permissions, scheduling, distributed execution, auditing, and diagnostics.

ReplicaDB Server: API and control plane

This release introduces the managed replicadb-server product:

  • Authenticated REST API for users, permissions, datasources, jobs, runs, and schedules.
  • Web control plane for dashboards, datasource management, job creation and editing, schedules, runs, users, permissions, profiles, and audit activity.
  • OpenAPI generated from the server's actual API contract.
  • HTTP sessions, CSRF protection, controlled administrator bootstrap, and login throttling.
  • Administrator permissions plus resource-level job and datasource permissions.
  • Redacted responses: credentials and sensitive values are never returned to the frontend or exposed in diagnostics.
  • Per-run diagnostics with bounded multiline logs and exception details for troubleshooting without leaking secrets.

API, workers, and distributed execution

The server introduces a PostgreSQL-backed distributed topology:

  • One or more API processes expose the control plane and coordinate local execution when configured.
  • One or more worker processes execute runs without exposing the product API; their management endpoint remains separate and private.
  • Configurable worker identity, concurrency limits, per-worker capacity, and connection pools.
  • Dispatch through PostgreSQL notifications, mandatory polling fallback, and lease heartbeats.
  • Hybrid admission control with jitter, cooldowns, adaptive backoff, and capacity-aware approximate distribution.
  • Normalized utilization metrics for evaluating fairness across workers with different capacities.
  • Lease fencing, abandoned-run recovery, and protection against stale workers.

Effective execution capacity can be estimated as:

workers * concurrent runs per worker * internal jobs per run

PostgreSQL, table creation, and scheduling

PostgreSQL becomes the server's durable source of truth. Flyway applies forward-only migrations and automatically creates the metadata schema required by the platform.

The release adds and evolves tables for:

  • job_definition: persistent job definitions.
  • job_run: durable execution state and run history.
  • job_schedule: product-level scheduling intent.
  • app_user, job_permission, and datasource_permission: users and ACLs.
  • SPRING_SESSION and SPRING_SESSION_ATTRIBUTES: API sessions.
  • managed_datasource: encrypted reusable datasource profiles.
  • audit_event: administrative and operational activity.
  • login_attempt: authentication-attempt reservations and cleanup.
  • run_trigger_idempotency: duplicate-trigger protection.
  • run_log: persisted per-run logs.
  • Constraints and indexes for active runs, retries, leases, availability, datasource resolution, and dependent-state cleanup.

The schema also includes Quartz JDBC tables (QRTZ_JOB_DETAILS, QRTZ_TRIGGERS, QRTZ_CRON_TRIGGERS, QRTZ_FIRED_TRIGGERS, QRTZ_SCHEDULER_STATE, QRTZ_LOCKS, and supporting tables). This provides:

  • Persistent scheduling in PostgreSQL.
  • Clustered Quartz across multiple API processes.
  • Shared locks and stable job keys.
  • Startup schedule reconciliation.
  • Duplicate-fire protection and explicit misfire handling.

State, session, audit, retry, lease, dispatch, and Quartz tables are created through Flyway. Quartz's automatic schema initializer is not required.

Lifecycle, retries, and cancellation

Managed execution adds guarantees that are not part of the standalone CLI workflow:

  • Durable job and run state.
  • From-the-beginning retries with maxAttempts and backoff policies.
  • Configurable retry after lease expiration.
  • Incremental watermarks committed only after a successful merge.
  • Persistent cancellation and mode-specific cancellation warnings.
  • Idempotency keys to prevent duplicate triggers.
  • Stable datasource resolution when a run is claimed.
  • PostgreSQL locks released before replication work begins.
  • Recovery as a new attempt, never as false resume semantics.
  • Constraints preventing duplicate active executions.

Local server without Docker

local mode provides a durable single-node server without Docker or a system PostgreSQL installation:

  • PostgreSQL managed and embedded by ReplicaDB.
  • First download from Maven Central with mandatory SHA-256 verification.
  • Reusable cache for network-free restarts.
  • Native bundles for macOS ARM64/x64, Linux x64, and Windows x64.
  • Flyway, Quartz, sessions, encrypted datasource catalog, and local execution in one installation.
  • Durable state under REPLICADB_SERVER_HOME.
  • Separate keyring for encrypting and decrypting datasource credentials.

The CLI keeps using REPLICADB_HOME; the server uses REPLICADB_SERVER_HOME. The installations are not automatically migrated or mixed.

CLI and replication

The CLI remains Spring-free and preserves its existing options-file contract and exit-code behavior. This release maintains and strengthens:

  • Complete, incremental, and parallel replication.
  • Incremental watermark injection.
  • Multi-table replication.
  • Destination-table creation and use according to each manager and execution mode.
  • SQL, MongoDB, CSV, Amazon S3, Kafka, and generic JDBC connectors within their documented capabilities.
  • Cross-platform Java 17 support on Linux, macOS, and Windows.
  • Non-intrusive batch transfer without agents or source-database triggers.

Security and operations

  • Datasource credentials encrypted before PostgreSQL persistence.
  • Versioned AES keyring for controlled key rotation.
  • ${env:VARIABLE} references for environment-managed secrets.
  • Passwords, tokens, DSNs, and security material redacted from logs, responses, and metrics.
  • API designed for TLS or an authenticated reverse proxy.
  • Worker management ports private by design.
  • Health probes, Prometheus metrics, Actuator endpoints, and control-plane health signals.
  • Coordinated backups of data/postgresql and master-key.json.

Distribution and artifacts

The release publishes the complete, validated, reproducible set:

  • ReplicaDB-1.0.0.tar.gz
  • ReplicaDB-1.0.0.zip
  • ReplicaDB-server-1.0.0.tar.gz
  • ReplicaDB-server-1.0.0.zip
  • replicadb-server-1.0.0.jar
  • SHA256SUMS

Versioned and latest Docker images are also published for the CLI and server, including the UBI variant. The direct JAR matches the JAR inside the server archive and contains no native PostgreSQL bundles.

Quality and validation

The release was validated with:

  • The CLI suite and the complete server suite.
  • Embedded PostgreSQL profiles on Linux, macOS, and Windows.
  • Integration tests for DB2, MariaDB, MySQL, Oracle, PostgreSQL, SQLite, SQL Server, and MongoDB.
  • Frontend E2E tests and OpenAPI contract-drift checks.
  • Fairness, multinode, load, resilience, and worker-loss tests.
  • Windows launcher contract tests and packaged-server smoke tests.
  • Docker image smoke tests and Compose validation.
  • Reproducible archives, JAR identity checks, and checksum verification.
  • CT, CodeQL, and GitHub Pages for the exact preparation commit.

Compatibility and operational notes

  • Requires Java 17 or newer.
  • local mode needs initial administrator credentials and network access on first start when the native bundle is not cached.
  • api and worker modes require external PostgreSQL, a shared keyring, and environment-managed secrets.
  • Do not expose the worker management port to the public network.
  • Keep the keyring with the PostgreSQL backup; without it, encrypted datasource credentials cannot be recovered.

Documentation: https://osalvador.github.io/ReplicaDB/

Comparison with the previous release:
v0.18.4...v1.0.0