v1.1.0

v1.1.0

This release brings M1/M2/M3 to a usable and demo-ready baseline, with online observability and release preflight quality gate integration.

Highlights

• M3 online observability is now available:
  • Added /metrics endpoint (Prometheus text format)
  • Added core metrics:
    • task_total
    • task_retry_total
    • worker_alive
    • stage_duration_seconds
• Added minimal Grafana dashboard:
  • dashboards/mrkit-observability-minimal.json
• Added observability docs and README chart references for throughput/worker-alive views.

Release Quality Gate

• M1 gate is now wired into release flow before Docker image publishing:
  • release-image workflow runs gate automatically for tagged releases
  • still suitable as non-required branch check policy while ensuring release-time visibility

Demo and Docs Improvements

• Multi-node (M2) troubleshooting expanded for common failure points:
  • port conflict
  • plugin ELF mismatch
  • worker advertised address misconfiguration
  • go-run cache permission issue and mitigation guidance
• Docker quickstart docs updated to use config-driven JSON patching flow for reproducible container runs.

Design Progress (M3.5)

• Added master failover design draft documentation:
  • state persistence
  • restart recovery
  • active/standby takeover boundaries

Included commits

• feat(observability): deliver M1/M2/M3 online observability and release preflight gate
• docs: fix Docker quickstart flow and M2 troubleshooting commands
• merge: dev into master for v1.1.0

---

Retargeted on 2026-02-26: tag v1.1.0 now points to commit c3b4361 (linear history sync for master/dev).

v1.0.7

v1.0.7

This release focuses on documentation usability and benchmark reproducibility, making onboarding and cross-framework comparison easier for new users.

Highlights

• Added a one-command quickstart script for end-to-end local validation:
  • scripts/quickstart.sh
  • Automatically starts MySQL/Redis (Docker), prepares data, runs seed/m2m/m2r/r2m/r2r, and validates output counts.
• Added a one-command Hadoop vs mrkit benchmark entry:
  • scripts/benchmark_hadoop_compare.sh
  • Runs runner/bin modes across reducer matrix and generates a merged summary CSV.
• Added explicit environment pre-check in README:
  • one-line Environment Check command for required tools.

Benchmark & Hadoop Script Improvements

• Updated benchmark scripts to be more cross-user compatible:
  • scripts/run_host_hadoop_vs_mrkit_count.sh
  • scripts/run_host_hadoop_vs_mrkit_count_bin.sh
• GO_BIN now defaults to go (PATH-based), with optional explicit override.
• Added clearer error messaging when Go command is unavailable.
• Added optional Hadoop auto-install behavior in comparison workflow (with switch to disable).

Documentation Updates

• README quickstart now emphasizes copy-paste runnable path.
• Added one-line benchmark entry and dependency behavior notes to:
  • docs/performance-analysis.md
  • docs/benchmark.md
  • docs/repro-checklist.md

Included Commits Since v1.0.6

• da8cb76 docs(quickstart): add one-command mysql/redis setup and validation
• 9abf2dc docs(benchmark): add one-line hadoop compare and env checks

v1.0.6

v1.0.6

Highlights

• Added host-vs-host benchmark scripts for mrkit-go vs local Hadoop Streaming:
  • scripts/run_host_hadoop_vs_mrkit_count.sh
  • scripts/run_host_hadoop_vs_mrkit_count_bin.sh
• Added a dedicated performance analysis page with same-host comparison matrix and interpretation:
  • docs/performance-analysis.md
• Clarified benchmark and runtime guidance in docs:
  • docs/benchmark.md
  • docs/config-driven-flow.md
  • README.md

Runtime and stability updates included in this merge

• Improved runtime stability and performance path details from dev merge:
  • plugin build race protection with file lock + atomic replace (batch/plugin_build.go)
  • worker/master runtime robustness and IMD codec path improvements
  • Docker and reproducible command fixes in docs

Notes

• For functional checks, go run is fine.
• For performance baselines and production-like runs, prefer prebuilt binaries (go build then run).

v1.0.5

v1.0.5

This release focuses on documentation reproducibility and onboarding clarity.

What's changed

• Added a new reproducible checklist:
  • docs/repro-checklist.md
• Checklist includes:
  • Local config-driven verification
  • Redis seed flow for redis -> *
  • Four cross-DB E2E paths:
    • mysql -> mysql
    • mysql -> redis
    • redis -> mysql
    • redis -> redis
  • Docker-based verification commands (legacy + config-driven)
  • Cleanup commands after test runs
• Added links to this checklist in:
  • Root README.md
  • example/batch-minimal/README.md

Why this release

• Users can now reproduce project validation with copy-paste commands.
• Reduces setup friction for first-time users across local and Docker environments.

v1.0.4

v1.0.4

This release focuses on runtime reliability and repeatable cross-database E2E verification.

Highlights

• Added a standardized E2E entrypoint:
  • make e2e-mysql-redis
  • scripts/e2e.sh
• Stabilized worker registration against startup races:
  • Worker register RPC now retries with backoff instead of failing fast on transient Unavailable / DeadlineExceeded.
• Improved E2E execution robustness:
  • Per-flow config check before run
  • Retry on transient failures
  • Per-run timeout guard
  • Built-in plugin cache cleanup to avoid ABI mismatch after package/runtime updates
  • Randomized transform port per run to reduce collisions

Included Commits

• 232a154 feat(e2e): add make target and mysql-redis end-to-end script
• b3ef83c fix(runtime): retry worker registration and stabilize cross-db e2e

Validation

• make e2e-mysql-redis completed with all flow categories:
  • seed (mysql -> redis event source)
  • m2m (mysql -> mysql)
  • m2r (mysql -> redis)
  • r2m (redis -> mysql)
  • r2r (redis -> redis)

v1.0.3

Highlights

• Simplified docs entrypoint: root README now focuses on quickstart + links, with detailed guidance moved to docs pages.
• Reorganized example flows by use case under example/batch-minimal/flows/:
  • smoke/
  • cross-db/
  • seed/
  • benchmark/
• Added a flow index table in example README to make scenario selection easier.

Runtime Refactor

• Introduced a runner abstraction in batch:
  • batch/runner.go
  • batch/runner_legacy.go
• batch flow and pipeline paths now execute via runner interface instead of directly coupling to legacy runtime internals.
• This improves extensibility for future runtime strategies.

Legacy CLI Cleanup

• Moved legacy entrypoints to explicit paths to reduce confusion for new users:
  • cmd/legacy/main/main.go
  • cmd/legacy/master/main.go
  • cmd/legacy/worker/main.go
• Updated docs accordingly.

Docs and UX Fixes

• Added dedicated Redis seed flow config for redis-source setup:
  • example/batch-minimal/flows/seed/flow.seed.redis_source_event.json
• Removed sed-based temporary setup from docs in favor of stable config files.
• Verified runnable docs paths for quickstart, cross-db, benchmark, and legacy workflows.

Validation Summary

• README/docs runnable command set was executed end-to-end in local environment.
• Config checks and key flows passed, including:
  • mysql -> mysql
  • mysql -> redis
  • redis -> mysql
  • redis -> redis

v1.0.2

Highlights

• Added a docs-focused cleanup so a new user can run MySQL/Redis config-driven flows with fewer environment pitfalls.
• Added dedicated docs pages for config-driven flow, transforms, benchmarks, library usage, and legacy entrypoints.

Fixes

• Fixed worker port allocation logic in the legacy MapReduce runtime:
  • worker ports now derive from MR_PORT/master base port using numeric increment.
  • removed fragile string concatenation pattern like :1000%v.
  • when a worker port is already occupied, runtime now skips to the next candidate port.
• This resolves the previous invalid-port pattern that could appear when worker count grew (for example 100016).

Docs Updates

• Reworked quickstart and cross-DB walkthroughs.
• Clarified runnable command order for MySQL/Redis end-to-end examples.
• Added guidance for config checks and practical rerun workflow.

Verification

• Re-ran config-driven E2E flows across:
  • mysql -> mysql
  • mysql -> redis
  • redis -> mysql
  • redis -> redis
• Re-ran legacy single-process and master/worker command paths.

v1.0.1

Full Changelog: v0.1.2...v1.0.1

v1.0.0

v1.0.0

This is the first stable release of mrkit-go, evolving the project from a MySQL-specific workflow into a config-driven and extensible batch processing framework.

Highlights

• Introduced a unified batch module for Source -> Transform -> Sink execution
• Added built-in transforms: count, minmax, and topN (no manual .so required)
• Enabled cross-database pipelines:
  • MySQL -> Redis
  • Redis -> MySQL
  • Redis -> Redis
• Added flow config validation via -check and config-based benchmark output
• Completed modular refactor to support future database/source/sink extensions

Major Changes

• Package rename: mysqlbatch -> batch
• CLI refactor: cmd/mysqlbatch -> cmd/batch
• Example layout refactor:
  • example/batch-minimal/mysqlbatch/*
  • example/batch-minimal/redisbatch/*
• Architecture split:
  • Core orchestration and abstractions in batch
  • MySQL implementations in batch/mysql_batch
  • Redis implementations in batch/redis_batch
• flow.go now directly calls subpackage factories (intermediate flow driver files removed)

Breaking Changes

The following changes are not backward-compatible:

• Go import path:
  • github.com/emptyOVO/mrkit-go/mysqlbatch
  • -> github.com/emptyOVO/mrkit-go/batch
• CLI command:
  • go run ./cmd/mysqlbatch ...
  • -> go run ./cmd/batch ...
• Example/config paths:
  • from example/mysqlbatch-minimal/*
  • to:
    • example/batch-minimal/mysqlbatch/*
    • example/batch-minimal/redisbatch/*

Upgrade Guide

1. Update imports to github.com/emptyOVO/mrkit-go/batch
2. Update scripts/commands to cmd/batch
3. Update config/example paths to example/batch-minimal/...
4. Run config validation first:
   • go run ./cmd/batch -check -config <your-flow.json>

Verification

This release has been validated with:

• Flow config checks (-check) passing
• Successful compilation for batch, mysql_batch, and redis_batch
• End-to-end Redis -> Redis benchmark execution with stage duration output

v0.1.2

Full Changelog: v0.1.1...v0.1.2