v3.1.0: Heartbeat/Lease/Recovery Integration

New Features

• Full heartbeat/lease/recovery integration into cluster service
• NodeManager, LeaseManager, Watchdog, Recovery wired into app
• Lease-based task scheduling and automatic recovery
• E2E tested on multi-machine Proxmox LXC environment

Bug Fixes

• Fixed HeartbeatRequest missing load field
• Fixed _on_watchdog_event triggering NodeEvent for Recovery
• Fixed RevokeAllForNode() handling both active and expired leases
• Fixed tasks_mod.init receiving lease_manager

Testing

• 31 new E2E tests (heartbeat lifecycle, lease lifecycle, failure recovery, integration, edge cases)
• 170+ total tests pass

v3.0.0 — Pure Python Rewrite

Changelog

All notable changes to hermes-agent-cluster will be documented in this file.

The format is based on Keep a Changelog,
and this project adheres to Semantic Versioning.

---

变更日志

所有对 hermes-agent-cluster 的重要变更都将记录在此文件中。

格式基于 Keep a Changelog，
本项目遵循 语义化版本。

---

[v3.0.0] - 2026-05-18

Added / 新增

• Pure Python backend: Complete rewrite from Go to Python — 7,090 LOC across 10 modules (core, hooks, lease, models, recovery, routers, state, sync) / 纯Python后端：从Go完全重写为Python — 10个模块共7,090行代码
• 48 REST API endpoints: Full CRUD for tasks, nodes, leases, federation, hooks, recovery, config, schedule, sync, workflow, visualization / 48个REST API端点：任务、节点、租约、联邦、钩子、恢复、配置、调度、同步、工作流、可视化
• Pydantic v2 data models: 79 model classes, 15 SQLite tables + 3 views / Pydantic v2数据模型：79个模型类，15个SQLite表+3个视图
• E2E test suite: 68 tests passing, covering all core modules / 端到端测试套件：68个测试全部通过

v1.2.1

Full Changelog: v1.2.0...v1.2.1

v1.2.0 — Config Management

Config Management

New Features

• Config API: GET/PUT /api/v1/config for reading and updating cluster configuration
• Config Validation: GET /api/v1/config/validate validates configuration and returns detailed errors
• Config YAML Export: GET /api/v1/config/yaml returns raw YAML output
• Hot Restart: PUT /api/v1/config/restart sends restart signal
• Dashboard Config Page: Full configuration management UI at /dashboard/config.html

Improvements

• WithConfig() server option for config persistence
• Auto-save configuration on PUT requests
• Config validation on all write operations

Bug Fixes

• Fixed missing /config/validate, /config/yaml, /config/restart endpoints (E2E discovered)

v1.1.0 — Auto-Start + Dashboard Guide

v1.1.0 Release

New Features

• Plugin auto-start: hermes-cluster service starts automatically when plugin loads
• Dashboard Quick Start Guide: Step-by-step onboarding for new users
• CLI subcommands: serve, status, health, config init, config validate
• Health endpoint: GET /health for load balancers
• Summary endpoint: GET /api/v1/summary for cluster overview
• Config validation: ValidateDetailed() with error messages and fix suggestions
• Federation auth: Token-based cross-cluster authentication

Bug Fixes

• Recovery stats field names now match API response
• Phantom Dependencies column removed from task table
• Federation dispatcher graceful shutdown (WaitGroup)

Testing

• 17/17 test packages pass with -race
• New scheduler tests (372 lines)
• Dashboard UI tests (53 tests, 88% pass rate)

Plugin

• Updated plugin.yaml to v1.1.0
• on_session_start hook for auto-start
• on_session_end hook for graceful shutdown

v1.0.0 — Stable Release

Full Changelog: v0.8.0...v1.0.0

v0.8.0: OpenTelemetry + Prometheus Observability Stack

v0.8.0 — OpenTelemetry + Prometheus Observability Stack

What's New

• Prometheus metrics instrumentation — all 27 hac_ domain metrics now fire from API handlers, background callbacks, and a periodic gauge updater. 11 new integration tests.
• E2E OpenTelemetry validation fixes — fixed forward-referenced promMetrics/stopCh in main.go, nil pointer guard in API handler, rewrote corrupted wiring_test.go.

Changes since v0.6.0

Commit	Description
adbe63c	feat: add Prometheus metrics instrumentation to API handlers
6a7bb1a	fix: E2E OpenTelemetry validation — fix forward references and nil guards

Verification

• Build: clean
• go vet: clean
• Tests: 7 packages pass with -race

Full Changelog: v0.6.0...v0.8.0

v0.7.0 — Prometheus Metrics

Prometheus Metrics Instrumentation

What's New

• 22 unique hac_ domain metrics covering nodes, tasks, leases, and sync
• API middleware records node registration, heartbeat, task creation, completion, and failure events
• Gauge updaters for nodes, tasks, leases, and sync version
• Duration metrics — task processing time measured from CreatedAt to completion/failure

Metrics Catalog

Metric	Type	Description
hac_nodes_registered_total	counter	Node registration events
hac_heartbeats_received_total	counter	Heartbeat events
hac_tasks_created_total	counter	Task creation events
hac_tasks_completed_total	counter	Task completion events
hac_tasks_failed_total	counter	Task failure events
hac_task_duration_seconds	histogram	Task processing duration
hac_nodes_online	gauge	Currently online nodes
hac_tasks_pending	gauge	Pending tasks
hac_leases_active	gauge	Active leases
hac_sync_version	gauge	Current sync version

Testing

• All 27 metrics tests pass with -race detector
• 11 new integration tests for API instrumentation
• Concurrent access race-clean

Commits

• adbe63c feat: add Prometheus metrics instrumentation to API handlers

Full diff: v0.6.0...v0.7.0

v0.6.0: Cluster Visualization

v0.6.0 — Cluster Visualization + Web Dashboard + OpenTelemetry

New Features

• Cluster Visualization — topology graph, metrics panel, timeline event stream
  • GET /api/v1/cluster/topology — cluster topology (nodes, tasks, leases)
  • GET /api/v1/cluster/metrics — aggregated metrics
  • GET /api/v1/cluster/timeline — event timeline
  • GET /api/v1/cluster/viz — combined visualization data
• Web Dashboard — embedded static HTML/CSS/JS dashboard
  • GET /dashboard/ — dashboard entry point
• OpenTelemetry — distributed tracing (OTLP/gRPC or stdout export)
• Prometheus metrics — /metrics endpoint with hac_ prefix

Quality

• Build clean, go vet clean
• All integration tests pass (25+ tests)
• Dashboard integration tests (15 tests)
• Visualization integration tests (10 tests)

Commits

• feat: OpenTelemetry integration (P2-4)
• feat: Web Dashboard implementation
• fix: promote prometheus/client_model to direct dependency
• test: add dashboard integration tests
• docs: update README for v0.6.0

v0.3.0: Global Status View + Dynamic Capability Updates

v0.3.0 — Global Status View + Dynamic Capability Updates

New Features

Global Status View (GET /api/v1/status)

• Cross-cluster unified query for nodes, tasks, and lease status
• Multi-dimensional filtering: ?node=, ?status=, ?capability=
• Returns per-node detail entries + aggregate summary

Dynamic Capability Updates (PATCH /nodes/{id}/capabilities)

• Runtime node capability modification without restart
• Automatic task rescheduling when capabilities change
• Callback chain: capability update → TriggerPendingTasks → reassign matching tasks

E2E Verified

• All 5 test packages pass with race detector (77 total tests)
• Live server confirmed: PATCH updates trigger automatic rescheduling
• Capability filter on status endpoint works correctly

Bug Fixes

• Race condition in TriggerPendingTasks — use atomic PromoteIfPending
• Race condition in TestTriggerAPI_ManualTrigger

Commits

• 7ed8251 feat: Global Status View tests + capability filter + dynamic capabilities
• 9942d11 docs: update README for v0.3.0