-
-
Notifications
You must be signed in to change notification settings - Fork 17
WAL compaction for long-running orchestrations #563
Copy link
Copy link
Closed
Labels
P1High priorityHigh priorityenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededpythonPythonPythonreliabilityServer resilience and fault toleranceServer resilience and fault toleranceroadmapRoadmap featureRoadmap feature
Metadata
Metadata
Assignees
Labels
P1High priorityHigh priorityenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededpythonPythonPythonreliabilityServer resilience and fault toleranceServer resilience and fault toleranceroadmapRoadmap featureRoadmap feature
Description
For orchestrations running 3+ hours with thousands of ticks, the WAL grows unboundedly. Implement WAL compaction that: checkpoints committed entries, truncates the WAL to only uncommitted entries, and preserves a summary of compacted entries for audit. Different from ORCH-007 (WAL replay) and ORCH-019 (checkpoint/restore) -- this is about WAL size management.
Metadata
Implementation Suggestions
src/bernstein/core/merge_queue.py) and worktree managementsrc/bernstein/core/audit.py) and policy engine (src/bernstein/core/policy_engine.py).sdd/runtime/for replay capabilityRelevant Files
src/bernstein/core/orchestrator.pysrc/bernstein/core/planner.pysrc/bernstein/core/plan_loader.pysrc/bernstein/cli/src/bernstein/core/wal.pyBacklog:
road-047-wal-compaction-for-long-running-orchestrations.yaml