feat: automated handoff compaction in daemon#47
Conversation
New nightshift/compact.py module with compact_handoffs() that auto-compacts numbered handoff files into weekly summaries when 7+ accumulate. Called by all 3 looping daemon scripts (builder, reviewer, overseer) before each cycle. Prevents context bloat from handoff accumulation. - CompactionResult TypedDict + HANDOFF_COMPACTION_THRESHOLD constant - Shell wrapper compact_handoffs() in lib-agent.sh - 14 new tests (650 total passing)
- Fix shell injection in compact_handoffs(): pass path via env var instead of interpolating into Python -c string - Move datetime import to module top level in compact.py - Add compact.py to OPERATIONS.md module table - Create follow-up tasks #43 (TypedDict for _parse_handoff) and #44 (fix same injection pattern in cleanup functions)
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5b33c804cb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for f in files: | ||
| try: | ||
| f.unlink() | ||
| compacted.append(f.name) | ||
| except OSError as exc: |
There was a problem hiding this comment.
Stop deleting handoffs when any file read fails
If _parse_handoff() raises for any numbered file (for example due to transient I/O or permission errors), the code records the error but still unlinks every file in files. In that scenario, at least one handoff was not included in the generated summary yet can still be deleted, which loses historical context irreversibly. Guard the delete phase by aborting when parse errors exist (or delete only files that were successfully parsed and summarized).
Useful? React with 👍 / 👎.
5-agent audit identified task selection as mesa-optimization: agent optimizes session success over project progress. 5 more agents audited the fix and found 7 issues, all resolved. Phase 1 — Prompt fixes: - Remove "smaller in scope" incentive from evolve-auto.md - Queue order authoritative, handoff advisory - "Tasks I Did NOT Pick and Why" in every handoff - Tracker delta required in session reports - Staleness multiplier (5+ sessions = 2x priority) - All-integration-tasks edge case handled Phase 2 — Task queue: - environment: internal | integration tags - blocked_reason subtypes: environment, dependency, design - .next-id for atomic task ID allocation - archive/ for done tasks (daemon auto-archives) - Tagged #12, #28, #29 blocked-environment - Fixed #43 broken frontmatter (duplicate of #40) Phase 3 — Overseer avoidance detection: - 6 new checks: stale tasks, cherry-picking, stuck integration, weak blocks, max attempts, skip accountability Learnings index: - INDEX.md with categorized one-line summaries (31/31 matched) - Agent reads index, opens files only when relevant - Updated CLAUDE.md, evolve.md Step 1 and Step 6l Meta layer vision (8 new tasks #46-#53): - #46 Healer — between-session trend observer - #47 Multi-agent PR review panel - #48 Human escalation (gh issue create + webhook) - #49 Self-evaluation loop against real repos - #50 Prompt self-refinement via strategist - #51 Cross-session cost intelligence - #52 Codebase world model (MODULE_MAP.md) - #53 Agent generates its own tasks across all dimensions Also: OPERATIONS.md stale task queue + --squash example fixed
Split the monolithic code-reviewer into 5 focused specialist agents. Complexity heuristic in evolve.md Step 8 determines review depth: docs-only/small PRs get 1 reviewer, medium 3, complex 4-5 in parallel. All must pass to merge. (task #47)
feat: multi-agent PR review panel (task #47)
Summary
nightshift/compact.pymodule withcompact_handoffs()that auto-compacts numbered handoff files into weekly summaries when 7+ accumulatelib-agent.shcalled by all 3 looping daemon scripts (builder, reviewer, overseer) before each cycleChanges
nightshift/compact.py— new module: parses handoff markdown, generates weekly summaries, deletes originalsnightshift/types.py—CompactionResultTypedDictnightshift/constants.py—HANDOFF_COMPACTION_THRESHOLD = 7scripts/lib-agent.sh—compact_handoffs()shell wrapperscripts/daemon.sh,daemon-review.sh,daemon-overseer.sh— compaction call in housekeeping stepCLAUDE.md— dependency flow updatedTest plan
make checkpasses (ruff + mypy + 650 tests + dry-runs + shell syntax + ASCII + install refs)