Skip to content

feat: automated handoff compaction in daemon#47

Merged
fazxes merged 3 commits intomainfrom
feat/handoff-compaction
Apr 5, 2026
Merged

feat: automated handoff compaction in daemon#47
fazxes merged 3 commits intomainfrom
feat/handoff-compaction

Conversation

@fazxes
Copy link
Copy Markdown
Member

@fazxes fazxes commented Apr 5, 2026

Summary

  • New nightshift/compact.py module with compact_handoffs() that auto-compacts numbered handoff files into weekly summaries when 7+ accumulate
  • Shell wrapper in lib-agent.sh called by all 3 looping daemon scripts (builder, reviewer, overseer) before each cycle
  • Prevents context bloat from handoff accumulation — previously compaction was just an instruction agents could skip

Changes

  • nightshift/compact.py — new module: parses handoff markdown, generates weekly summaries, deletes originals
  • nightshift/types.pyCompactionResult TypedDict
  • nightshift/constants.pyHANDOFF_COMPACTION_THRESHOLD = 7
  • scripts/lib-agent.shcompact_handoffs() shell wrapper
  • scripts/daemon.sh, daemon-review.sh, daemon-overseer.sh — compaction call in housekeeping step
  • CLAUDE.md — dependency flow updated
  • 14 new tests, 650 total passing

Test plan

  • make check passes (ruff + mypy + 650 tests + dry-runs + shell syntax + ASCII + install refs)
  • Tests cover: below/at/above threshold, weekly format, decisions/issues preservation, non-numbered file exclusion, duplicate weekly suffix, nonexistent/empty dir, custom threshold, auto-create weekly dir, session range ordering, ISO week filename

fazxes added 2 commits April 4, 2026 23:31
New nightshift/compact.py module with compact_handoffs() that auto-compacts
numbered handoff files into weekly summaries when 7+ accumulate. Called by
all 3 looping daemon scripts (builder, reviewer, overseer) before each cycle.
Prevents context bloat from handoff accumulation.

- CompactionResult TypedDict + HANDOFF_COMPACTION_THRESHOLD constant
- Shell wrapper compact_handoffs() in lib-agent.sh
- 14 new tests (650 total passing)
- Fix shell injection in compact_handoffs(): pass path via env var
  instead of interpolating into Python -c string
- Move datetime import to module top level in compact.py
- Add compact.py to OPERATIONS.md module table
- Create follow-up tasks #43 (TypedDict for _parse_handoff) and
  #44 (fix same injection pattern in cleanup functions)
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b33c804cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread nightshift/compact.py
Comment on lines +295 to +299
for f in files:
try:
f.unlink()
compacted.append(f.name)
except OSError as exc:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Stop deleting handoffs when any file read fails

If _parse_handoff() raises for any numbered file (for example due to transient I/O or permission errors), the code records the error but still unlinks every file in files. In that scenario, at least one handoff was not included in the generated summary yet can still be deleted, which loses historical context irreversibly. Guard the delete phase by aborting when parse errors exist (or delete only files that were successfully parsed and summarized).

Useful? React with 👍 / 👎.

Upstream had task #43 (CONTRIBUTING.md). Renumber:
- #43: CONTRIBUTING.md (from upstream)
- #44: ParsedHandoff TypedDict (review follow-up)
- #45: Shell injection fix in cleanup functions (review follow-up)
@fazxes fazxes merged commit 63b8a6f into main Apr 5, 2026
2 checks passed
@fazxes fazxes deleted the feat/handoff-compaction branch April 5, 2026 03:40
fazxes added a commit that referenced this pull request Apr 5, 2026
5-agent audit identified task selection as mesa-optimization: agent
optimizes session success over project progress. 5 more agents audited
the fix and found 7 issues, all resolved.

Phase 1 — Prompt fixes:
- Remove "smaller in scope" incentive from evolve-auto.md
- Queue order authoritative, handoff advisory
- "Tasks I Did NOT Pick and Why" in every handoff
- Tracker delta required in session reports
- Staleness multiplier (5+ sessions = 2x priority)
- All-integration-tasks edge case handled

Phase 2 — Task queue:
- environment: internal | integration tags
- blocked_reason subtypes: environment, dependency, design
- .next-id for atomic task ID allocation
- archive/ for done tasks (daemon auto-archives)
- Tagged #12, #28, #29 blocked-environment
- Fixed #43 broken frontmatter (duplicate of #40)

Phase 3 — Overseer avoidance detection:
- 6 new checks: stale tasks, cherry-picking, stuck integration,
  weak blocks, max attempts, skip accountability

Learnings index:
- INDEX.md with categorized one-line summaries (31/31 matched)
- Agent reads index, opens files only when relevant
- Updated CLAUDE.md, evolve.md Step 1 and Step 6l

Meta layer vision (8 new tasks #46-#53):
- #46 Healer — between-session trend observer
- #47 Multi-agent PR review panel
- #48 Human escalation (gh issue create + webhook)
- #49 Self-evaluation loop against real repos
- #50 Prompt self-refinement via strategist
- #51 Cross-session cost intelligence
- #52 Codebase world model (MODULE_MAP.md)
- #53 Agent generates its own tasks across all dimensions

Also: OPERATIONS.md stale task queue + --squash example fixed
fazxes added a commit that referenced this pull request Apr 5, 2026
Split the monolithic code-reviewer into 5 focused specialist agents.
Complexity heuristic in evolve.md Step 8 determines review depth:
docs-only/small PRs get 1 reviewer, medium 3, complex 4-5 in parallel.
All must pass to merge. (task #47)
fazxes added a commit that referenced this pull request Apr 5, 2026
feat: multi-agent PR review panel (task #47)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant