Skip to content

feat(context): HiAgent subgoal-aware context compaction for long-horizon tasks (#2022)#2061

Merged
bug-ops merged 5 commits intomainfrom
issue-2022-subgoal-compaction
Mar 20, 2026
Merged

feat(context): HiAgent subgoal-aware context compaction for long-horizon tasks (#2022)#2061
bug-ops merged 5 commits intomainfrom
issue-2022-subgoal-compaction

Conversation

@bug-ops
Copy link
Owner

@bug-ops bug-ops commented Mar 20, 2026

Summary

Implements HiAgent-inspired subgoal-aware context compaction for improved handling of long-horizon multi-step tasks. Instead of purely token/recency-based compaction, the agent now:

  1. Extracts subgoal state (current vs completed) after each LLM turn via fire-and-forget background task
  2. Tags context segments with their subgoal relevance tier (Active/Completed/Outdated)
  3. Prioritizes compaction of completed/outdated sections, protecting active working memory
  4. Exposes subgoal state in debug dumps and optional TUI status indicator

Key Changes

Architecture

  • SubgoalRegistry: In-memory tracking of subgoal spans and states with dual-phase index rebuild
  • Three-tier relevance: Active (1.0, protected) → Completed (0.3, summarizable) → Outdated (0.1, evictable)
  • Fire-and-forget extraction: Non-blocking LLM call via tokio::spawn + now_or_never
  • Extract-reinsert pattern: For hard compaction, temporarily remove active messages to apply pruning, then re-insert them

New LLM Call

  • Structured prompt: "CURRENT: <1-sentence subgoal>\nCOMPLETED: <1-sentence or NONE>"
  • Configurable via [agent.compression.subgoal_config] with enable/disable flag
  • Fallback to task_aware compaction if LLM call fails or extraction is disabled

Compaction Strategy: SubgoalMig

  • Combines MIG redundancy scoring with subgoal-aware prioritization
  • Scores blocks as: MIG_score * (1.0 if Active | 0.3 if Completed | 0.1 if Outdated)
  • Ensures SideQuests don't trigger eviction of active subgoal messages

Code Review Fixes Applied

  • S1: Preserve tier distinction in rebuild_after_compaction() using span-aware matching
  • S2: Add 4 unit tests for parse_subgoal_extraction_response() covering edge cases
  • S3: Add missing debug dump call in prune_tool_outputs_subgoal_mig()

Testing

  • 6103 tests passing (including new subgoal extraction tests)
  • Zero clippy warnings
  • Ready for code review and CI

@bug-ops bug-ops added feature New functionality context Context management and message handling research Research-driven improvement labels Mar 20, 2026
@github-actions github-actions bot added documentation Improvements or additions to documentation rust Rust code changes core zeph-core crate enhancement New feature or request size/XL Extra large PR (500+ lines) labels Mar 20, 2026
@bug-ops bug-ops force-pushed the issue-2022-subgoal-compaction branch from 9f39418 to a32c04d Compare March 20, 2026 19:03
bug-ops added 5 commits March 20, 2026 20:15
Add SubgoalId, SubgoalState, Subgoal, and SubgoalRegistry to
compaction_strategy.rs. Implements all registry operations:
- push_active with auto-completion defense (M3 fix)
- complete_active, extend_active with incremental tagging (S6 fix)
- tag_range for retroactive tagging (S4 fix)
- subgoal_state lookup, active_subgoal query
- rebuild_after_compaction for post-drain index repair (S1 fix)

All unit tests pass. Gated behind context-compression feature flag.
…ases B-E)

Phase B: SubgoalExtractionResult + CompressionState fields (subgoal_registry,
pending_subgoal, subgoal_user_msg_hash). maybe_refresh_subgoal() two-phase
fire-and-forget using last 6 agent_visible messages (M2 fix). LLM signal
drives transitions via COMPLETED: prefix (S3 fix). Retroactive tagging of
pre-extraction messages on first subgoal creation (S4 fix).

Phase C: score_blocks_subgoal() with Active=1.0/Completed=0.3/Outdated=0.1
tiers plus recency tiebreaker. score_blocks_subgoal_mig() combines subgoal
tier relevance with pairwise MIG redundancy.

Phase D: PruningStrategy::Subgoal and SubgoalMig variants with serde and
FromStr support. prune_tool_outputs_subgoal/subgoal_mig() using shared
evict_sorted_blocks() helper. Dispatch in prune_tool_outputs(). maybe_compact()
Soft tier dispatches to maybe_refresh_subgoal() when subgoal strategy active.
Rebuild after apply_deferred_summaries() to repair shifted indices (S5 fix).

Phase E: Extract active-subgoal messages before drain in compact_context()
and re-insert after pinned messages (S2 fix). rebuild_after_compaction() call
after drain+reinsert to maintain index consistency (S1 fix).
- debug_dump: add dump_subgoal_registry() writing NNN-subgoal-registry.txt
  alongside pruning score dumps for subgoal strategy runs
- /status command: show Pruning strategy, subgoal count, and active subgoal
  description when pruning_strategy is Subgoal or SubgoalMig
- --init wizard: add subgoal and subgoal_mig to pruning strategy selection
  menu (items 4 and 5 in the Select list)
- runner.rs: M4 fix — hard anyhow::bail! if SideQuest eviction and Subgoal
  pruning are both enabled (mutually exclusive subsystems)
S1: Preserve tier distinction in rebuild_after_compaction() - use span-aware matching to ensure Active vs Completed subgoal messages retain their tier assignment after compaction, preventing incorrect eviction by subsequent pruning operations.

S2: Add unit test coverage for parse_subgoal_extraction_response() - implement 4 comprehensive tests covering well-formed input, malformed input with missing CURRENT prefix, and edge case of empty CURRENT line with fallback behavior.

S3: Add missing debug dump call to prune_tool_outputs_subgoal_mig() - ensure consistent debug observability when dumping SubgoalRegistry state, matching the call present in the sibling prune_tool_outputs_subgoal() function.

Also fix empty CURRENT detection in parser by avoiding trim_start() before newline split, ensuring malformed responses correctly trigger fallback to treating entire response as current subgoal.
…feature disabled

The 'applied' variable is only used when context-compression feature is enabled.
Use conditional compilation to avoid unused variable warning in other bundles.
@bug-ops bug-ops force-pushed the issue-2022-subgoal-compaction branch from c8c9ead to 526d7fd Compare March 20, 2026 19:15
@bug-ops bug-ops enabled auto-merge (squash) March 20, 2026 19:15
@bug-ops bug-ops merged commit 6287598 into main Mar 20, 2026
25 checks passed
@bug-ops bug-ops deleted the issue-2022-subgoal-compaction branch March 20, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

context Context management and message handling core zeph-core crate documentation Improvements or additions to documentation enhancement New feature or request feature New functionality research Research-driven improvement rust Rust code changes size/XL Extra large PR (500+ lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

research(context): HiAgent subgoal-aware context compaction for long-horizon task coherence

1 participant