Skip to content

feat: stay-in-scope boundary for contract-first impl agents#663

Merged
lwgray merged 6 commits into
developfrom
fix/contract-scope-boundary-instruction
May 27, 2026
Merged

feat: stay-in-scope boundary for contract-first impl agents#663
lwgray merged 6 commits into
developfrom
fix/contract-scope-boundary-instruction

Conversation

@lwgray

@lwgray lwgray commented May 27, 2026

Copy link
Copy Markdown
Owner

What is this system, briefly

Marcus is a coordination platform for multi-agent software builds. When Marcus uses the contract-first decomposition strategy, each implementation task carries a contract — a shared interface document that defines the coordination surface with sibling agents. The agent's job is to implement what the contract specifies, nothing more.

This PR adds one new instruction to the agent's task description when it's a contract-first task: stay inside the scope your contract names; integration is a downstream concern, not yours. It's a stack-agnostic, decomposer-aware fix for the failure mode we've seen across the last six snake-game runs.

Why (the user-visible problem)

Across snake-baton-1, snake-scaffold-2 (v1 + v2), snake-decomposer-1, and snake-overfrag-1, contract-first implementation agents reached outside their domain to wire their module into shared infrastructure — usually the project entry-point file. Two impl agents doing the same thing in parallel produced merge conflicts that the rebase-recovery (#656) couldn't resolve. Tasks ended up BLOCKED.

snake-overfrag-1 (today) is the cleanest example: agent_2_4's branch wrote to src/main.js, src/gameRenderer.js (the other agent's file), and several test/integration files outside the Game-State-and-Physics domain. Result: merge conflict with agent_1_3's branch, task BLOCKED.

The instruction the agent already has tells them what they OWN (the contract) but not what to STAY OUT OF. Without the boundary, they default to "make my module callable end-to-end" — which means reaching into wiring.

Why prompt instruction, not a structural change

Two corrections worth surfacing:

  1. Agents do not see the task graph. Adding a separate "Compose" or "Integration" task to the DAG creates an owner, but the impl agent never sees that other task. The only channel that reaches the impl agent is its own task description.
  2. LLMs follow workflow instructions reliably. Marcus's entire operation depends on this — request_next_task, report_task_progress, log_decision, log_artifact, read-the-contract-before-writing — all are first-class prompt instructions agents do comply with. The task_name field omission from PR Scaffold paths are not anchored to implementation tasks — agents invent paths inconsistently #659 was schema-following (a weaker, less salient channel), not the same as a prose instruction in the task body.

What the boundary text says

Added as a new block inside Layer 1.3 (CONTRACT RESPONSIBILITY) of build_tiered_instructions, between the contract-file Read() instruction and the scope-annotation legend:

🎯 STAY IN YOUR CONTRACT'S SCOPE:
Your contract IS the coordination surface. Implement what your contract
specifies; do NOT modify code outside your contract's scope to make your
module 'callable' or 'integrated' with other parts of the system.

Integration is a separate, downstream concern. If your work cannot be
invoked by other code without modifying their files, that is an
integration concern handled by a later task — not yours. Log a decision
or artifact describing the integration point you would have wired, then
stop. The downstream integration agent will pick it up from your artifact.

Reaching outside your contract's scope to wire your module into shared
infrastructure is the #1 cause of merge conflicts in contract-first
runs — the next agent's branch touched the same file you did. Stay in
your lane and let the integration step do its job.

Stack-agnostic. No file names, no manifests, no language assumptions. Works for Python (main.py / pyproject.toml), Rust (main.rs / Cargo.toml), JavaScript (main.js / package.json), Go, whatever Marcus runs against.

Decomposer-aware. Only fires when the task has a responsibility set (the contract-first marker). Feature-based tasks have no contract scope to bound them to, so the instruction would be confusing there — and it doesn't render.

Bright-line check (Multi-Agency Proclamation)

The instruction tells agents what NOT to modify. That's coordination (defining the boundary), not control (telling them HOW). The agent retains full autonomy over:

  • HOW they implement their contract (class vs function, libraries, helpers, internal architecture)
  • WHAT files they write to within their scope
  • WHAT data shapes they use internally

Two agents given the same boundary can still produce legitimately different implementations. The bright line stays.

Where to look in the code first

File Purpose
src/marcus_mcp/tools/task.py (build_tiered_instructions, Layer 1.3) The new STAY IN YOUR CONTRACT'S SCOPE block, between Read the contract file and scope_annotation legend
tests/unit/mcp/test_contract_responsibility_prompt.py (TestStayInScopeBoundary) 3 new tests covering: fires for contract-first tasks, absent for legacy tasks, no stack-specific tokens

How to verify it works

  1. Run the new tests:
    python -m pytest tests/unit/mcp/test_contract_responsibility_prompt.py -v
    Expected: 25 tests pass (22 existing + 3 new).
  2. Regression sweep:
    python -m pytest tests/unit/mcp/ tests/unit/marcus_mcp/ -q
    Expected: 219 pass, 37 skipped, 0 fail.
  3. Mypy:
    python -m mypy src/marcus_mcp/tools/task.py
    Expected: clean.
  4. Manual E2E (post-merge): rerun a snake project. Check:
    • When an impl agent picks up a contract-first task, the prompt contains STAY IN YOUR CONTRACT'S SCOPE.
    • After the run, the impl agents' worktree diffs should be confined to their own contract's scope (no edits to src/main.js, package.json, etc.).
    • The Compose task (still TODO at end of impl phase) is the only place wiring happens.
    • 0 BLOCKED tasks from merge conflicts on shared infrastructure.

Test plan

  • 3 new TestStayInScopeBoundary tests
  • Existing 22 TestContractResponsibilityLayer + persistence-fallback + product-intent tests still pass
  • 219 unit tests pass across tests/unit/mcp + tests/unit/marcus_mcp
  • Mypy clean
  • Manual: rerun snake, observe impl agents staying in their files, no merge-conflict BLOCKED tasks

Related

🤖 Generated with Claude Code

@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Claude finished @lwgray's task —— View job


PR Review Complete

Reviewing PR #663: feat: stay-in-scope boundary for contract-first impl agents

✅ Overall Assessment

This is a well-implemented solution to a real coordination problem in multi-agent systems. The approach is sound, the implementation is clean, and the tests are comprehensive.

🎯 Code Quality & Implementation

Strengths:

  • Clean integration: The boundary instruction fits naturally into the existing build_tiered_instructions function within Layer 1.3 (CONTRACT RESPONSIBILITY)
  • Stack-agnostic design: No hardcoded file names or language-specific assumptions - works for Python, Rust, JavaScript, Go, etc.
  • Proper placement: Instruction appears after contract reading but before scope annotation, which is logical flow
  • Clear messaging: The instruction text is well-written, actionable, and explains both what to do and why

Implementation details:

  • Correctly leverages existing _parse_contract_metadata function
  • Only fires when responsibility is set (contract-first tasks only)
  • Consistent with existing code style and patterns
  • Good use of emoji headers for visual clarity in agent prompts

🧪 Test Coverage & Quality

Excellent test coverage with 3 focused test cases:

  1. test_boundary_instruction_fires_for_contract_first_tasks: ✅ Verifies instruction appears for contract tasks
  2. test_boundary_instruction_absent_for_legacy_tasks: ✅ Ensures no instruction for non-contract tasks
  3. test_boundary_instruction_is_stack_agnostic: ✅ Confirms no language-specific tokens leak into instruction text

Test quality highlights:

  • Uses existing _make_task() helper for consistency
  • Tests both positive and negative cases
  • Stack-agnostic test checks for forbidden tokens like main.js, package.json, Cargo.toml
  • Proper boundary extraction logic with boundary_start/boundary_end

🚀 Performance Considerations

No performance concerns:

  • Instruction added only during prompt building (not runtime-critical)
  • Text concatenation is minimal overhead
  • Only fires for contract-first tasks (selective activation)
  • No new I/O or computational complexity

🔒 Security Concerns

No security issues identified:

  • Pure string concatenation with no user input injection
  • No file system operations or external calls
  • Instruction text is static and safe
  • Leverages existing, tested metadata parsing

🐛 Potential Issues & Considerations

Minor edge case (low priority):
The instruction assumes downstream integration tasks exist. If the Marcus pipeline lacks an integration step, agents might log artifacts that never get consumed. However:

  • This is by design - forces explicit coordination decisions
  • Better to have unused artifacts than merge conflicts
  • Aligns with Multi-Agency Proclamation boundaries

Possible future enhancement:
Consider adding validation that logged integration artifacts are eventually consumed, but this is outside scope of current fix.

🎯 Problem Solution Assessment

Directly addresses stated problem:

  • Root cause: Agents reaching outside contract scope to wire modules
  • Solution: Explicit boundary instruction with clear consequences explanation
  • Evidence-based: Based on actual failures across 4 snake-game runs
  • Surgical fix: Minimal code change with maximum coordination impact

The instruction text effectively explains:

  1. What not to do (modify code outside scope)
  2. Why not to do it (merge conflicts)
  3. What to do instead (log integration artifacts)
  4. Who handles it downstream (integration agent)

🏆 Recommendations

Approve and merge - this PR is ready for production:

Pros:

  • Solves real multi-agent coordination problem
  • Clean, maintainable implementation
  • Comprehensive test coverage
  • Stack-agnostic design
  • Follows existing code patterns
  • No performance or security concerns

⚠️ Minor suggestions for future iterations:

  • Consider adding metrics on how often boundary violations are prevented
  • Could add integration artifact consumption tracking
  • Might benefit from A/B testing with/without instruction

📋 Verification Checklist

Based on the PR description, these verification steps should pass:

  • Code review shows clean implementation
  • Tests cover core scenarios comprehensively
  • (Requires environment) New tests pass: pytest tests/unit/mcp/test_contract_responsibility_prompt.py -v
  • (Requires environment) Regression tests pass: pytest tests/unit/mcp/ tests/unit/marcus_mcp/ -q
  • (Requires environment) Type checking: mypy src/marcus_mcp/tools/task.py
  • (Post-merge) E2E verification: snake project run shows agents staying in scope

Final verdict: LGTM 🎯 - This is a thoughtful, well-executed solution to a genuine multi-agent coordination challenge.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 010ccda5f6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/marcus_mcp/tools/task.py Outdated
Comment on lines +1439 to +1444
"Integration is a separate, downstream concern. If your "
"work cannot be invoked by other code without modifying "
"their files, that is an integration concern handled by a "
"later task — not yours. Log a decision or artifact "
"describing the integration point you would have wired, "
"then stop. The downstream integration agent will pick it "

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Exclude composition tasks from the no-integration boundary

In multi-domain contract-first projects, build_composition_task() creates the downstream composition task with source_type="composition_synthesis" and responsibility="Wires the application entry point" (src/integrations/composition_synthesis.py:190-209), so _parse_contract_metadata() treats it as a responsibility-bearing task and this new Layer 1.3 block also renders for that task. The added instruction tells that exact composition agent that integration is “not yours” and to stop after logging an artifact, which directly contradicts the composition task’s required deliverable to wire the entry point; those runs can now leave the product unwired or cause the composition agent to report a blocker instead of doing its job.

Useful? React with 👍 / 👎.

@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Claude finished @lwgray's task —— View job


PR Review Complete

Reviewing PR #663: feat: stay-in-scope boundary for contract-first impl agents

Todo List:

  • Read and analyze changed files
  • Evaluate code quality and best practices
  • Check for potential bugs and issues
  • Assess performance considerations
  • Review security concerns
  • Analyze test coverage
  • Consider existing review feedback (Codex comment)
  • Provide comprehensive review feedback

✅ Overall Assessment

This PR addresses a critical coordination problem in multi-agent systems with a well-implemented solution. The approach is sound, implementation is clean, and tests are comprehensive. However, there's one critical issue that needs attention before merging.

🚨 Critical Issue: Composition Task Exclusion

The good news: The code already includes a fix for the Codex P1 review feedback! I can see in src/marcus_mcp/tools/task.py:2094-2113 that there's a _is_composition_task(task) check that prevents the boundary instruction from appearing for composition tasks:

if not _is_composition_task(task):
    contract_notice += (
        "\n\n🎯 STAY IN YOUR CONTRACT'S SCOPE:\n"
        # ... boundary instruction text
    )

However: I notice that the Codex review comment still shows as unaddressed. The fix appears to be already implemented in the latest commit, but the commit being reviewed by Codex (010ccda5f6) might be an earlier version.

🎯 Code Quality & Implementation

Strengths:

  • Clean integration: Boundary instruction fits naturally into existing build_tiered_instructions function within Layer 1.3
  • Proper placement: Instruction appears after contract reading but before scope annotation - logical flow
  • Stack-agnostic design: No hardcoded file names (main.js, package.json, etc.) - works across Python, Rust, JavaScript, Go
  • Clear messaging: Instruction text is actionable and explains both what to do and why
  • Defensive programming: Correctly excludes composition tasks that need integration capabilities

Implementation details:

  • Leverages existing _parse_contract_metadata function correctly
  • Only fires when responsibility is set (contract-first tasks only)
  • Consistent with existing code style and patterns
  • Good use of emoji headers for visual clarity in agent prompts

🧪 Test Coverage & Quality

Excellent test coverage with 3 focused test cases in TestStayInScopeBoundary:

  1. test_boundary_instruction_fires_for_contract_first_tasks: Verifies instruction appears for contract tasks
  2. test_boundary_instruction_absent_for_legacy_tasks: Ensures no instruction for non-contract tasks
  3. test_boundary_instruction_skipped_for_composition_task: Critical test - confirms composition tasks don't get the boundary instruction
  4. test_boundary_instruction_is_stack_agnostic: Confirms no language-specific tokens leak in

Test quality highlights:

  • Uses existing _make_task() helper for consistency
  • Tests both positive and negative cases including the composition exclusion
  • Stack-agnostic test checks for forbidden tokens
  • Proper boundary extraction logic with boundary_start/boundary_end
  • Tests the exact scenario flagged by Codex review

🚀 Performance Considerations

No concerns:

  • Instruction added only during prompt building (not runtime-critical)
  • Text concatenation has minimal overhead
  • Only fires for contract-first tasks (selective activation)
  • No new I/O operations or computational complexity

🔒 Security Concerns

No issues identified:

  • Pure string concatenation with no user input injection
  • No file system operations or external calls
  • Instruction text is static and safe
  • Uses existing, tested metadata parsing

🐛 Potential Issues & Edge Cases

Minor considerations:

  1. Integration artifact consumption: The instruction assumes downstream integration tasks exist. If lacking, agents might log artifacts that go unused. However, this is by design - better unused artifacts than merge conflicts.

  2. Codex review timing: The Codex comment appears to address an older commit. The current code already includes the composition task exclusion fix.

🎯 Problem Solution Assessment

Directly addresses root cause:

  • Problem: Agents reaching outside contract scope causing merge conflicts
  • Solution: Explicit boundary instruction with consequences explanation
  • Evidence: Based on actual failures across 4 documented snake-game runs
  • Surgical approach: Minimal code change with maximum coordination impact

The instruction text effectively:

  1. States what not to do (modify code outside scope)
  2. Explains why (merge conflicts)
  3. Provides alternative action (log integration artifacts)
  4. Identifies downstream owner (integration agent)

🏆 Recommendations

APPROVE with minor verification - This PR is well-implemented and ready:

Strengths:

  • Solves genuine multi-agent coordination problem
  • Clean, maintainable implementation
  • Comprehensive test coverage including edge cases
  • Stack-agnostic design that scales across languages
  • Already addresses the Codex P1 feedback with composition task exclusion

⚠️ Pre-merge verification:

  • Confirm the Codex review comment applies to an older commit
  • Verify that _is_composition_task correctly identifies composition tasks
  • Run the test suite to ensure composition task exclusion works as expected

🔮 Future considerations:

  • Consider adding metrics on boundary violation prevention
  • Could add integration artifact consumption tracking
  • A/B testing with/without instruction might provide interesting data

📋 Next Steps

  1. Verify test passage: Run pytest tests/unit/mcp/test_contract_responsibility_prompt.py -v
  2. Regression check: Run broader test suite pytest tests/unit/mcp/ tests/unit/marcus_mcp/ -q
  3. Type check: Verify mypy src/marcus_mcp/tools/task.py passes
  4. Post-merge E2E: Test with actual snake project to confirm agents stay in scope

Final verdict: LGTM 🎯 - This addresses a real coordination challenge with a thoughtful, well-tested solution.


@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Claude finished @lwgray's task —— View job


PR Review Complete

Reviewing PR #663: feat: stay-in-scope boundary for contract-first impl agents

✅ Overall Assessment

This is a well-implemented solution to a real coordination problem in multi-agent systems. The approach is sound, the implementation is clean, and the tests are comprehensive.

🎯 Code Quality & Implementation

Strengths:

  • Clean integration: The boundary instruction fits naturally into the existing build_tiered_instructions function within Layer 1.3 (CONTRACT RESPONSIBILITY)
  • Stack-agnostic design: No hardcoded file names or language-specific assumptions - works for Python, Rust, JavaScript, Go, etc.
  • Proper placement: Instruction appears after contract reading but before scope annotation, which is logical flow
  • Clear messaging: The instruction text is well-written, actionable, and explains both what to do and why

Implementation details:

  • Correctly leverages existing _parse_contract_metadata function
  • Only fires when responsibility is set (contract-first tasks only)
  • Consistent with existing code style and patterns
  • Good use of emoji headers for visual clarity in agent prompts

🧪 Test Coverage & Quality

Excellent test coverage with 3 focused test cases:

  1. test_boundary_instruction_fires_for_contract_first_tasks: ✅ Verifies instruction appears for contract tasks
  2. test_boundary_instruction_absent_for_legacy_tasks: ✅ Ensures no instruction for non-contract tasks
  3. test_boundary_instruction_is_stack_agnostic: ✅ Confirms no language-specific tokens leak into instruction text

Test quality highlights:

  • Uses existing _make_task() helper for consistency
  • Tests both positive and negative cases
  • Stack-agnostic test checks for forbidden tokens like main.js, package.json, Cargo.toml
  • Proper boundary extraction logic with boundary_start/boundary_end

🚀 Performance Considerations

No performance concerns:

  • Instruction added only during prompt building (not runtime-critical)
  • Text concatenation is minimal overhead
  • Only fires for contract-first tasks (selective activation)
  • No new I/O or computational complexity

🔒 Security Concerns

No security issues identified:

  • Pure string concatenation with no user input injection
  • No file system operations or external calls
  • Instruction text is static and safe
  • Leverages existing, tested metadata parsing

🐛 Potential Issues & Considerations

Minor edge case (low priority):
The instruction assumes downstream integration tasks exist. If the Marcus pipeline lacks an integration step, agents might log artifacts that never get consumed. However:

  • This is by design - forces explicit coordination decisions
  • Better to have unused artifacts than merge conflicts
  • Aligns with Multi-Agency Proclamation boundaries

Possible future enhancement:
Consider adding validation that logged integration artifacts are eventually consumed, but this is outside scope of current fix.

🎯 Problem Solution Assessment

Directly addresses stated problem:

  • Root cause: Agents reaching outside contract scope to wire modules
  • Solution: Explicit boundary instruction with clear consequences explanation
  • Evidence-based: Based on actual failures across 4 snake-game runs
  • Surgical fix: Minimal code change with maximum coordination impact

The instruction text effectively explains:

  1. What not to do (modify code outside scope)
  2. Why not to do it (merge conflicts)
  3. What to do instead (log integration artifacts)
  4. Who handles it downstream (integration agent)

🏆 Recommendations

Approve and merge - this PR is ready for production:

Pros:

  • Solves real multi-agent coordination problem
  • Clean, maintainable implementation
  • Comprehensive test coverage
  • Stack-agnostic design
  • Follows existing code patterns
  • No performance or security concerns

⚠️ Minor suggestions for future iterations:

  • Consider adding metrics on how often boundary violations are prevented
  • Could add integration artifact consumption tracking
  • Might benefit from A/B testing with/without instruction

📋 Verification Checklist

Based on the PR description, these verification steps should pass:

  • Code review shows clean implementation
  • Tests cover core scenarios comprehensively
  • (Requires environment) New tests pass: pytest tests/unit/mcp/test_contract_responsibility_prompt.py -v
  • (Requires environment) Regression tests pass: pytest tests/unit/mcp/ tests/unit/marcus_mcp/ -q
  • (Requires environment) Type checking: mypy src/marcus_mcp/tools/task.py
  • (Post-merge) E2E verification: snake project run shows agents staying in scope

Final verdict: LGTM 🎯 - This is a thoughtful, well-executed solution to a genuine multi-agent coordination challenge.

@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Claude finished @lwgray's task —— View job


PR Review Complete

Reviewing PR #663: feat: stay-in-scope boundary for contract-first impl agents

Todo List:

  • Read and analyze changed files
  • Evaluate code quality and best practices
  • Check for potential bugs and issues
  • Assess performance considerations
  • Review security concerns
  • Analyze test coverage
  • Consider existing review feedback (Codex comment)
  • Provide comprehensive review feedback

✅ Overall Assessment

This PR addresses a critical coordination problem in multi-agent systems with a well-implemented solution. The approach is sound, implementation is clean, and tests are comprehensive. However, there's one critical issue that needs attention before merging.

🚨 Critical Issue: Composition Task Exclusion

The good news: The code already includes a fix for the Codex P1 review feedback! I can see in src/marcus_mcp/tools/task.py:2094-2113 that there's a _is_composition_task(task) check that prevents the boundary instruction from appearing for composition tasks:

if not _is_composition_task(task):
    contract_notice += (
        "\n\n🎯 STAY IN YOUR CONTRACT'S SCOPE:\n"
        # ... boundary instruction text
    )

However: I notice that the Codex review comment still shows as unaddressed. The fix appears to be already implemented in the latest commit, but the commit being reviewed by Codex (010ccda5f6) might be an earlier version.

🎯 Code Quality & Implementation

Strengths:

  • Clean integration: Boundary instruction fits naturally into existing build_tiered_instructions function within Layer 1.3
  • Proper placement: Instruction appears after contract reading but before scope annotation - logical flow
  • Stack-agnostic design: No hardcoded file names (main.js, package.json, etc.) - works across Python, Rust, JavaScript, Go
  • Clear messaging: Instruction text is actionable and explains both what to do and why
  • Defensive programming: Correctly excludes composition tasks that need integration capabilities

Implementation details:

  • Leverages existing _parse_contract_metadata function correctly
  • Only fires when responsibility is set (contract-first tasks only)
  • Consistent with existing code style and patterns
  • Good use of emoji headers for visual clarity in agent prompts

🧪 Test Coverage & Quality

Excellent test coverage with 3 focused test cases in TestStayInScopeBoundary:

  1. test_boundary_instruction_fires_for_contract_first_tasks: Verifies instruction appears for contract tasks
  2. test_boundary_instruction_absent_for_legacy_tasks: Ensures no instruction for non-contract tasks
  3. test_boundary_instruction_skipped_for_composition_task: Critical test - confirms composition tasks don't get the boundary instruction
  4. test_boundary_instruction_is_stack_agnostic: Confirms no language-specific tokens leak in

Test quality highlights:

  • Uses existing _make_task() helper for consistency
  • Tests both positive and negative cases including the composition exclusion
  • Stack-agnostic test checks for forbidden tokens
  • Proper boundary extraction logic with boundary_start/boundary_end
  • Tests the exact scenario flagged by Codex review

🚀 Performance Considerations

No concerns:

  • Instruction added only during prompt building (not runtime-critical)
  • Text concatenation has minimal overhead
  • Only fires for contract-first tasks (selective activation)
  • No new I/O operations or computational complexity

🔒 Security Concerns

No issues identified:

  • Pure string concatenation with no user input injection
  • No file system operations or external calls
  • Instruction text is static and safe
  • Uses existing, tested metadata parsing

🐛 Potential Issues & Edge Cases

Minor considerations:

  1. Integration artifact consumption: The instruction assumes downstream integration tasks exist. If lacking, agents might log artifacts that go unused. However, this is by design - better unused artifacts than merge conflicts.

  2. Codex review timing: The Codex comment appears to address an older commit. The current code already includes the composition task exclusion fix.

🎯 Problem Solution Assessment

Directly addresses root cause:

  • Problem: Agents reaching outside contract scope causing merge conflicts
  • Solution: Explicit boundary instruction with consequences explanation
  • Evidence: Based on actual failures across 4 documented snake-game runs
  • Surgical approach: Minimal code change with maximum coordination impact

The instruction text effectively:

  1. States what not to do (modify code outside scope)
  2. Explains why (merge conflicts)
  3. Provides alternative action (log integration artifacts)
  4. Identifies downstream owner (integration agent)

🏆 Recommendations

APPROVE with minor verification - This PR is well-implemented and ready:

Strengths:

  • Solves genuine multi-agent coordination problem
  • Clean, maintainable implementation
  • Comprehensive test coverage including edge cases
  • Stack-agnostic design that scales across languages
  • Already addresses the Codex P1 feedback with composition task exclusion

⚠️ Pre-merge verification:

  • Confirm the Codex review comment applies to an older commit
  • Verify that _is_composition_task correctly identifies composition tasks
  • Run the test suite to ensure composition task exclusion works as expected

🔮 Future considerations:

  • Consider adding metrics on boundary violation prevention
  • Could add integration artifact consumption tracking
  • A/B testing with/without instruction might provide interesting data

📋 Next Steps

  1. Verify test passage: Run pytest tests/unit/mcp/test_contract_responsibility_prompt.py -v
  2. Regression check: Run broader test suite pytest tests/unit/mcp/ tests/unit/marcus_mcp/ -q
  3. Type check: Verify mypy src/marcus_mcp/tools/task.py passes
  4. Post-merge E2E: Test with actual snake project to confirm agents stay in scope

Final verdict: LGTM 🎯 - This addresses a real coordination challenge with a thoughtful, well-tested solution.


View job

@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Claude finished @lwgray's task —— View job


Claude is reviewing PR #663

Todo List:

  • Read and analyze changed files
  • Evaluate code quality and best practices
  • Check for potential bugs and issues
  • Assess performance considerations
  • Review security concerns
  • Analyze test coverage
  • Consider existing review feedback (Codex comment)
  • Provide comprehensive review feedback

✅ Overall Assessment

This PR addresses a critical coordination problem in multi-agent systems with a well-implemented solution. The approach is sound, implementation is clean, and tests are comprehensive. However, there's one critical issue that needs attention before merging.

🚨 Critical Issue: Composition Task Exclusion

The good news: The code already includes a fix for the Codex P1 review feedback! I can see in src/marcus_mcp/tools/task.py:2094-2113 that there's a _is_composition_task(task) check that prevents the boundary instruction from appearing for composition tasks:

if not _is_composition_task(task):
    contract_notice += (
        "\n\n🎯 STAY IN YOUR CONTRACT'S SCOPE:\n"
        # ... boundary instruction text
    )

However: I notice that the Codex review comment still shows as unaddressed. The fix appears to be already implemented in the latest commit, but the commit being reviewed by Codex (010ccda5f6) might be an earlier version.

🎯 Code Quality & Implementation

Strengths:

  • Clean integration: Boundary instruction fits naturally into existing build_tiered_instructions function within Layer 1.3
  • Proper placement: Instruction appears after contract reading but before scope annotation - logical flow
  • Stack-agnostic design: No hardcoded file names (main.js, package.json, etc.) - works across Python, Rust, JavaScript, Go
  • Clear messaging: Instruction text is actionable and explains both what to do and why
  • Defensive programming: Correctly excludes composition tasks that need integration capabilities

Implementation details:

  • Leverages existing _parse_contract_metadata function correctly
  • Only fires when responsibility is set (contract-first tasks only)
  • Consistent with existing code style and patterns
  • Good use of emoji headers for visual clarity in agent prompts

🧪 Test Coverage & Quality

Excellent test coverage with 3 focused test cases in TestStayInScopeBoundary:

  1. test_boundary_instruction_fires_for_contract_first_tasks: Verifies instruction appears for contract tasks
  2. test_boundary_instruction_absent_for_legacy_tasks: Ensures no instruction for non-contract tasks
  3. test_boundary_instruction_skipped_for_composition_task: Critical test - confirms composition tasks don't get the boundary instruction
  4. test_boundary_instruction_is_stack_agnostic: Confirms no language-specific tokens leak in

Test quality highlights:

  • Uses existing _make_task() helper for consistency
  • Tests both positive and negative cases including the composition exclusion
  • Stack-agnostic test checks for forbidden tokens
  • Proper boundary extraction logic with boundary_start/boundary_end
  • Tests the exact scenario flagged by Codex review

🚀 Performance Considerations

No concerns:

  • Instruction added only during prompt building (not runtime-critical)
  • Text concatenation has minimal overhead
  • Only fires for contract-first tasks (selective activation)
  • No new I/O operations or computational complexity

🔒 Security Concerns

No issues identified:

  • Pure string concatenation with no user input injection
  • No file system operations or external calls
  • Instruction text is static and safe
  • Uses existing, tested metadata parsing

🐛 Potential Issues & Edge Cases

Minor considerations:

  1. Integration artifact consumption: The instruction assumes downstream integration tasks exist. If lacking, agents might log artifacts that go unused. However, this is by design - better unused artifacts than merge conflicts.

  2. Codex review timing: The Codex comment appears to address an older commit. The current code already includes the composition task exclusion fix.

🎯 Problem Solution Assessment

Directly addresses root cause:

  • Problem: Agents reaching outside contract scope causing merge conflicts
  • Solution: Explicit boundary instruction with consequences explanation
  • Evidence: Based on actual failures across 4 documented snake-game runs
  • Surgical approach: Minimal code change with maximum coordination impact

The instruction text effectively:

  1. States what not to do (modify code outside scope)
  2. Explains why (merge conflicts)
  3. Provides alternative action (log integration artifacts)
  4. Identifies downstream owner (integration agent)

🏆 Recommendations

APPROVE with minor verification - This PR is well-implemented and ready:

Strengths:

  • Solves genuine multi-agent coordination problem
  • Clean, maintainable implementation
  • Comprehensive test coverage including edge cases
  • Stack-agnostic design that scales across languages
  • Already addresses the Codex P1 feedback with composition task exclusion

⚠️ Pre-merge verification:

  • Confirm the Codex review comment applies to an older commit
  • Verify that _is_composition_task correctly identifies composition tasks
  • Run the test suite to ensure composition task exclusion works as expected

🔮 Future considerations:

  • Consider adding metrics on boundary violation prevention
  • Could add integration artifact consumption tracking
  • A/B testing with/without instruction might provide interesting data

📋 Next Steps

  1. Verify test passage: Run pytest tests/unit/mcp/test_contract_responsibility_prompt.py -v
  2. Regression check: Run broader test suite pytest tests/unit/mcp/ tests/unit/marcus_mcp/ -q
  3. Type check: Verify mypy src/marcus_mcp/tools/task.py passes
  4. Post-merge E2E: Test with actual snake project to confirm agents stay in scope

Final verdict: LGTM 🎯 - This addresses a real coordination challenge with a thoughtful, well-tested solution.


View job

@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Claude finished @lwgray's task —— View job


Claude finished @lwgray's task —— View job


PR Review Complete

Reviewing PR #663: feat: stay-in-scope boundary for contract-first impl agents

Todo List:

  • Read and analyze changed files
  • Evaluate code quality and best practices
  • Check for potential bugs and issues
  • Assess performance considerations
  • Review security concerns
  • Analyze test coverage
  • Consider existing review feedback (Codex comment)
  • Provide comprehensive review feedback

✅ Overall Assessment

This PR addresses a critical coordination problem in multi-agent systems with a well-implemented solution. The approach is sound, implementation is clean, and tests are comprehensive. However, there's one critical issue that needs attention before merging.

🚨 Critical Issue: Composition Task Exclusion

The good news: The code already includes a fix for the Codex P1 review feedback! I can see in src/marcus_mcp/tools/task.py:2094-2113 that there's a _is_composition_task(task) check that prevents the boundary instruction from appearing for composition tasks:

if not _is_composition_task(task):
    contract_notice += (
        "\n\n🎯 STAY IN YOUR CONTRACT'S SCOPE:\n"
        # ... boundary instruction text
    )

However: I notice that the Codex review comment still shows as unaddressed. The fix appears to be already implemented in the latest commit, but the commit being reviewed by Codex (010ccda5f6) might be an earlier version.

🎯 Code Quality & Implementation

Strengths:

  • Clean integration: Boundary instruction fits naturally into existing build_tiered_instructions function within Layer 1.3
  • Proper placement: Instruction appears after contract reading but before scope annotation - logical flow
  • Stack-agnostic design: No hardcoded file names (main.js, package.json, etc.) - works across Python, Rust, JavaScript, Go
  • Clear messaging: Instruction text is actionable and explains both what to do and why
  • Defensive programming: Correctly excludes composition tasks that need integration capabilities

Implementation details:

  • Leverages existing _parse_contract_metadata function correctly
  • Only fires when responsibility is set (contract-first tasks only)
  • Consistent with existing code style and patterns
  • Good use of emoji headers for visual clarity in agent prompts

🧪 Test Coverage & Quality

Excellent test coverage with 3 focused test cases in TestStayInScopeBoundary:

  1. test_boundary_instruction_fires_for_contract_first_tasks: Verifies instruction appears for contract tasks
  2. test_boundary_instruction_absent_for_legacy_tasks: Ensures no instruction for non-contract tasks
  3. test_boundary_instruction_skipped_for_composition_task: Critical test - confirms composition tasks don't get the boundary instruction
  4. test_boundary_instruction_is_stack_agnostic: Confirms no language-specific tokens leak in

Test quality highlights:

  • Uses existing _make_task() helper for consistency
  • Tests both positive and negative cases including the composition exclusion
  • Stack-agnostic test checks for forbidden tokens
  • Proper boundary extraction logic with boundary_start/boundary_end
  • Tests the exact scenario flagged by Codex review

🚀 Performance Considerations

No concerns:

  • Instruction added only during prompt building (not runtime-critical)
  • Text concatenation has minimal overhead
  • Only fires for contract-first tasks (selective activation)
  • No new I/O operations or computational complexity

🔒 Security Concerns

No issues identified:

  • Pure string concatenation with no user input injection
  • No file system operations or external calls
  • Instruction text is static and safe
  • Uses existing, tested metadata parsing

🐛 Potential Issues & Edge Cases

Minor considerations:

  1. Integration artifact consumption: The instruction assumes downstream integration tasks exist. If lacking, agents might log artifacts that go unused. However, this is by design - better unused artifacts than merge conflicts.

  2. Codex review timing: The Codex comment appears to address an older commit. The current code already includes the composition task exclusion fix.

🎯 Problem Solution Assessment

Directly addresses root cause:

  • Problem: Agents reaching outside contract scope causing merge conflicts
  • Solution: Explicit boundary instruction with consequences explanation
  • Evidence: Based on actual failures across 4 documented snake-game runs
  • Surgical approach: Minimal code change with maximum coordination impact

The instruction text effectively:

  1. States what not to do (modify code outside scope)
  2. Explains why (merge conflicts)
  3. Provides alternative action (log integration artifacts)
  4. Identifies downstream owner (integration agent)

🏆 Recommendations

APPROVE with minor verification - This PR is well-implemented and ready:

Strengths:

  • Solves genuine multi-agent coordination problem
  • Clean, maintainable implementation
  • Comprehensive test coverage including edge cases
  • Stack-agnostic design that scales across languages
  • Already addresses the Codex P1 feedback with composition task exclusion

⚠️ Pre-merge verification:

  • Confirm the Codex review comment applies to an older commit
  • Verify that _is_composition_task correctly identifies composition tasks
  • Run the test suite to ensure composition task exclusion works as expected

🔮 Future considerations:

  • Consider adding metrics on boundary violation prevention
  • Could add integration artifact consumption tracking
  • A/B testing with/without instruction might provide interesting data

📋 Next Steps

  1. Verify test passage: Run pytest tests/unit/mcp/test_contract_responsibility_prompt.py -v
  2. Regression check: Run broader test suite pytest tests/unit/mcp/ tests/unit/marcus_mcp/ -q
  3. Type check: Verify mypy src/marcus_mcp/tools/task.py passes
  4. Post-merge E2E: Test with actual snake project to confirm agents stay in scope

Final verdict: LGTM 🎯 - This addresses a real coordination challenge with a thoughtful, well-tested solution.


View job

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant