Infinite tool loops leading to too_many_requests due to AOAI tool/function behavior

[marklicata]

Type: Bug (runtime control / orchestration)
Priority: Critical (causes hard failure + cost explosion)

Problem Statement
Amplifier frequently enters infinite or near‑infinite execution loops when using AOAI, especially in scenarios involving tool or function calls.
These loops:

• Burn massive token volumes
• Trigger 429 (too_many_requests) errors
• Leave users unsure whether Amplifier is making progress or stalled

Observations

• Multiple users independently observed the same looping → 429 pattern [Amplifier...Recording | Video]
• Loops often occur after seemingly valid planning phases
• This is a known OpenAI/AOAI issue tied to tool/function call semantics
• Users were forced to manually interrupt and abandon runs

Impact

• Hard task failure after long waits
• Massive, unbounded token consumption
• Loss of trust in autonomous mode
• Makes Amplifier unsafe to run unattended

Requested Changes
Loop detection

• Detect repeated tool/function call patterns
• Identify non‑advancing execution states

Hard execution guards

• Max tool call count per task
• Max wall‑clock duration per run

Graceful failure

• Stop execution early
• Report why execution halted
• Preserve partial artifacts (plans, drafts)

Acceptance Criteria

• Amplifier cannot run indefinitely without progress
• 429 errors are handled with retry/backoff or early termination
• Users always understand why a run stopped

[robotdad]

Custom Safety Guards via Hooks

Thanks for the detailed description of the looping behavior you're experiencing. The symptoms you describe (repeated tool calls, high token consumption, eventual 429 errors) sound like they could benefit from custom safety policies tailored to your specific environment and risk tolerance.

Amplifier's hook system is designed for exactly this kind of use case - implementing custom execution policies that fit your situation. Since the right thresholds and intervention strategies will vary by environment and use case, building your own safety guard hook gives you precise control and lets you unblock yourself immediately.

Starting Point: Safety Guard Hook

Here's a starting point for a custom safety guard hook that implements the patterns you mentioned:

# safety_guard_hook.py
import time
from collections import defaultdict
from typing import Any, Dict

class SafetyGuardHook:
    """
    Custom hook for detecting and preventing runaway execution patterns.
    
    Configurable thresholds let you tune behavior for your environment.
    """
    
    def __init__(self, config: Dict[str, Any]):
        # Configurable thresholds
        self.max_wall_clock_seconds = config.get("max_wall_clock_seconds", 1800)  # 30 min default
        self.max_tool_calls_per_session = config.get("max_tool_calls", 200)
        self.loop_detection_window = config.get("loop_detection_window", 5)
        
        # Session tracking
        self.session_start_times = {}
        self.session_tool_counts = defaultdict(int)
        self.recent_tool_calls = defaultdict(list)  # Track recent calls for loop detection
    
    async def __call__(self, event: str, data: Dict[str, Any]) -> Dict[str, Any]:
        session_id = data.get("session_id")
        
        # Track session start
        if event == "session:start":
            self.session_start_times[session_id] = time.time()
            return {"action": "continue"}
        
        # Monitor tool execution
        if event == "tool:execute:pre":
            tool_name = data.get("tool_name")
            tool_args = str(data.get("arguments", {}))
            
            # Increment tool call counter
            self.session_tool_counts[session_id] += 1
            
            # Check total tool calls threshold
            if self.session_tool_counts[session_id] > self.max_tool_calls_per_session:
                return {
                    "action": "deny",
                    "reason": f"Safety limit: Exceeded {self.max_tool_calls_per_session} tool calls"
                }
            
            # Track for loop detection
            call_signature = f"{tool_name}:{tool_args}"
            self.recent_tool_calls[session_id].append(call_signature)
            
            # Keep only recent calls in memory
            if len(self.recent_tool_calls[session_id]) > self.loop_detection_window:
                self.recent_tool_calls[session_id].pop(0)
            
            # Detect loops: same tool with same args repeated
            if len(self.recent_tool_calls[session_id]) >= self.loop_detection_window:
                recent_calls = self.recent_tool_calls[session_id]
                if len(set(recent_calls)) == 1:  # All calls identical
                    return {
                        "action": "deny",
                        "reason": f"Loop detected: {tool_name} called {self.loop_detection_window}+ times with same arguments"
                    }
        
        # Monitor execution time
        if event == "turn:start":
            elapsed = time.time() - self.session_start_times.get(session_id, time.time())
            
            if elapsed > self.max_wall_clock_seconds:
                return {
                    "action": "inject_context",
                    "context_injection": f"⚠️ Session has been running for {elapsed/60:.1f} minutes. Please conclude your work and provide final results.",
                    "user_message": f"Safety timeout approaching ({elapsed/60:.1f} min elapsed)"
                }
        
        return {"action": "continue"}

Configuration

Add to your settings.yaml:

hooks:
  - module: safety-guard  # Your custom hook module
    config:
      max_wall_clock_seconds: 1800      # 30 minutes
      max_tool_calls: 200                # Total tool calls per session
      loop_detection_window: 5           # Identical calls to trigger loop detection

Building Your Hook Module with Amplifier

Rather than building this manually, you can have Amplifier create the hook module for you:

amplifier run "Read this GitHub issue thread (https://github.com/microsoft/amplifier/issues/155) and create a custom safety guard hook module based on the discussion. Package it as an installable Python module with the hook code, configuration, and documentation."

Important: Save the session ID from this build session. You can return to it later to evolve the hook as you learn what works for your situation:

amplifier resume <session-id>
# Then: "Adjust the loop detection to trigger after 3 identical calls instead of 5"

Testing and Tuning Your Hook

Once you have the hook deployed, use it in your normal workflow. When you encounter a session where the hook intervened (or should have), analyze it:

amplifier task --agent foundation:session-analyst --instruction "
Analyze session <SESSION_ID> to evaluate how my safety guard hook performed.

Show me:
1. When and why the hook intervened (denied actions, injected context)
2. Tool call patterns - did the hook catch loops correctly?
3. False positives - did the hook block legitimate work?
4. False negatives - did loops occur that the hook missed?
5. Recommended threshold adjustments based on this session's behavior
"

This lets you iteratively refine your hook based on real-world performance in your environment.

Tuning for Your Environment

The thresholds in the starting point are conservative defaults. Based on your specific symptoms, you might adjust:

• Lower max_tool_calls if you're seeing 100+ tool calls in loops
• Shorter max_wall_clock_seconds if 30+ minute sessions are problematic
• Adjust loop_detection_window based on legitimate vs problematic repetition patterns
• Add token rate monitoring if you want to catch high-velocity token consumption

Resources

• Hook Contract: See core:docs/contracts/HOOK_CONTRACT.md for the full hook API
• Hook Examples: Check foundation:examples/07-basic-hook-example.py for a simple hook implementation
• Event Types: All available events are documented in core:docs/HOOKS_API.md

---

This is a starting point you can tailor to your specific situation. Does this approach work for your use case? Let us know how it goes!

[robotdad]

Hopefully this helped resolve your problem, reopen if it did not and let me know what is outstanding.