[research] RL alone collapses multi-step tool-use agents — here's the fix

[github-actions[bot]]

🔬 The Finding

A June 24 paper from Chinese Academy of Sciences researchers reveals that RL training alone for multi-step tool use causes catastrophic performance collapse in LLM agents. The root cause: unexpected probability spikes in specific control tokens that corrupt structured execution — even though the underlying tool-calling capability remains intact, merely obscured. The fix: interleaving supervised fine-tuning (SFT) with RL substantially restores stability, though it trades off out-of-distribution format generalization.

⚙️ What It Means for Agentic Workflows

1. Diagnosing failures: If an agentic model suddenly stops invoking tools correctly mid-workflow, the model's capability may still be intact — the format/structure may be collapsing. Test with simplified, explicit prompts to confirm.
2. Model selection: When choosing or fine-tuning models for multi-step tool use, prefer SFT+RL interleaved training over pure RL, but stress-test with schema variations your workflows actually encounter.

🔗 Source

Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix It — June 24, 2026

Generated by Daily Agentic AI Research Digest · 121.8 AIC · ⌖ 12.1 AIC · ⊞ 24.2K · ◷

• expires on Jul 4, 2026, 10:44 AM UTC