Skip to content

v0.6.0-rc77

Pre-release
Pre-release

Choose a tag to compare

@buger buger released this 15 Sep 15:03
· 353 commits to main since this release
3edefca
Fix probe agent JSON schema output for attempt_completion tool (#151)

* Fix probe agent JSON schema output for attempt_completion tool

The probe agent was returning markdown instead of JSON when using attempt_completion
with a JSON schema because the schema formatting step was skipped for completion results.

## Problem
- When agent uses attempt_completion tool + JSON schema provided
- Agent returned markdown text instead of JSON format
- Schema validation/correction was bypassed due to `completionAttempted` flag

## Solution
- Added comprehensive JSON validation and correction for attempt_completion results
- Schema definition detection for attempt_completion responses
- Multi-retry correction loop with escalating prompt urgency (up to 3 attempts)
- Telemetry integration for attempt_completion JSON validation
- Preserves original two-step flow: agent works normally, then gets formatting instructions

## Key Changes
- Modified ProbeAgent.js lines 1013-1125: Added full JSON validation for attempt_completion
- Same correction logic as regular responses (schema confusion detection, JSON correction)
- Enhanced debug logging and telemetry tracking
- Added comprehensive test coverage

## Testing
- All existing tests continue to pass (64 tests total)
- Added new test file for attempt_completion JSON schema scenarios
- Verified schema definition confusion detection and correction works

The agent now properly returns JSON when using attempt_completion with JSON schemas.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix test to avoid API key requirements

Updated attemptCompletionJsonFix.test.js to test the core schema processing
logic directly without requiring ProbeAgent instantiation or API keys.

- Tests schema detection, validation, and correction logic
- Simulates the complete attempt_completion JSON correction flow
- Tests edge cases and error conditions
- No longer requires mocking complex AI SDK dependencies
- All 8 tests now pass successfully

This approach follows the pattern of other integration tests in the codebase
that test the schema utilities directly rather than the full agent workflow.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>