Skip to content

feat: Add get_error_details() for tool failure analysis#61

Merged
evansenter merged 3 commits into
mainfrom
issue-60-error-details
Jan 8, 2026
Merged

feat: Add get_error_details() for tool failure analysis#61
evansenter merged 3 commits into
mainfrom
issue-60-error-details

Conversation

@evansenter

Copy link
Copy Markdown
Owner

Summary

  • Add get_error_details() MCP tool to show which specific parameters caused errors
  • Add error-details CLI command
  • Update documentation

Problem

analyze_failures() shows "932 Glob errors" but not which patterns failed. Without this, workflow improvements can only identify that errors happen, not why.

Solution

Use SQLite's json_extract() on existing tool_input_json column - no schema changes needed.

Example output:

Glob errors by pattern:
  "*" - 922 errors (rust-genai)  ← problematic pattern found!
  "**/*.rs" - 6 errors

Test plan

  • All 293 tests pass
  • CLI error-details command works
  • MCP tool functional

Closes #60

🤖 Generated with Claude Code

Add endpoint to retrieve detailed error information including the tool
parameters that caused failures. This enables drill-down from aggregate
error counts to actionable specifics.

New MCP tool:
- get_error_details(days, tool, limit) - shows which patterns/commands/files failed

New CLI command:
- session-analytics-cli error-details [--tool NAME] [--limit N]

Key insight: No schema changes needed - uses json_extract() on existing
tool_input_json column to extract Glob/Grep patterns, Bash commands, etc.

Example output:
  Glob errors by pattern:
    "*" - 922 errors (rust-genai)
    "**/*.rs" - 6 errors

Closes #60

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@claude

claude Bot commented Jan 8, 2026

Copy link
Copy Markdown

Prompt: evansenter/dotfiles/.../claude-review.md

Code Review

Summary

This PR adds a new get_error_details() MCP tool and error-details CLI command to show which specific parameters (patterns, commands, files) caused tool failures, enabling drill-down from aggregate error counts to actionable specifics.

Issues Found

Critical

None

Important

  • Missing tests for query_error_details(): The new query function in queries.py:1904-2007 has no corresponding tests. Other query functions like query_tool_frequency have tests in test_queries.py. This is a new public function that should be tested to verify:
    • Basic error aggregation works
    • Tool filter parameter works
    • Limit parameter works correctly
    • The join between tool_use and tool_result events is correct

Suggestions

  • src/session_analytics/cli.py:807-817 - No custom CLI formatter registered for error-details output. Without a formatter, the command will output raw JSON even without --json. Consider adding a @_register_formatter similar to _format_failures (line 381) for better human-readable output.
  • Partial implementation of RFC requirements: The linked issue RFC: Add get_error_details() for tool failure analysis #60 proposed group_by parameter and sample_error field in output, which are not implemented. While the current implementation is useful, documenting that these were intentionally deferred would help future maintainers.

Verdict

REQUEST_CHANGES - Missing tests for the new query function is an important gap that should be addressed before merge.


Automated review by Claude Code

Addresses reviewer feedback on PR #61 requesting test coverage for
the new error details query function.

Tests cover:
- Basic error aggregation by tool and parameter
- Tool filter parameter
- Limit parameter caps errors per tool
- File path errors for Edit/Read/Write
- Grep/Glob pattern extraction with search_path
- Empty database handling
- Days filter excludes old errors

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@claude

claude Bot commented Jan 8, 2026

Copy link
Copy Markdown

Prompt: evansenter/dotfiles/.../claude-review.md

Code Review

Summary

This PR adds a get_error_details() MCP tool and error-details CLI command that enables drill-down from aggregate error counts (e.g., "932 Glob errors") to actionable specifics (e.g., "pattern '*' failed 922 times in rust-genai"). The implementation uses SQLite's json_extract() on existing tool_input_json columns with no schema changes required.

Previous Review Status

The previous review's Important issue (missing tests) has been fully addressed - comprehensive tests were added in TestQueryErrorDetails class covering: basic error aggregation, tool filtering, limit parameter, file path errors, Grep with search_path, empty results, and days filtering.

Issues Found

Critical

None

Important

None

Suggestions

  • src/session_analytics/cli.py:807-817 - No custom CLI formatter registered for error-details output. Without a formatter, the command outputs raw JSON (nested dict structure) even without --json flag. Other similar commands like failures have formatters (line 381) for human-readable output. Consider adding a @_register_formatter that formats errors grouped by tool, e.g.:
    Error Details (last 7 days)
    Total errors: 928
    
    Glob (922 errors):
      "*": 917 errors (rust-genai)
      "**/*.rs": 5 errors (rust-genai)
    
    Bash (6 errors):
      "git status": 6 errors (dotfiles)
    

Verdict

REQUEST_CHANGES - One suggestion for improved CLI UX.


Automated review by Claude Code

Addresses reviewer suggestion for human-readable output instead of raw JSON.

Output now shows errors grouped by tool with counts and context:

  Error Details (last 7 days)
  Total errors: 3403

  Bash (2053 errors):
    'pwd': 621 errors (genai)
    'ls': 329 errors (gemicro)
    ...

  Glob (943 errors):
    '*': 922 errors in /path/to/rust-genai
    ...

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@claude

claude Bot commented Jan 8, 2026

Copy link
Copy Markdown

Prompt: evansenter/dotfiles/.../claude-review.md

Code Review

Summary

This PR adds a get_error_details() MCP tool and error-details CLI command that enables drill-down from aggregate error counts to actionable specifics (e.g., which Glob pattern failed 922 times). Uses SQLite's json_extract() on existing tool_input_json column with no schema changes required.

Previous Review Status

Both issues from prior reviews have been addressed:

  • Missing tests: Comprehensive test suite added in TestQueryErrorDetails class (7 test methods covering aggregation, filtering, limits, file paths, search paths, empty results, and date filtering)
  • Missing CLI formatter: _format_error_details() formatter is now present at cli.py:404-437

Issues Found

Critical

None

Important

None

Suggestions

None

Verdict

APPROVE - Code looks good, no issues found. The implementation follows project conventions (CLI/MCP parity, proper storage API usage), includes thorough tests, and the documentation updates in guide.md and README.md are accurate.


Automated review by Claude Code

@evansenter evansenter merged commit 417e77b into main Jan 8, 2026
3 checks passed
@evansenter evansenter deleted the issue-60-error-details branch January 8, 2026 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RFC: Add get_error_details() for tool failure analysis

1 participant