Metrics tracking for Claude Code trajectories and command executions

[evansenter]

Summary

Build a metrics analyzer that parses existing Claude Code session logs (~/.claude/projects/**/*.jsonl) to extract trajectory data and usage patterns. Use this data to provide evidence-based workflow improvement suggestions via the /improve-workflow command.

Problem / Motivation

The /improve-workflow command currently suggests improvements based on static analysis of the codebase and CLAUDE.md. Without usage data, it can't identify:

• Which commands are actually used frequently vs rarely
• Common tool call sequences that could be automated
• Friction points where users repeat operations or encounter errors
• Opportunities for new commands based on observed patterns

Example use cases for metrics:

• "You run git status && git diff in 80% of sessions - create a /git-summary command?"
• "You always run /pr-feedback twice in a row - should it auto-rerun after fixes?"
• "Your most common workflow: status → diff → edit → commit - streamline this?"

Context

• Discovered during: Discussion about improving the /improve-workflow command
• Key discovery: Claude Code already logs all sessions to ~/.claude/projects/<project>/*.jsonl
• Log structure: JSONL format with timestamps, tool calls, token usage, user messages
• No new instrumentation needed: Can parse existing logs directly

Session Log Structure

Each .jsonl file contains one JSON object per line:

{
  "type": "assistant",
  "timestamp": "2025-12-30T01:26:10.964Z",
  "sessionId": "8c7bdffa-6aa3-49de-8b4d-8f925596faeb",
  "cwd": "/Users/evansenter/Documents/projects/dotfiles",
  "gitBranch": "main",
  "message": {
    "role": "assistant",
    "content": [
      {
        "type": "tool_use",
        "name": "Bash",
        "input": {
          "command": "git status",
          "description": "Check git status"
        }
      }
    ],
    "usage": {
      "input_tokens": 10,
      "output_tokens": 8,
      "cache_read_input_tokens": 46629
    }
  }
}

Proposed Solution

Design Decisions (Resolved)

• Parser language: Bash + jq (simple, no dependencies, fast JSONL parsing)
• Time window: Last 7 days (recent behavior, fast parsing)
• Scope: Both project-specific and global (with toggle/option)
• Performance: Parse on-demand (simple, always fresh data)

High-Level Approach

1. Parser: Read .jsonl logs and extract structured data using bash + jq
2. Analyzer: Identify patterns, frequencies, sequences
3. Suggester: Generate data-driven workflow improvements
4. Integration: Extend /improve-workflow to use metrics

Implementation

Phase 1: Metrics Parser (~/.claude/contrib/parse-session-logs.sh)

Extract from last 7 days of logs:

• Tool calls (name, frequency, timestamp)
• Command executions (/slash-command patterns)
• Common sequences (e.g., Read → Edit → Write)
• Error patterns (failed tool calls)
• Session metadata (duration, token usage)

Phase 2: Pattern Analyzer (same script or separate)

• Identify top N most-used tools/commands
• Detect repeated sequences (same 3+ tools always run together)
• Find friction points (repeated failures, multiple attempts)
• Calculate session statistics (avg duration, tools per session)
• Support both --project and --global flags

Phase 3: Suggestion Engine

• For each pattern, generate actionable suggestions
• Categorize by impact (high-value vs nice-to-have)
• Provide data evidence (e.g., "Used in 80% of sessions")
• Differentiate project-specific vs global patterns

Phase 4: Workflow Integration

• Update /improve-workflow to call parser + analyzer
• Display data-driven suggestions first, static suggestions second
• Show usage stats alongside each suggestion
• Add --metrics-only flag to show just data-driven suggestions

Assumptions

Assumption	Confidence	Impact if Wrong
All tool calls are logged in .jsonl files	High	Would need fallback instrumentation
JSONL format is stable across CC versions	Medium	Would need version-aware parser
Parsing 7 days on-demand is fast enough (<2s)	High	Bash+jq is very fast for JSONL
Logs contain sufficient data for pattern analysis	High	Verified log structure includes tool calls, timestamps
User wants both project and global metrics	High	Confirmed by user

Actionable Requirements

#	Requirement	Owner	Blocked By
1	Build JSONL parser (bash + jq) for tool extraction	Claude	None
2	Add time-window filtering (7 days)	Claude	#1
3	Implement frequency analyzer	Claude	#1
4	Implement sequence detector	Claude	#1
5	Implement friction detector	Claude	#1
6	Add project vs global scope support	Claude	#2
7	Build suggestion engine	Claude	#3, #4, #5
8	Integrate into /improve-workflow	Claude	#7
9	Test with real session data (7+ days)	Human	#8
10	Iterate on suggestions based on usefulness	Claude	#9

Implementation Checklist

• Create ~/.claude/contrib/parse-session-logs.sh
• Implement JSONL parser with jq to extract tool_use events
• Add 7-day time window filtering
• Add project-specific log filtering
• Add global log aggregation (all projects)
• Build frequency analyzer (top tools/commands)
• Build sequence detector (common tool patterns)
• Build friction detector (repeated failures)
• Create suggestion generator from patterns
• Integrate into /improve-workflow command
• Add --project / --global / --both flags
• Test with real usage data
• Document metrics format and usage in CLAUDE.md

Example Workflow Improvement Output

Before (static analysis only):

Suggested improvements:
- Consider adding a /commit-review command

After (with metrics from session logs):

Data-driven suggestions (dotfiles project, last 7 days):
- 🔥 High-value: You run `git status && git diff` in 12/15 sessions → create /git-summary alias?
- 📊 Pattern: You always run Read → Edit → Write on settings.json → create /edit-settings helper?
- ⚠️ Friction: /pr-feedback was run twice consecutively in 8/10 cases → auto-rerun after fixes?
- 📈 Usage: Top commands: /status-report (18x), /pr-feedback (15x), /watch-ci (12x)

Global patterns (all projects, last 7 days):
- 🔥 Bash tool used 450x across 25 sessions - most common: `git status` (85x), `git diff` (72x)
- 📊 Skill invocations: pr-review-toolkit:review-pr (22x), commit-commands:commit-push-pr (18x)

Static suggestions:
- Consider adding a /commit-review command

Advantages of Using Session Logs

1. No new instrumentation: Leverages existing logging infrastructure
2. Historical data: Can analyze past behavior immediately
3. Complete trajectory: Captures all tool calls, not just commands
4. Rich context: Includes timestamps, session IDs, token usage
5. Cross-session analysis: Can aggregate patterns across multiple sessions
6. Fast: Bash + jq can parse thousands of log lines in <1 second

Next Steps

1. Immediate: Create ~/.claude/contrib/parse-session-logs.sh (ready to start)
2. Then: Implement tool extraction with jq
3. Then: Build pattern analysis logic
4. Finally: Integrate into /improve-workflow

[evansenter]

Path Structure Clarification

Updated bootstrap.sh to support the contrib/ directory:

• Scripts will live in home/.claude/contrib/ (in dotfiles repo)
• bootstrap.sh symlinks to ~/.claude/contrib/
• Same pattern as hooks/ and commands/ directories

When implementing:

1. Create home/.claude/contrib/ directory
2. Add parse-session-logs.sh to that directory
3. Run ./bootstrap.sh -f to symlink to ~/.claude/contrib/