Add Serena MCP tool usage statistical analysis with request/response size metrics

[Copilot]

Statistical analysis of Serena MCP server tool usage from workflow run 21560089409.

Deliverables

• scratchpad/serena-tools-analysis.md - Complete analysis with Mermaid visualizations
• scratchpad/serena-tools-quick-reference.md - At-a-glance metrics and insights
• scratchpad/serena-tools-data.json - Raw statistics for programmatic access

Key Metrics

Metric	Value
Total tool calls	44 (100% response rate)
Serena tool calls	9 (20.45%)
Builtin tool calls	34 (77.27%)
Tools registered	23
Tools used	6 (26.09% adoption)
Most used Serena tool	search_for_pattern (3 calls)

Request/Response Size Metrics

Metric	Value
Total data transferred	425.69 KB
Request data	72.60 KB
Response data	353.09 KB
Response/request ratio	4.86x
Serena data transfer	12.32 KB (2.89% of total)
Builtin data transfer	246.53 KB (57.91% of total)
Bash data transfer	181.17 KB (42.56% of total)

Serena Efficiency: Serena tools use ~700-840 bytes per request and 386-771 bytes per response (<1x amplification), compared to Bash at 10KB average response (11.8x amplification). Serena tools transfer 10x less data per call than Bash operations.

Findings

Tool adoption: 17/23 Serena tools unused (74%), including all code modification tools, most memory management tools, and meta-cognitive capabilities.

Usage pattern: Agent heavily favors builtin operations (Bash, Read, Write) over language-aware Serena tools despite their availability.

Tool categories by adoption:

• Configuration: 66% (2/3)
• Symbol Analysis: 50% (2/4)
• Memory Management: 20% (1/5)
• Code Modification: 0% (0/4)

Reliability: 100% response rate across all tool calls - no timeouts or failures.

Bandwidth efficiency: Serena tools are highly bandwidth-efficient, accounting for only 2.89% of total data transfer despite representing 20.45% of tool calls. They return structured, compact data compared to verbose text outputs from Bash.

Optimization Opportunities

1. Lazy-load or subset tool registration (74% unused overhead)
2. Update agent prompts to encourage language-aware tool usage
3. Enable memory persistence tools for cross-run learning
4. Create workflow-specific toolsets to reduce cognitive load
5. Leverage Serena's bandwidth efficiency for network-constrained environments

Custom agent used: agentic-workflows
GitHub Agentic Workflows (gh-aw) - Create, debug, and upgrade AI-powered workflows with intelligent prompt routing

Original prompt

Reference: https://github.com/githubnext/gh-aw/actions/runs/21560089409/job/62122702303#step:33:1

Do a deep statistical analysis of the Serena tools usage (request vs responses ...). Generate a scratchpad report.

Custom agent used: agentic-workflows
GitHub Agentic Workflows (gh-aw) - Create, debug, and upgrade AI-powered workflows with intelligent prompt routing

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[pelikhan]

@copilot analyze tools request, response sizes as well.

[Copilot]

@copilot analyze tools request, response sizes as well.

Added comprehensive request/response size analysis in commit e587a48.

Key findings:

• Total data: 425.69 KB (72.60 KB requests + 353.09 KB responses)
• Serena tools: 12.32 KB (2.89% of total) - highly bandwidth-efficient
• Bash dominates: 181.17 KB (42.56% of data) despite being 38.64% of calls
• Efficiency: Serena tools use ~700-840B per request, 386-771B per response (<1x amplification) vs. Bash at 10KB average response (11.8x amplification)

Full analysis includes:

• Size breakdown by tool and category
• Request/response ratio analysis
• 2 new Mermaid visualizations for data transfer volumes
• Per-tool min/max/avg metrics in JSON data file

All three report files updated with size metrics.