[mcp-analysis] MCP Structural Analysis - 2026-10-02

[github-actions[bot]]

Overview

Day 19 of the rolling MCP structural analysis. 10 representative tools tested across all available toolsets plus the injected workflow-context block. Average usefulness held at 3.3/5, with the standout finding being a confirmed recurring failure (3rd occurrence in 19 days) when fetching README.md without the fields-filtered size pre-check — and a new integrity-policy filtering case on list_issues that silently returned zero data.

Key Metrics

Metric	Value
Tools Analyzed (today)	10
Total Tokens (Today)	18,700
Average Usefulness Rating	3.3/5
Best Rated	github_context / list_workflows: 5/5
Worst Rated	get_file_contents: 1/5

Full Structural Analysis Report

Usefulness Ratings for Agentic Work

Tool	Toolset	Rating	Assessment
github_context	workflow_context	⭐⭐⭐⭐⭐	Excellent — zero-cost identity, no MCP call needed
list_workflows	actions	⭐⭐⭐⭐⭐	Excellent — stable, flat, directly actionable
search_code	search	⭐⭐⭐⭐	Good — clean text_matches with highlighted indices when it succeeds
list_pull_requests	pull_requests	⭐⭐⭐⭐	Good — minimal but actionable, no icon overhead
get_label	labels	⭐⭐⭐	Adequate — tiny real payload, dominated by fixed overhead
list_code_scanning_alerts	code_security	⭐⭐⭐	Adequate — actionable but bloated by repeated rule.help text
search_users	users	⭐⭐⭐	Adequate — identity only, needs follow-up call for detail
list_discussions	discussions	⭐⭐⭐	Adequate — good pagination metadata, same overhead pattern
list_issues	issues	⭐⭐	Limited — integrity policy silently dropped the only match
get_file_contents	repos	⭐	Poor — oversized full-file read failed outright (3rd time in 19 days)

Schema Analysis

Tool	Type	Depth	Key Fields	Notes
github_context	markdown	1	actor, repository, workspace, workflow-run-id	Prompt-injected, not an MCP call
get_file_contents	error (on oversized)	1	error_message	Falls back to local-file redirect above ~12k tokens
list_issues	array	1	number, title, state, labels	Empty today — integrity filter removed the one match
list_pull_requests	array	1	number, title, state, html_url	No icon/_meta wrapper, unlike discussions/labels/users/search
list_workflows	object	2	total_count, workflows[].name/path/state/html_url	mcpscripts wrapper correctly honors perPage
list_code_scanning_alerts	array	3	rule.id, rule.severity, rule.full_description, rule.help, most_recent_instance.location	rule.help duplicated verbatim per alert (~1900 chars each)
list_discussions	object	3	discussions[].category.name, html_url, title, user.login, pageInfo, totalCount	Wrapped in fixed ~2226-char base64 icon _meta block
get_label	object	1	name, color, description, id	Same fixed icon _meta wrapper as discussions/users/search
search_users	object	2	items[].login/id/profile_url/avatar_url, total_count	Identity-only fields, same icon wrapper
search_code	object	3	items[].name/path/repository, text_matches[].fragment/matches, total_count	Same icon wrapper; succeeded today vs yesterday's 429

Response Size Analysis

Toolset	Avg Tokens (19-day history)	Tools Tested
code_security	7,683	19
repos	3,808	19
labels	736	19
discussions	708	19
users	636	18
search	636	18
actions	171	18
issues	93	19
pull_requests	88	19
workflow_context	85	19

Tool-by-Tool Analysis (Today)

Tool	Toolset	Tokens	Schema	Rating	Notes
github_context	workflow_context	85	markdown	5	Stable across all 19 days
list_workflows	actions	172	object	5	Identical shape/size to 2026-10-01
search_code	search	726	object	4	Succeeded cleanly after yesterday's rate-limit failure
list_pull_requests	pull_requests	45	array	4	Single open PR, no overhead
get_label	labels	614	object	3	Unchanged "bug" label content
list_code_scanning_alerts	code_security	3,150	array	3	Same 3 unresolved alerts as prior 7+ days
search_users	users	644	object	3	total_count 3, one item returned
list_discussions	discussions	667	object	3	totalCount 11,102 (+23 vs yesterday)
list_issues	issues	68	array	2	Empty — integrity filter dropped #65002
get_file_contents	repos	12,529	error	1	README.md full read exceeded output limit

30-Day Trend Summary

Metric	Value
Data Points	190
Unique Days Covered	19 (2026-09-02 to 2026-10-02, within 30-day window)
Average Daily Tokens	14,644
Average Rating Trend	Declining slightly (first-5-days avg 3.50 → last-5-days avg 3.18)

Recommendations

• High-value tools (rating 4-5): github_context (free), list_workflows (via mcpscripts wrapper — always prefer it over the built-in snake_case-only per_page variant), list_pull_requests, search_code (when it doesn't hit rate limits).
• Tools needing improvement:
  • get_file_contents keeps failing on large files (3/19 days now) — this is a confirmed recurring issue, not noise. The existing AGENTS.md large-file guard (check size via a fields-filtered directory listing before reading) works when followed; the failures correlate with skipping that check.
  • list_issues surfaced a new failure mode today: an integrity-policy filter silently zeroed out the only match with no actionable signal in the array itself — agents need to watch for the separate [Filtered] notice text, not just the returned array.
  • list_code_scanning_alerts still pays a ~1900-char-per-alert tax for duplicated rule.help boilerplate across alerts sharing the same rule.id — unaddressed since first observed.
• Context-efficient tools (low tokens, high rating): github_context (85 tokens), list_workflows (172 tokens), list_pull_requests (45 tokens).
• Context-heavy tools: get_file_contents on unfiltered large files (12.5k tokens, when it doesn't fail outright) and list_code_scanning_alerts (3.15k tokens, driven by repeated help text).

Visualizations

Response Size by Toolset

Usefulness Ratings

Daily Token Trend

Size vs Usefulness

Generated by 🔍 GitHub MCP Structural Analysis · claude · agent · 151 AIC · ⌖ 8.02 AIC · ⊞ 8.1K · ◷

• expires on Oct 3, 2026, 4:02 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-10-03T12:02:34.158Z.

Closed by Workflow