[mcp-analysis] MCP Structural Analysis - 2026-10-07

[github-actions[bot]]

MCP Structural Analysis — 2026-10-07

10 representative tools tested across 9 toolsets + the injected workflow-context block. Average usefulness today: 2.6/5 (down from 3.2 yesterday) — driven by a get_file_contents size blowout and two integrity-policy redactions that hid data an agent would otherwise expect.

Best rated: github_context / list_pull_requests (5/5) — flat, immediately actionable, zero follow-up calls needed.
Worst rated: get_file_contents, list_issues, search_code (1/5 each) — an oversized file read, a policy-redacted issue list, and a GitHub secondary-rate-limit hit respectively.

Key recommendations

• Never call get_file_contents on a full file without a size check first — README.md (52,804 chars / ~13.2k tokens) blew past the harness's tool-output limit for the third day running (12.5k → 1.5k → 13.2k tokens, repo-size dependent, not something perPage fixes). This validates the AGENTS.md large-file guard: check size via directory listing first, read with ranged/grep tools for anything over ~20KB.
• list_code_scanning_alerts has no recovery path once a repo has many alerts — Oct 6 hit the 25k-token ceiling with required filters already applied and no perPage/head_limit support; Oct 7 instead had 30 matching items silently dropped by integrity policy. Either way the agent cannot reliably learn "how many high/critical alerts exist" from this tool alone.
• Integrity-policy redaction is indistinguishable from "no data" — both list_issues and list_code_scanning_alerts returned clean empty results today while a side-channel note disclosed that real items were filtered. An agent reading only the primary response would wrongly conclude the repo has zero open issues/alerts.
• Icon-metadata overhead is a persistent, repo-independent tax — every single-item/search call through the main GitHub MCP server (list_discussions, get_label, search_users, search_code) repeats ~2.1KB of base64 server-icon data in _meta on every response, routinely 2-5x larger than the actual payload. The mcpscripts wrapper (list_workflows) avoids this entirely via plain CLI passthrough — prefer wrapper tools where available.
• Rate limiting is now a recurring search_code failure mode — hit a 429 today (13s retry) same as a prior run; treat search_code as unreliable for time-sensitive agentic steps without backoff handling.

Full Structural Analysis Report

Executive Summary

Metric	Value
Tools Analyzed	10
Total Tokens (Today, est.)	16,285
Average Usefulness Rating	2.6/5
Best Rated Tool	github_context / list_pull_requests: 5/5
Worst Rated Tool	get_file_contents, list_issues, search_code: 1/5

Usefulness Ratings for Agentic Work

Tool	Toolset	Rating	Assessment
github_context	workflow_context	⭐⭐⭐⭐⭐	Zero-cost identity metadata, no MCP call needed
list_pull_requests	pull_requests	⭐⭐⭐⭐⭐	Flat, field-filtered, immediately actionable
list_workflows (mcpscripts)	actions	⭐⭐⭐⭐	Clean pagination envelope, no icon bloat
list_discussions	discussions	⭐⭐⭐	Usable but totalCount ignores pagination scope; icon bloat
list_code_scanning_alerts	code_security	⭐⭐	Can't distinguish "no alerts" from "alerts hidden by policy"
get_label	labels	⭐⭐	Tiny real payload buried under repeated icon metadata
search_users	users	⭐⭐	Thin fields, needs follow-up get_user call
get_file_contents	repos	⭐	Oversized file broke the tool-output limit entirely
list_issues	issues	⭐	Sole matching issue redacted by integrity policy
search_code	search	⭐	Hit GitHub secondary rate limit, zero data returned

Schema Analysis

Tool	Type	Depth	Key Fields	Notes
github_context	markdown	1	actor, repository, workspace, workflow-run-id	Prompt-injected, no call overhead
list_pull_requests	array	2	number, title, state, html_url	Field-filtered, clean
list_workflows	object	3	total_count, per_page, page, workflows	CLI passthrough, no envelope bloat
list_discussions	object	3	discussions, pageInfo, totalCount, category, user	_meta.icons dominate byte count
get_label	object	1	name, color, description, id	Payload ~110 chars vs ~2KB of icon metadata
search_users	object	2	total_count, incomplete_results, items	login/id/profile_url/avatar_url only
list_code_scanning_alerts	array	1	(none returned)	Empty array masks policy-redacted items
get_file_contents	primitive	1	content	Single unstructured line, no range/partial-read param
list_issues	array	1	(none returned)	Empty array masks policy-redacted item
search_code	object	1	error	429 error body, no usable schema

Response Size Analysis

Toolset	Avg Tokens (est., 3-day)	Tools Tested
code_security	11,244	3 (5 → 33,668 → 60: wildly variable, driven by alert count + redaction, not controllable via filters)
repos	9,076	3 (12,528 / 1,500 / 13,201: file-size dependent, no safe default)
users	1,000	3
discussions	836	3
labels	754	3
search	747	3
actions	172	3
workflow_context	92	3
pull_requests	67	3
issues	27	3

Tool-by-Tool Analysis (today)

Tool	Toolset	Tokens	Schema	Rating	Notes
get_file_contents	repos	13,201	primitive	⭐	README.md (52,804 chars) broke the tool-output limit; forced to a saved-file fallback
list_discussions	discussions	747	object	⭐⭐⭐	perPage honored for items; totalCount still repo-wide (11,230)
search_code	search	653	object	⭐	429 secondary rate limit, 13s retry-after
search_users	users	649	object	⭐⭐	Thin item fields, needs follow-up calls
get_label	labels	626	object	⭐⭐	Tiny real payload, icon-metadata bloat
list_workflows	actions	172	object	⭐⭐⭐⭐	mcpscripts wrapper, clean envelope
github_context	workflow_context	75	markdown	⭐⭐⭐⭐⭐	Zero-cost identity metadata
list_code_scanning_alerts	code_security	60	array	⭐⭐	30 matching items redacted by integrity policy
list_issues	issues	60	array	⭐	Sole matching issue redacted by integrity policy
list_pull_requests	pull_requests	42	array	⭐⭐⭐⭐⭐	Clean, field-filtered, no bloat

30-Day Trend Summary

Metric	Value
Data Points	30 (3 days: 2026-10-05 to 2026-10-07)
Average Daily Tokens (est.)	24,015
Average Rating Trend	Declining (2.9 → 3.2 → 2.6) — driven by today's redaction + rate-limit hits, not a structural regression

Recommendations

• High-value tools (rating 4-5): github_context (free), list_pull_requests, list_workflows (via mcpscripts wrapper) — prefer these as first-choice reads.
• Tools needing improvement: get_file_contents (needs a range/partial-read parameter), list_code_scanning_alerts (needs perPage/head_limit support to avoid both the 25k-token ceiling and the "redacted-looks-like-empty" ambiguity), list_issues (same redaction-ambiguity issue).
• Context-efficient tools (low tokens, high rating): github_context, list_pull_requests, list_workflows.
• Context-heavy tools: get_file_contents and list_code_scanning_alerts — both file-size/alert-count dependent with no reliable way to bound the response via input parameters alone; always check size/use grep-style tools first per AGENTS.md's large-file guard.

Visualizations

Response Size by Toolset

Usefulness Ratings

Daily Token Trend

Size vs Usefulness

References: §37617456961

Generated by 🔍 GitHub MCP Structural Analysis · claude · agent · 173.5 AIC · ⌖ 7.64 AIC · ⊞ 7.9K · ◷

• expires on Oct 8, 2026, 4:04 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by GitHub MCP Structural Analysis.

A newer discussion is available at Discussion #66871.