[mcp-analysis] MCP Structural Analysis - 2026-09-08

[github-actions[bot]]

Tested one representative tool from each of 10 GitHub MCP toolsets (plus the injected workflow-context block) against github/gh-aw. Best-rated: github_context and list_workflows (5/5 each) — clean, minimal, actionable. Worst-rated: search_code (1/5), which hit a GitHub secondary rate limit (429) and returned a hard error instead of data — the first outright failure seen across 10 days of measurements. Average usefulness today: 3.4/5, roughly flat with the 10-day trend average of ~3.6/5.

Note: Chart generation was not possible this run — see the missing_tool report filed alongside this discussion (PyPy has no pandas/matplotlib wheels; the only CPython on the host fails to start due to a glibc mismatch). Data tables below cover the same information the charts would have shown.

Full Structural Analysis Report

Executive Summary

Metric	Value
Tools Analyzed	10
Total Tokens (Today)	6,211
Average Usefulness Rating	3.4/5
Best Rated Tool	github_context & list_workflows: 5/5
Worst Rated Tool	search_code: 1/5 (429 rate-limit error)

Usefulness Ratings for Agentic Work

Tool	Toolset	Rating	Assessment
github_context	workflow_context	⭐⭐⭐⭐⭐	Excellent — zero-cost, immediately actionable identity block
list_workflows (mcpscripts wrapper)	actions	⭐⭐⭐⭐⭐	Excellent — clean pagination metadata, no icon/_meta bloat
get_file_contents	repos	⭐⭐⭐⭐	Good — directory listing gives a clean file map
list_pull_requests	pull_requests	⭐⭐⭐⭐	Good — flat, minimal, tiny
list_code_scanning_alerts	code_security	⭐⭐⭐⭐	Good — trivially small and unambiguous when empty
list_discussions	discussions	⭐⭐⭐	Adequate — good metadata, but icon overhead dominates payload
list_label (mcpscripts wrapper)	labels	⭐⭐⭐	Adequate — returns full 100-of-935 label set, no perPage control
search_users	users	⭐⭐⭐	Adequate — minimal identity fields only
list_issues	issues	⭐⭐	Limited — redacted item blocks assessment (4th+ consecutive day)
search_code	search	⭐	Poor — 429 rate limit, hard error, zero usable data

Schema Analysis

Tool	Type	Depth	Key Fields	Notes
github_context	markdown	1	actor, repository, workspace, workflow-run-id	Prompt-injected, no MCP call
get_file_contents	array	1	name, path, size, type	Directory-listing mode via fields filter
list_issues	array	2	number, title, state, labels	One item redacted by integrity policy
list_pull_requests	array	1	number, title, state, html_url	Flat, field-filtered
list_workflows	object	2	total_count, per_page, workflows[]	mcpscripts wrapper, clean JSON
list_code_scanning_alerts	array	1	(empty)	No open critical alerts today
list_discussions	object	3	discussions[].category/user, pageInfo, totalCount	Nested relationships, icon _meta overhead
list_label	array	1	labels[].name/color/description/id	Full-set dump, no pagination control
search_code	object	1	error message only	429 rate-limited
search_users	object	2	items[].login/id/profile_url/avatar_url	Minimal identity fields

Response Size Analysis (Today)

Toolset	Tokens	Tool Tested
workflow_context	62	github_context
actions	172	list_workflows
repos	1,052	get_file_contents
issues	64	list_issues
pull_requests	46	list_pull_requests
code_security	10	list_code_scanning_alerts
discussions	687	list_discussions
labels	2,902	list_label
search	610	search_code (error)
users	606	search_users

Tool-by-Tool Analysis

Tool	Toolset	Tokens	Schema	Rating	Notes
github_context	workflow_context	62	markdown, depth 1	5	Zero-cost identity block
list_workflows	actions	172	object, depth 2	5	Best-in-class wrapper schema, 5th consecutive day
get_file_contents	repos	1,052	array, depth 1	4	Directory map, 60 entries, no content bloat
list_pull_requests	pull_requests	46	array, depth 1	4	Tiny, flat, actionable
list_code_scanning_alerts	code_security	10	array, depth 1	4	Empty today; high variance by repo state historically
list_discussions	discussions	687	object, depth 3	3	~556-token icon _meta overhead vs. ~131 tokens of content
list_label	labels	2,902	array, depth 1	3	Most expensive tool today; full 100/935 labels, no perPage
search_users	users	606	object, depth 2	3	Identity-only, needs follow-up call for profile detail
list_issues	issues	64	array, depth 2	2	Redacted item blocks assessment (persistent pattern)
search_code	search	610	object, depth 1	1	429 secondary rate limit — hard failure, no data

30-Day Trend Summary

Metric	Value
Data Points	100 (across 10 measured days, 2026-08-25 → 2026-09-08)
Average Daily Tokens	~7,066
Average Rating Trend	Stable (~3.3–3.7/5 range every measured day, no clear drift)

Daily Token Totals (30-Day Window)

Date	Total Tokens	Avg Rating
2026-08-25	8,235	3.6
2026-08-26	8,120	3.7
2026-08-27	3,135	3.6
2026-08-28	4,262	3.6
2026-08-31	4,385	3.6
2026-09-02	5,845	3.3
2026-09-03	9,490	3.6
2026-09-04	6,142	3.6
2026-09-07	14,832	3.6
2026-09-08	6,211	3.4

All-Time Average Tokens by Toolset

Toolset	Avg Tokens	Tools Tested
repos	2,315	10
code_security	1,487	10
labels	991	10
discussions	684	10
search	615	10
users	610	10
workflow_context	94	10
actions	170	10
issues	61	10
pull_requests	39	10

Recommendations

• High-value tools (rating 4-5): github_context (free), list_workflows via the mcpscripts wrapper (clean pagination, no bloat), get_file_contents in directory-listing mode, list_pull_requests, list_code_scanning_alerts when the repo is clean.
• Tools needing improvement: list_issues — the integrity-policy redaction has now blocked visibility into the same flagged issue for 5+ consecutive measured days with zero explanation surfaced to the agent; list_label — no perPage control means every call pays for the full 935-label set; search_code — no visible backoff/retry guidance when secondary rate limits hit, so agents can't self-correct.
• Context-efficient tools (low tokens, high rating): github_context (62 tokens), list_pull_requests (46 tokens), list_workflows (172 tokens) — all field-filtered or wrapper-based.
• Context-heavy tools: list_label (2,902 tokens, dominated by returning the full label set), get_file_contents full-repo directory listing (1,052 tokens), list_discussions and search_users (both ~600-700 tokens dominated by a fixed ~556-token icon _meta block present on every non-wrapper GitHub MCP response).
• Recurring inefficiency: the fixed icon/serverInfo _meta block adds ~556 tokens of pure overhead to every direct (non-mcpscripts-wrapped) GitHub MCP tool call regardless of payload size — for small lookups like get_label or search_users this overhead has historically exceeded the actual content.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 🔍 GitHub MCP Structural Analysis · claude · agent · 262.9 AIC · ⌖ 10.6 AIC · ⊞ 11.5K · ◷

• expires on Sep 9, 2026, 4:12 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by GitHub MCP Structural Analysis.

A newer discussion is available at Discussion #59719.