[mcp-analysis] MCP Structural Analysis - 2026-07-31

[github-actions[bot]]

Overview

Tested 11 tools across 10 toolsets against github/gh-aw (plus the prompt-injected <github-context> block). Total tokens today: 8,168. Average usefulness rating: 3.36/5. Best: github_context and list_discussions/list_pull_requests (⭐⭐⭐⭐⭐). Worst: get_teams (⭐, permission denied — consistent with every prior run).

New finding this run: search_repositories with an explicit repo: qualifier returned an unrelated top hit (github/gitignore, ranked by star count) instead of the targeted repo — the qualifier doesn't appear to scope results reliably.

Metric	Value
Tools Analyzed	11
Total Tokens (Today)	8,168
Average Usefulness Rating	3.36/5
Best Rated Tool	github_context / list_discussions / list_pull_requests: 5/5
Worst Rated Tool	get_teams: 1/5

Full Structural Analysis Report

Usefulness Ratings for Agentic Work

Tool	Toolset	Rating	Assessment
github_context	workflow_context	⭐⭐⭐⭐⭐	Zero-cost prompt-injected identity, always available
list_discussions	discussions	⭐⭐⭐⭐⭐	Compact, cursor pagination works, embeds category+author
list_pull_requests	pull_requests	⭐⭐⭐⭐⭐	fields[] projection cuts payload to ~50 tok, fully actionable
get_label	labels	⭐⭐⭐⭐	Flat 4-field object, zero bloat
get_file_contents	repos	⭐⭐⭐⭐	Actionable content+SHA, but no partial/range fetch — scales with file size
list_code_scanning_alerts	code_security	⭐⭐⭐⭐	Clean empty-array signal, near-zero cost when no alerts
search_users	users	⭐⭐⭐	Adequate identity resolution, minimal fields only
list_workflows	actions	⭐⭐	per_page/perPage silently ignored — returns ~30 of 342 workflows regardless
list_issues	issues	⭐⭐	Integrity-filtered results present as an empty array with no in-band marker
search_repositories	search	⭐⭐	repo: qualifier not honored — returned an unrelated popular repo
get_teams	context	⭐	Permission not granted under integration token (recurring across all runs)

Schema Analysis

Tool	Type	Depth	Key Fields	Notes
github_context	markdown	1	actor, repository, workspace, workflow-run-id	Prompt text, not an MCP call
get_file_contents	object	3	content[].text(SHA), resource.uri, resource.text	Clean resource envelope
list_issues	array	1	(none surfaced)	Empty due to integrity filtering
list_pull_requests	array	1	number, title, state, html_url	Field-projected, flat
list_workflows	object	2	total_count, workflows[].id/name/path/state/html_url	Triple *_url per item, unbounded page
list_code_scanning_alerts	array	0	—	Empty array
list_discussions	object	3	discussions[].category.name, .user.login, pageInfo, totalCount	GraphQL cursor pagination
get_label	object	1	name, color, description, id	Flat
search_users	object	2	total_count, items[].login/id/profile_url	Minimal identity fields
search_repositories	array	1	full_name, stargazers_count, topics	Ranking/scoping unreliable for repo: qualifier

Response Size Analysis (30-day average by toolset)

Toolset	Avg Tokens
repos	4,855
actions	2,257
code_security	1,572
pull_requests	1,530
issues	1,161
search	789
labels	214
discussions	183
users	135
workflow_context	76
context	36

Tool-by-Tool Analysis (2026-07-31)

Tool	Toolset	Tokens	Schema	Rating	Notes
github_context	workflow_context	75	markdown	5	Zero MCP overhead
get_teams	context	35	error	1	Permission not granted
get_file_contents	repos	3,600	object	4	README full-file pull, size scales with file
list_issues	issues	65	array	2	Integrity-filtered, opaque to agent
list_pull_requests	pull_requests	50	array	5	fields[] projection near-optimal
list_workflows	actions	3,900	object	2	Pagination param ignored
list_code_scanning_alerts	code_security	10	array	4	Clean empty signal
list_discussions	discussions	165	object	5	Cursor pagination, compact
get_label	labels	38	object	4	Flat lookup primitive
search_users	users	90	object	3	Minimal identity fields
search_repositories	search	140	array	2	repo: qualifier not honored

30-Day Trend Summary

Metric	Value
Data Points	183
Dates Covered	17 (2026-07-01 through 2026-07-31)
Average Daily Tokens	~13,669
Average Rating Trend	Stable (early-window avg 3.55 vs late-window avg 3.50)

Recommendations

• High-value tools (rating 4-5): github_context, list_discussions, list_pull_requests (with fields[]), get_label, list_code_scanning_alerts (empty case), get_file_contents.
• Tools needing improvement: list_workflows (pagination param per_page is silently ignored — a recurring finding across many runs), list_issues/search_issues (integrity-filtered results are opaque — no in-band marker distinguishes "genuinely empty" from "filtered"), search_repositories (the repo: qualifier does not reliably scope to the named repository), get_teams (unusable under the read-only integration token in this environment).
• Context-efficient tools (low tokens, high rating): github_context (75 tok), list_discussions (165 tok), list_pull_requests with field projection (50 tok), get_label (38 tok).
• Context-heavy tools (high tokens): get_file_contents on large files (README ~3.6k-12k+ tok depending on growth), list_workflows (~3.9k tok, unbounded).

Visualizations

Response Size by Toolset

Usefulness Ratings

Daily Token Trend

Size vs Usefulness

Generated by 🔍 GitHub MCP Structural Analysis · agent · 134.6 AIC · ⌖ 29 AIC · ⊞ 11.6K · ◷

• expires on Aug 1, 2026, 4:46 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-01T12:46:36.400Z.

Closed by Workflow