Context bloat - the bottle neck of MCP

[NeelakandanNC]

Problem: Tool context bloat

When MCP clients connect to several servers — or even one large one (Postman, GitHub, etc.) — the full tools/list payload is injected into every model turn. Three things degrade:

1. Latency — larger prompts slow first tokens
2. Cost — every turn pays for schemas the model mostly doesn't use
3. Decision quality — models pick worse tools as the list grows and options blur together

The spec currently offers no primitive for grouping or partial loading: tools/list returns a flat list, and every workaround (HyperTool, RAG-MCP, GitHub's toolsets, Continue's tool filtering) lives above the protocol layer. That means each client reinvents the wheel, servers can't express their natural functional boundaries, and there's no interop between approaches.

Proposal: SmartMCP

Two complementary pieces — one a spec change, one a client-side pattern the spec change enables.

1. Subseg — Subject Segmentation (spec-level)

Add a lightweight grouping primitive to tools/list so servers can declare functional groups and clients can load only what they need for a given turn.

Today:

tools/list → [tool1, tool2, tool3, tool4, tool5, tool6]

Proposed:

tools/list →
  group "creation":  [tool1, tool2, tool3]
  group "deletion":  [tool4, tool5]
  group "query":     [tool6]

Concretely, this could be an optional group field on each tool plus an optional groups filter parameter on tools/list — fully backward-compatible.

2. DMA — Dual Model Architecture (client pattern)

A pattern Subseg makes practical. The core distinction, which is easy to mischaracterize:

• Model 1 (expensive, reasoning) sees no tool schemas at all and outputs a natural-language description of the action to perform.
• Model 2 (cheap) sees the schemas for the relevant Subseg group and maps Model 1's description to a concrete tools/call.

Model 1 is action-descriptive, not tool-selective. It never touches schemas. Consequences:

• The expensive model's context stays small no matter how many tools exist downstream
• Schema changes don't perturb Model 1's prompt
• The schema-matching task is mechanical enough that a cheap model handles it well

Tradeoffs (honest)

	Time	Cost	Quality
Stock MCP	T	X	baseline
SmartMCP	T + Δ (extra hop)	< X	same or better

Latency takes a small hit from the second model call. Subseg alone improves all three axes without that tradeoff, which is why the spec primitive is valuable even for clients that don't adopt DMA.

Why spec-level rather than middleware

Every existing grouping solution sits above the protocol. That fragments the ecosystem: a Postman-style group definition is unusable by another client, servers can't advertise their own groupings, and clients keep reinventing the same primitive in slightly incompatible ways. A small, optional, backward-compatible spec change unlocks all of this once.

Open questions

• Naming: group, category, namespace?
• Flat vs. hierarchical groups?
• Should tools/list without a filter return groups-only metadata, or the full flat list for backward compat?
• Does this belong as an SEP, or start as a discussion thread first?

Happy to draft an SEP if there's appetite.

[SolvoFounder]

The tool schema problem is real and compounds fast once you're connecting to several servers. A lot depends on whether the bloat is coming from having many tools across servers versus individual schemas that are just oversized — those are two different levers.

MCP Logger instruments your servers to show exactly what's being included in each context payload and when, so you can see whether the weight is coming from tool count, schema size, or something else hitting your context window. Could help identify which server or which tool is the biggest offender before you start trimming.

What servers are you running against? And are you seeing the bloat from many tools total, or from a smaller set with very large schemas?

— Solvo

[NeelakandanNC]

Good distinction — I've been treating "context bloat" as one thing but you're right that they're different levers.
For my case it's mostly the first: many tools across servers (Postman + a couple others on Gemini CLI was where I first hit it). But the schema-size point is interesting because it changes how you'd think about SmartMCP's two parts:

Subseg mainly attacks tool-count bloat — segmenting by subject (creation/deletion/etc.) so only the relevant group loads.
DMA ends up helping with schema-size bloat almost as a side effect: Model 1 only describes the action in natural language and never sees tool schemas, so the heavy reasoning context stays clean. Only Model 2 (the cheaper one) eats the schema cost.

For servers, mainly Postman, plus a few smaller ones I was experimenting with. I haven't profiled where the bytes are actually going through — will probably need to once I start measuring SmartMCP vs vanilla. I'll take a look at MCP Logger when I get there.