Should tools be a means for capability discovery?

[bokand]

IIUC the primary motivation for the declarative form of tool registration is to enable discovery (e.g. by indexing in a search engine). Filing this issue for a broader discussion as to whether tools should be the means for discovery for an agent. We also initially went down this road but I think I convinced myself that tools shouldn't be the primitive for capability discovery but I'm curious to hear other perspectives. My reasoning:

1. Tool != capability. Tools are meant as an alternative to human UI so the capability should exist on the site even without the tool. e.g. For very simple apps/sites, if agents work perfectly well using the human UI, do we want to force authors to provide tools just to enable discovery?

2. Capabilities are a higher level abstraction than tools. An image editor might have tools like "Make Selection", "Select Brush", "Undo", "Redo", etc. These are all low level actions that only make sense in the broader context of the app. The app's capability is "Image Editor" which would be better captured by some other mechanism.

3. Tools will become SEO'd. Authors will have an incentive to add spurious tools to their pages in order to surface themselves to agents, regardless of whether the add value.

4. Authors will now have a second axis along which to optimize tool descriptions for: agent usage and search engines. Given the difficult challenge of the former this seems like it will compromise agent quality

Thoughts?

[mikechao]

1. For very simple apps/sites, if agents work perfectly well using the human UI, do we want to force authors to provide tools just to enable discovery?

I agree that for simple apps/sites current agents can work perfectly well using the human UI. But exposing tools to agents provide an additional benefits including faster speed of execution and cheaper costs.

Currently most(all?) web browsing agents use the tactic of take a screenshot, feed it to the LLM to figure out where the buttons are and click the button or use something like playwright. With the tools exposed to the agent, the LLM already know what actions it can take when the user asks it a question.

So for the MCP-B Chrome Extension I created this demo webpage on the page we expose the same UI controls and MCP Tools.

Here is the demo page and the extension in action. We prompt the agent/Chrome extension with the following

Can you do all of the following tasks?
1. add sleep to my todo
2. set the current project to mcp-b.ai
3. update the mood this is bananas
4. record my thought of "this is cool"
5. Increment the count by 42

mcp-extension-demo.mp4

The tasks are completed in about 10 seconds. Some internal testing I have done with BrowserBase with the same task has taken 2 plus minutes. Maybe this isn't a fair test. I am open to testing with other tools/agents to discover potential benefits of exposing tools.

[khushalsagar]

Thanks for the feedback @mikechao. This issue isn't questioning the value of Agents using WebMCP based tools over UI actuation. It's specifically about whether the API should be structured to allow Agents to use these tools as way to understand what the site is capable of.

When the user makes a query to the Agent and the currently loaded pages in their browser (if any) are not suitable for their task, the Agent has to decide which site to use instead. In the absence of WebMCP, that would be driven by a search engine which indexes the site's content meant for human consumption. But the WebMCP tools could become another source of understanding the site's capabilities. @bokand's comment highlights the caveats with that idea.

Whether this is a goal influences the API design. If we want to ensure it's a good discovery mechanism then we need to strive for a declarative API. Or at least ensure that anything imperative we build first can be extended to a declarative shape.

[khushalsagar]

Summarizing the offline feedback I’ve received from multiple folks on this topic. TLDR: WebMCP tools shouldn’t be the only source of discovery but it will be one of the inputs. We should strive for a more declarative shape to make that easier.

It’s true that at the beginning tools will be a subset of the site’s capabilities expressed in the Web UI targeted for a human user. Tools will be added incrementally and browsers will have to manage using both the Web UI and WebMCP tools. So the additional information from them won’t be as useful.

But that won’t remain true long term, sites will add capabilities that a human can’t use directly but can via an Agent. The discovery mechanism for Agents will need a way to crawl and index these capabilities. The explainer makes the same point here, coming from the Microsoft folks IIRC.

A declarative API shape will make that easier so we should strive for that. I’m not sure how yet for tools defined in script. The explainer has an alternative API shape where the tools are defined in a json manifest here but the con “Slightly harder to keep definition and implementation in sync” resonates with me. Especially if there’s different components on the page which can update the set of tools as they are added/removed from the UI with registerTool/unregisterTool API.

I’d be happy to take a resolution to conclude that we want these tools to be a part of discovery. And revisit the API shape towards this goal separately.

[anssiko]

RESOLUTION: The group wants to make the tools be part of the discovery mechanism and continues to explore and prototype API shapes that satisfy this requirement. This includes the declarative API proposal that complements the imperative API, as well as the JSON manifest, with pros/cons documented.