Clarifying the scope of the proposal

[43081j]

Ultimately, an end-to-end WebMCP setup consists of these:

• Browser implementation (i.e. the window.agent API)
• Client (e.g. an MCP server capable of using CDP to call the browser API)

Right now, this repo seems to be both of these tangled up into one. Most likely that is a result of all prior art existing this way (e.g. "MCP-B" implements both).

This doesn't feel like how a web standard should work to me though. The client could be any of the following, for example:

• An MCP server which uses CDP to call the browser API
• A regular LLM tool which uses CDP to call the browser API
• A Chrome extension which interacts directly with the browser API
• A library which uses puppeteer to interact with the browser API
• etc etc

If we accept that the client isn't our concern (community and companies will build these clients inevitably), that means the only thing this proposal needs to solve is the browser API.

If that was the case, this would be an extremely small proposal I think. Since we just need to bikeshed the name and the shape of the tools/results.

If I'm missing something here, please do let me know.

[anssiko]

Thanks for your feedback. This is a web spec incubation and not all design considerations are written down yet, so those are reasonable questions to ask.

Currently, we have an agreement on the group's Scope of Work and are working on an explainer and proposal docs. The explainer also has a section on MCP-B. See also open issues and PRs for active design discussions.

Our OSS project contributors @MiguelsPizza and @jasonjmcghee have explored this space ahead of browser implementations with a broader scope and that's been hugely valuable. The name "WebMCP" may be overloaded and some of the confusion may arise from that.

I hope this helps. @bwalderman @khushalsagar are working on the said docs and will consider this feedback.

@43081j, if you're interested in making further contributions, please join this community.

[43081j]

i have been keeping up with the various issues and read through the explainer/proposal, but it did seem like a lot of discussion happens around the client part of this (the thing which interacts with the browser API).

much of this reads more complicated than it needs to be i think. defining the browser API would be enough that people can start building interfaces to that (MCP, tools, extensions), and web tools for it.

i have joined the community 👍

[anssiko]

Welcome on board! We’re all set to work together on this.

[khushalsagar]

If we accept that the client isn't our concern (community and companies will build these clients inevitably), that means the only thing this proposal needs to solve is the browser API.

@43081j Thanks for outlining the multiple ways clients could access the WebMCP functionality. It's quite difficult to design the browser API without at least thinking through how all these clients would consume that functionality. As an example, #16 discussed additions which would be targeted for extensions.

So while this group isn't designing the client side of the API, those details come up as we discuss feature additions to the Web API. During the discussion for #21, we concluded to continue the API design focusing on the in-browser Agent integration (which is an internal detail for each browser).

[43081j]

makes sense 👍

the larger part of an e2e implementation is usually the client. the browser API is actually pretty small in the grand scheme of things, so I just wanted to be sure this proposal is solely about that rather than defining how clients should work.

totally makes sense to consider and discuss them at least, to help shape that API.

in our case, we have an MCP server, an extension, and a Prompt API implementation. All of them are just implementation detail, though, and the real core of it is the browser API (which just needs list/get/define/call).

[MiguelsPizza]

@43081j Is there any reason you did not list a client which lives in the same document as the WebMCP tools? To me this would be a more common use-case, as it is effectively an SDK for a website's agent to interact with the functionality on the page.

The most common request I've received from users of MCP-B is asking how they can use the WebMCP tools with their website's existing agent/chat-feature.

It seems like designing the client alongside the server would be a requirement for a v1 of the API. Otherwise we are just building tools for built-in browser agents to consume directly.

[43081j]

it wasn't an exhaustive list is all

all of these things are just applications or libraries which call the browser API we're trying to standardise in this proposal

i don't think we should be trying to standardise clients because its an infinite list and not part of the browser. whether its a chrome extension, a runtime library or an MCP server - its just a piece of code that calls the browser API

we shouldn't be trying to propose how people build MCP servers, extensions, etc. it doesn't really matter how those get built. if the browser API exists, all of those and more we haven't thought of can call it.

that certainly doesn't limit any of this to "built-in browser agents".

if this proposal solely defined the browser API, communities and companies can start building "bridges" to communicate with it (clients - like extensions, MCP servers, etc.).

[pengw272]

@43081j

Ultimately, an end-to-end WebMCP setup consists of these:

Browser implementation (i.e. the window.agent API)
Client (e.g. an MCP server capable of using CDP to call the browser API)

Could you clarify the "Browser implementation" part? How would browser implement something like add-to-cart, which is different on different sites? Or are you saying that window.agent exposes primitives (that already exist in CDP) like click, scroll. Then the custom add-to-cart can utilize these primitives?

[43081j]

I'm saying these are the parts which exist in an end-to-end solution:

• browser implementation (window.agent)
• a bridge (chrome extension, MCP server, whatever. a piece of code that calls window.agent methods somehow)

the browser implementation here is literally the definition and implementation of window.agent.

for example, in our implementation, we have a ToolsRegistry just like a CustomElementsRegistry which exists on window.agent.tools. you can then define, list, etc. that is the thing i think we should be standardising in this proposal. That and the event which fires to call a tool.

beyond that, and outside of the scope of this proposal, would be "bridges" which call this.

a non-exhaustive list of example bridges:

• a chrome extension which calls tools in the page
• an MCP server which uses CDP to call tools in the page
• an in-browser library which calls tools in the page

all of these are just implementations the community and companies will build over time once there's a definition for window.agent.

so to visit your "add to cart" example:

• the page would define a tool for how something is added to cart (window.agent.tools.define(toolDef))
• the user's client (the "bridge" mentioned above) calls this tool
  • this could be by using CDP to evaluate injected code that call the tool in the page, for example

Note: ofc loads of bike shedding will happen and the various names here will change over time. but hopefully the concept makes sense

[khushalsagar]

this could be by using CDP to evaluate injected code that call the tool in the page, for example

@43081j this is exactly where the web API needs to be opinionated though and was the topic of discussion in #16. APIs to list/execute tools implies any entity which can run script on the page will be able to use these tools. On the surface that sounds reasonable, if the entity has privileges to execute script on the page then it is a trusted source that should be able to use WebMCP tools as well.

But if we support that paradigm then there's a lot of problems which will be forced into user space, some which can't be solved without primitives from the Web API.

• Security policies and permissions will have to be designed to manage the composition of tools across cross-origin pages. Any consumer of these tools will need to faithfully apply those. As an example, an embedder could indicate whether tools from an iframe should be mixed with it's own tools.

• Tool execution from multiple Agents could happen concurrently. Should that be allowed? Does it unnecessarily increase the complexity the site needs to manage, they'll need a way to understand which Agent/task a tool call corresponds to. If the browser is arbitrating, it can ensure only 1 task from 1 Agent is running at a time.

• Managing action specific permissions #44 brought up the need for action specific permissions. These permissions by design would be scoped to a specific Agent. Maybe the user trusts Agent x to use an action but not Agent y (it's less reliable etc). Should that be done by the author? At the very least they need the browser to provide a way to store agent specific state. Or it could be a high level API, the site indicates whether an operation should be permission guarded. And the browser manages asking the user and storing the permission state for each Agent.

I don't think exposing access to consumers by injecting script on the page is a good idea. It's better for the browser to arbitrate this so we can minimize the complexity site authors have to deal with.

[43081j]

Security policies and permissions will have to be designed to manage the composition of tools across cross-origin pages. Any consumer of these tools will need to faithfully apply those. As an example, an embedder could indicate whether tools from an iframe should be mixed with it's own tools.

Let's say this exists, it just means the window.agent registry populates its tools with those from child frames, too. The client isn't aware of this and doesn't need to care - as far as it is concerned, the page as a whole exposes tools and it doesn't matter to the client where they came from.

Tool execution from multiple Agents could happen concurrently. Should that be allowed? Does it unnecessarily increase the complexity the site needs to manage, they'll need a way to understand which Agent/task a tool call corresponds to. If the browser is arbitrating, it can ensure only 1 task from 1 Agent is running at a time.

I don't think this is for us to solve. A defined tool can do anything, because it is whatever the page author chose to make it do. I think it is their responsibility to deal with the fact their tool could be called multiple times. Whether multiple agents caused that, or an in-page script, doesn't really matter since anything could've caused it.

#44 brought up the need for action specific permissions. These permissions by design would be scoped to a specific Agent. Maybe the user trusts Agent x to use an action but not Agent y (it's less reliable etc).

this one is interesting and there's no clean solution I can think of off the top of my head. but yes, if this were to be supported, we would need a way of tying tool calls to agents (and the permissions too).

---

Imagine we, as a page author, have defined some tools. These are just:

• regular functions which return tool results (but nothing is stopping us from returning nonsense, we can do whatever we want, it just wouldn't work)
• callable through some means, probably a toolcall event on the window
• listable and retrievable (their definition) from the registry

Now, given this, here are just a few ways I can call these tools:

• an MCP server which uses CDP to emit the event and wait for the callback
• an in-page library which emits the event to its own window and waits for the callback
• a Chrome extension which emits the event into the tab's window and waits for the callback
• a test runner which uses playwright (CDP) to emit the event and wait for the callback
• an MCP server which uses webdriver or similar to emit the event and wait for the callback
• an in-page library which emits the event to its parent window and waits for the callback

you could go on forever because this is userland code, we can do whatever we want, and people will.

so is this proposal really going to try define all of those (including the ones i haven't thought of)?

because of this, i think we should be defining the web API which exists in the window, along with the permissions model and what not. but who calls it... we don't need to care.

Google can add a helper to CDP for example, but its still going to do what every other client did: emit an event and wait for the callback.

hopefully this makes some more sense now. I can see everything you listed being solvable this way, without needing to concern ourselves with what clients people build. Of course, we'd still want to know to help design the web API.

[khushalsagar]

Thanks for the extensive feedback. Just want to emphasize I appreciate the thought you're putting here!

Let's say this exists, it just means the window.agent registry populates its tools with those from child frames, too. The client isn't aware of this and doesn't need to care - as far as it is concerned, the page as a whole exposes tools and it doesn't matter to the client where they came from.

The state in window.agent is limited to what came from the Document associated to this window. It by design can not have tools from it's embedder's Document or any child frames it's embedding. That would be a violation of the isolation guaranteed by iframes. So if script is injected to find tools from the window object, it won't include iframes. That script will have to be injected across all frames on the page.

And I think this proves my point. :) The user agent implementations have to be very cautious in how they expose this functionality to Agents. It's not reasonable to expect consumers to get it right by injecting script and invoking tools directly from the window object. Especially in the context of Web's security principle of origin isolation.

I don't think this is for us to solve. A defined tool can do anything, because it is whatever the page author chose to make it do. I think it is their responsibility to deal with the fact their tool could be called multiple times. Whether multiple agents caused that, or an in-page script, doesn't really matter since anything could've caused it.

You're suggesting that tools are stateless functions so multiple Agents should be able to invoke them concurrently. I don't think that's the right way to look at it. A tool execution is bound to have a side-effect on another. It's absolutely possible that some tools can be executed concurrently (readOnly ones come to mind). But a tool like undo is inherently stateful. If one Agent is connected to the site at a time, we can expect a reasonable execution sequence. But multiple Agents would stomp on each other.

Even MCP is a stateful protocol, there's a persistent connection between the client <-> Agent. Conceptually the site seems like an MCP server which can connect to a single MCP client (Agent) at a time. And the browser should arbitrate what that Agent is on the user's behalf.

you could go on forever because this is userland code, we can do whatever we want, and people will.

Absolutely! And you nailed it, if it's all custom userland code/hooks then the author is in complete control of what they're exposing, to which Agent and the expectations.

But the point of a spec is to explicitly define the API contract for the developer, ensure it can support all targeted use-cases, define what the browser is responsible for and ensure the API gives browsers enough flexibility to implement it without being too prescriptive of details which can differ across browsers.

---

A lot of good points raised on this issue so just want to summarize the high level questions I'm seeing below. We should take these in separate issues.

• Should more than 1 Agent be allowed to connect to a site at a time? MCP handles this by having a stateful protocol that tool calls go through. By design only 1 Agent can invoke tools at a time.

• Related to above, should we also have the concept of a stateful session? I wasn't sure if it's necessary if the browser is ensuring only 1 Agent at a time. The browser can abort any ongoing tool calls from current Agent before another one connects.

• On Add API to list / execute tools? #16 (comment) we concluded that the registry of tools should be internal browser state. No JS APIs to list/execute/tools_changed notification. Hiding it gives browser implementors flexibility in exposing the functionality with good security practices. I'd love to hear feedback if this is limiting for any case and happy to revisit the resolution!

[43081j]

And I think this proves my point. :) The user agent implementations have to be very cautious in how they expose this functionality to Agents. It's not reasonable to expect consumers to get it right by injecting script and invoking tools directly from the window object. Especially in the context of Web's security principle of origin isolation.

To me, this is just the security model of the web API we'd be introducing. Again, clients don't need to care.

If a child window can't expose tools to a parent, it is what it is. That means the parent page is responsible for defining a tool which internally interacts with the child window via the existing browser security model.

The client didn't need to know about this at any point, its just an implementation detail of the tools the page provides.

You're suggesting that tools are stateless functions so multiple Agents should be able to invoke them concurrently. I don't think that's the right way to look at it. A tool execution is bound to have a side-effect on another. It's absolutely possible that some tools can be executed concurrently (readOnly ones come to mind). But a tool like undo is inherently stateful. If one Agent is connected to the site at a time, we can expect a reasonable execution sequence. But multiple Agents would stomp on each other.

To be clear, I'm not suggesting tools are stateless functions.

All of these tools are plain functions running in a window whatever way we do this. Which means the author is responsible for ensuring functions which can cause conflicting/overlapping actions take each other into account. Outside of MCP, this is just basic stuff someone should consider when exposing functions to an external tool.

So again i would not expect the client to be aware of any of this. It is the responsibility of the tool author to ensure their tool runs correctly, including handling concurrency issues.

But the point of a spec is to explicitly define the API contract for the developer, ensure it can support all targeted use-cases, define what the browser is responsible for and ensure the API gives browsers enough flexibility to implement it without being too prescriptive of details which can differ across browsers.

This sounds like something we'd achieve by defining the web API (window.agent) and associated permission model, etc.

---

At this point, nothing you said tells me we need to care about how someone builds a client. For prior art, and use cases, yes. Standardising how a client is architected, probably not.

Not all clients will be LLM agents for example. Looking at my list from before, they can be code in the same page, in an extension, etc.

We can't possibly standardise how those are built. I think the web API needs to be standardised so anything, regardless of how it is built, can call it in a standard way.

For example, handling multiple agents can be delegated to the page author. Or we can come up with some locking-like behaviour the window.agent API implements. But none of that defines how a client is built.

[pengw272]

@khushalsagar, @43081j thank you for the thoughtful discussion!

@khushalsagar

APIs to list/execute tools implies any entity which can run script on the page will be able to use these tools. On the surface that sounds reasonable, if the entity has privileges to execute script on the page then it is a trusted source that should be able to use WebMCP tools as well.

I'm trying to understand this better by visualizing the end to end product flow. Specifically, how does a browser agent discover/use the tools. From the Readme, a browser agent is "An autonomous assistant as described above but provided by or through the browser. This could be an agent built directly into the browser or hosted by it, for example, via an extension or plug-in.".

What about "3rd party browser agents"? By this, I mean AI Chatbot from companies like Zendesk, Gorgias, etc. Let's say I own an e-comm website, and I use one of these AI Chatbots on my site, and I see it as a shopping companion to help my users shop and help me boost conversion. Conceivably, the AI chatbot may need to do things like adding items to users shopping cart. Consider the following example:

User: What are best sellers?
Bot: A, B, C
User: Tell me more about A
Bot: <Describes A>, would you like me to add it to cart?
User: Yes

At this point, the AI Chatbot need to call the addToCart tool. How does it do it?

And if we don't allow it to do it, then isn't this creating some sort of anti-competitiveness that favors browser vendors?