The Agent-Native Digital Art Gallery powered by WebMCP
Where AI acts as the physical "Director of Attention" inside the browser.
Hackathon Submission: Built for The WebMCP Challenge on Devpost.
🎬 Click above to watch the 3-minute video walkthrough showcasing autonomous agent actuation, spatial navigation, and the 12 WebMCP tools.
Mostra d'Arte is an immersive digital art gallery designed from the ground up for the W3C WebMCP (Web Model Context Protocol) standard.
Traditional web applications require humans to navigate complex menus, sliders, and coordinate inputs, while typical AI chatbots remain isolated in a disconnected chat box, unable to touch or actuate the visual experience.
Mostra d'Arte inverts this paradigm:
By exposing client-side tools to browser AI agents via document.modelContext.registerTool(), the AI transitions from a passive conversationalist into an active "Director of Attention" — steering high-precision camera zooms, adjusting museum illumination, playing spatial room acoustics, opening curatorial research dossiers, and even issuing digital VIP admission passes in real time.
📐 Looking for complete protocol specifications, sequence diagrams, and component models? Check out System Architecture Specification (ARCHITECTURE.md).
In accordance with Google Chrome's WebMCP Architecture Guidelines, Mostra d'Arte is built to power three distinct Critical User Journeys (CUJ):
- Goal: The visitor wants to visually understand subtle symbolism, hidden details, and geometric composition without manually panning and pinching.
- Verified Agent Prompt: "Exhibit the Mona Lisa, zoom in on her hands, and turn on the focused spotlight for dramatic atmosphere."
- Agent Execution Chain:
switch_masterpiece({ artwork_id: "mona-lisa" })➔zoom_painting({ x: 42, y: 83, zoom: 3.8, detail_name: "The Restful Folded Hands" })➔toggle_spotlight({ active: true }).
- Goal: The visitor explores the canonical collection on the grand salon wall.
- Verified Agent Prompt: "Show me the entire gallery collection on the salon wall"
- Agent Execution Chain:
open_salon_browser({ open: true }).
- Goal: The visitor wants to book real-world admission tickets through the agent conversation without filling out complex multi-page booking forms.
- Verified Agent Prompt: "Book 2 VIP passes for Alice on tomorrow's evening session"
- Agent Execution Chain:
reserve_gallery_pass({ visitor_name: "Alice", tickets_count: 2, date: "Tomorrow", session: "Evening VIP & Nocturne" })➔ Generates official digital VIP pass modal with confirmation ID and QR verification on screen.
All 12 tools are registered in real time via document.modelContext.registerTool() with cooperative cancellation (AbortSignal):
| # | Tool Name | Description | Annotations | Key Parameters |
|---|---|---|---|---|
| 1 | zoom_painting |
Physically directs canvas camera to focus on microscopic details and brushstrokes. | readOnlyHint: true |
x (0-100), y (0-100), zoom (1.5-8.0), detail_name |
| 2 | switch_masterpiece |
Dynamically changes the exhibited artwork on the gallery wall. | readOnlyHint: true |
artwork_id (starry-night, mona-lisa, etc.), reason |
| 3 | reset_view |
Restores canvas zoom back to 1.0x wide perspective. | readOnlyHint: true |
note (optional string) |
| 4 | toggle_spotlight |
Actuates dramatic museum vignette spotlight beam for heightened focus. | readOnlyHint: true |
active (boolean) |
| 5 | get_artwork_details |
Retrieves provenance, color palette symbolism, and curated hotspot coordinates. | readOnlyHint: true |
artwork_id (optional string) |
| 6 | start_guided_tour |
Launches autonomous thematic tour with audio narration and camera waypoints. | readOnlyHint: true |
tour_id (optional string) |
| 7 | toggle_curatorial_dossier |
Opens slide-out research drawer, optionally switching to palette, overview, or tours. |
readOnlyHint: true |
open (boolean), tab (overview, palette, focal, tours) |
| 8 | toggle_ambient_acoustics |
Starts or mutes spatial museum room reverberation synthesized via Web Audio API. | readOnlyHint: true |
active (boolean) |
| 9 | reserve_gallery_pass |
Transactional: Issues confirmed VIP exhibition pass with reservation ID and QR badge. | (Action) | visitor_name, date, tickets_count, session |
| 10 | toggle_composition_grid |
Overlays or hides the Golden Ratio dynamic geometric analysis grid on the canvas. | readOnlyHint: true |
active (boolean) |
| 11 | open_salon_browser |
Opens the architectural Salon Wall browser with period filtering (Dutch Golden Age, etc.) and search. |
readOnlyHint: true |
open (boolean), period (string), search (string) |
| 12 | docent_speak |
Projects curatorial narration, educational insights, or answers directly into the Virtual Docent dialogue window on screen. | readOnlyHint: true |
message (string) |
Mostra d'Arte strictly adheres to the W3C Web Machine Learning Working Draft and Chromium Origin Trial specifications:
-
Imperative JavaScript API Architecture (§ 4.2):
Mostra d'Arte adopts the Imperative WebMCP API (document.modelContext.registerTool) rather than the declarative HTML form attribute approach (<form toolname>). Because our digital gallery actuates real-time canvas spring physics, Web Audio harmonic synthesis, and stateful React camera viewports, the imperative API grants direct programmatic actuation, JSON Schema validation, and bidirectional feedback with the browser agent. -
Multimodal Vision via Annotated Page Content (APC) (§ 5.2):
Chromium agents do not rely solely on DOM trees; they take real-time visual observations (APC snapshots) of the canvas. Mostra d'Arte connects this visual cortex with physical tool execution — the agent visually perceives details on canvas and translates them into precise viewport coordinates viazoom_painting. -
Dynamic Tool Registry via
getTools()&toolchange(§ 4.4):
The UI subscribes to nativedocument.modelContext.addEventListener("toolchange", ...)events and queries active tools dynamically viadocument.modelContext.getTools(), updating the telemetry badge without hardcoded lists. -
Security Intent Declarations (
readOnlyHint) (§ 4.2.1):
Exploratory tools declareannotations: { readOnlyHint: true }to eliminate confirmation friction, while transactional tools likereserve_gallery_passexecute real-world booking flows. -
Cooperative Cancellation (
AbortSignal) (§ 3.1):
All tool execution callbacks honor{ signal: AbortSignal }, immediately aborting active camera springs or network requests if the user changes commands. -
Initial State Awareness & Contextual Binding (Google Framework):
Adhering to Google's "Define the Initial State" guideline, tools are architected with full awareness of the user's active viewport and screen context (currentArtworkRef.current). Callingget_artwork_details({})without parameters immediately returns deep context on the painting currently mounted on the wall,start_guided_tour()automatically narrates the exhibited masterpiece, andreserve_gallery_pass()binds the active exhibition title to the confirmed pass. The agent never operates in the dark. -
Graceful Error Recovery & Natural Language Variance Tolerance:
Adhering to Google Chrome's WebMCP architectural guidelines ("Fail gracefully and enable recovery" & "Address variance"), tools never terminate in raw protocol crashes (throw Error). Instead, they return actionable context-aware guidance (e.g., suggesting all canonical masterpiece IDs if an unrecognized painting is queried). Furthermore, tools absorb conversational variance through tab synonym mapping ("colors"/"pigments"➔"palette"), fuzzy movement matching ("Dutch"➔"Dutch Golden Age"), default fallback parameters, and defensive mathematical sanitization guarding againstNaNcoordinates.
You can test and actuate the gallery tools using two official Google methods:
- In Google Chrome, navigate to
chrome://flags/#enable-webmcp-testing. - Set the flag to Enabled and relaunch Chrome.
(Official Origin Trial ID:
4163014905550602241)
Chrome includes a dedicated WebMCP DevTools Debugger built directly into the Application panel:
- Open Mostra d'Arte in Chrome.
- Press
F12to open Chrome DevTools. - Go to the Application tab, and select WebMCP from the left sidebar.
- Under Available Tools, click any tool (e.g.
switch_masterpiece,zoom_painting,toggle_spotlight). - In the Run Tool pane on the right, enter parameters (e.g.
artwork_id: mona-lisa), click Run tool, and watch the React canvas actuate live with real-timeCompletedlogs!
- Install the official Model Context Tool Inspector Extension from the Chrome Web Store.
- Open Mostra d'Arte in Chrome.
- Open the extension side panel — you will immediately see all 12 WebMCP tools detected automatically.
- Verified Evals & Benchmark Suite (Google Evals Framework):
Run these 12 verified benchmark prompts to test model tool selection, parameter binding, and real-time UI actuation:
| # | Target WebMCP Tool | Verified Agent Prompt | Expected UI Actuation |
|---|---|---|---|
| 1️⃣ | zoom_painting |
"Show me the details of the moon in this painting" | Camera glides and magnifies canvas details |
| 2️⃣ | switch_masterpiece |
"Show me Michelangelo painting" | Wall exhibition mounts the requested masterpiece |
| 3️⃣ | reset_view |
"Show me the edge of the cliff with flowers, then reset the view" | Zooms into detail, then smoothly restores 1.0x wide canvas and room lighting |
| 4️⃣ | toggle_spotlight |
"Exhibit the Mona Lisa, zoom in on her hands, and turn on the focused spotlight for dramatic atmosphere." | Dynamically tracks target coordinates and darkens room for dramatic museum lighting |
| 5️⃣ | get_artwork_details |
"Can you switch the gallery display to wave. Tell me the historical background and key focal points of this painting, then zoom in on the most symbolic detail." | Retrieves provenance & archives, switches painting, and steers camera |
| 6️⃣ | start_guided_tour |
"Start a guided tour of The Starry Night." | Launches autonomous waypoint tour with synchronized audio narration |
| 7️⃣ | toggle_curatorial_dossier |
"Show me pearl." "What pigments and colors did Vermeer use for this masterpiece? Show me the palette." "Close the dossier and return to full canvas" |
Slides out research drawer on Palette or Overview; closes on command |
| 8️⃣ | toggle_ambient_acoustics |
"Turn on the gallery ambient music to set a contemplative mood." "Mute the gallery ambient sound." |
Web Audio synthesizer starts or mutes gallery acoustics |
| 9️⃣ | reserve_gallery_pass |
"Book 2 VIP passes for Alice on tomorrow's evening session" | Generates verified digital VIP pass modal with QR badge |
| 🔟 | toggle_composition_grid |
"Switch to Botticelli." "Analyze the geometric balance of this painting and show the composition grid." "Hide the composition grid." |
Displays or hides Golden Ratio dynamic geometric overlay |
| 1️⃣1️⃣ | open_salon_browser |
"Show me the entire gallery collection on the salon wall" | Opens architectural full-screen salon gallery collection |
| 1️⃣2️⃣ | docent_speak |
"Explain the pigments used in Starry Night and project your curatorial insights into the Virtual Docent." | Streams agent's curatorial commentary into on-screen Virtual Docent dialogue (with voice) |
- Web Standard: W3C WebMCP (Imperative JS API via
document.modelContext+webmcp-types) - Development Environment: Google Antigravity IDE
- Design & Prototyping: Google AI Studio (Gemini-powered UI/UX design & curatorial prototyping)
- Framework: React 19 + TypeScript + Vite
- Styling: Vanilla CSS + Tailwind CSS v4
- Animations: Motion (
motion/react) - Sound: Web Audio API harmonic chord synthesizer
- Speech: Web Speech API (
SpeechSynthesis) docent narration - Icons: Lucide React
This project is open-source and available under the MIT License.