Vitre

WebView automation for Kotlin Multiplatform. Describe what you want done to a page as a list of steps, then run it inside an embedded WebView on Android, iOS and the desktop: from your app, from a test, or from an LLM agent over the Model Context Protocol or Koog.

val workflow = workflow("hn-top-story", "Hacker News top story") {
    navigate("https://news.ycombinator.com/")
    waitFor(".titleline > a", timeoutMs = 15_000)
    extract(".titleline > a", into = "headline")
    extract(".titleline > a", into = "url", from = Source.Attribute("href"))
}

WorkflowEngine(controller).run(workflow).collect { event ->
    if (event is WorkflowEvent.Completed) println(event.variables["headline"])
}

The same vocabulary is available four ways: as the Kotlin DSL above, as a Compose composable you drop into a screen, and, for an agent that has never seen the page, as MCP tools or as Koog tools. Those last two are two deliveries of one set of semantics rather than two implementations of them.

The block is a builder rather than a script. It runs once, up front, to assemble the step list that WorkflowEngine then walks, so ordinary Kotlin if inside it chooses what the workflow contains. When the decision belongs to the run instead — an element that is only sometimes there, a value an earlier step extracted — runIf appends a step the engine evaluates against the page. The WorkflowStep constructors remain public and equivalent; they are what vitre-mcp uses, since an agent's steps arrive as JSON rather than as Kotlin.

Mobile apps already embed WebViews, and everything people want to do with them (read a page, fill a form, talk to the page's own script, scrape a table, run four sites at once) ends up as one-off evaluateJavascript calls glued to a callback. Those calls race each other, they race the UI, and none of it is shared between Android, iOS and the desktop.

Vitre makes the page something you can drive: one ordering guarantee, one step vocabulary, one codebase for every platform, and a snapshot format an agent can read.

• The steps are declarative: Navigate, LoadHtml, WaitFor, Click, Input, Extract, ExtractRows, Snapshot, EvaluateJs, PostMessage, AwaitMessage, and If for the branches the page decides rather than the builder.
• Elements are addressed three ways: CSS, XPath, and handles issued by a page snapshot.
• evaluateJs returns a script's result JSON-encoded on every platform, so controller.evaluate<Boolean>(…) decodes it instead of comparing it against "true".
• postMessage works in both directions, with an inbox so a message that arrives before you wait for it is not lost. Payloads are typed on both ends: bridge.request<Ack, Token>(…) for a round trip, decodePayload<Token>(…) for a workflow's.
• controller.cookies reads, writes and clears the WebView's cookie jar, including the HttpOnly cookie the site keeps its session in and document.cookie cannot see, with host/path/Secure/SameSite scoping applied the way a request would apply it. Android and iOS have this today; the desktop reports null until CEF's two jars are reconciled.
• A lane pool drives up to four sites at once: one WebView per lane, one workflow engine each, queued so six workflows on a two-lane device run three deep instead of losing four.
• vitre-mcp exposes the whole vocabulary as MCP tools over an in-process transport, and vitre-koog exposes it as native Koog tools with a plugin that holds the page for the length of an agent run. Both read one set of semantics out of vitre-agent, so neither can drift into telling a model something the other does not.
• Every platform call is confined to the WebView thread and totally ordered, so the engine, your UI, and an agent can drive the same page without special-casing each other.

Web is out of scope as a target. Browser CORS rules make a general-purpose web automation framework impractical there; see docs/PLAN.md.

The sample gallery on Android. The same screens run on iOS and the desktop from the same composeApp; past 720dp wide the list and the runner sit side by side instead of taking turns.

The index. The agent chat, the parallel-lane scenarios, then one card per single-page workflow.	A workflow, run. The page on top, every step underneath with what it actually did. Steps 4 and 5 address elements by the handles Snapshot issued, not by selectors.
Price scout. An example workflow to retrieve info from third-party websites.	Agent chat. A mocked model over a real MCP server: each action on the page is a tools/call, shown with the JSON that came back rather than hidden behind the answer.

What each entry demonstrates is spelled out in What's in the sample gallery; to run it yourself, see Running the sample.

Published to Maven Central, so mavenCentral() in your repositories is all the setup there is:

// build.gradle.kts
kotlin {
    sourceSets {
        commonMain.dependencies {
            implementation("dev.ggoggam.vitre:vitre-core:0.1.2")
            implementation("dev.ggoggam.vitre:vitre-compose:0.1.2") // optional
            implementation("dev.ggoggam.vitre:vitre-mcp:0.1.2")     // optional
            implementation("dev.ggoggam.vitre:vitre-koog:0.1.2")    // optional
        }
    }
}

Or consume it as a source dependency: clone the repo next to your project, add includeBuild("../vitre") to settings.gradle.kts and drop the versions above. If you vendor the sources instead, add the modules directly with include(":vitre-core").

Mount a WebView, take its controller, run a workflow against it:

@Composable
fun Screen() {
    val state = rememberVitreWebViewState("https://example.com")
    val controller = state.controller

    LaunchedEffect(controller) {
        val page = controller ?: return@LaunchedEffect
        WorkflowEngine(page).run(SampleWorkflows.ExampleDotComTitle).collect { event ->
            when (event) {
                is WorkflowEvent.StepStarted -> log("→ ${event.step}")
                is WorkflowEvent.Completed -> log(event.variables["title"])
                is WorkflowEvent.Failed -> log("step ${event.stepIndex}: ${event.message}")
                else -> Unit
            }
        }
    }

    VitreWebView(state = state, modifier = Modifier.fillMaxSize())
}

state.controller is null before the WebView mounts and null again after it leaves the composition, so an effect keyed on it starts when the page arrives and tears down when it goes away, and nothing runs against a dead WebView.

The smallest thing worth doing: navigate, wait for the element that means the page is ready, read what you came for. Text is textContent; href is genuinely an attribute, so it is read as one.

workflow("hn-top-story", "Hacker News top story") {
    navigate("https://news.ycombinator.com/")
    waitFor(".titleline > a", timeoutMs = 15_000)
    extract(".titleline > a", into = "headline")
    extract(".titleline > a", into = "url", from = Source.Attribute("href"))
}

ExtractRows returns one JSON record per matching row, with each column resolved within that row. That scoping is the point: a row missing a price yields an empty string in that record instead of shifting every later record onto the wrong product.

extractRows(rows = xpath("//li[@data-sku]"), into = "results", limit = 10) {
    // "." is the row itself — read its own attribute.
    column("sku", xpath("."), from = Source.Attribute("data-sku"))
    column("price", xpath(".//span[@class='price']"))
    // Matched on the text a human reads, not on a class hook.
    column("stock", xpath(".//*[normalize-space()='In stock']"))
    // Up to the row and back down — CSS has no parent combinator.
    column("seller", xpath(".//span[@class='price']/ancestor::li[1]//span[@class='seller']"))
}

If the page is yours, the bridge beats scraping it. PostMessage sends a MessageEvent('vitre') into the page; AwaitMessage waits for window.vitre.postMessage coming back. The inbox buffers, so a handler that posts synchronously on click is still matched by an AwaitMessage that starts afterwards, which is the case that silently loses messages elsewhere.

@Serializable data class Ack(val seen: Boolean)
@Serializable data class Token(val value: String, val expiresAt: Long)

loadHtml(html = checkoutHtml, baseUrl = "https://app.example.com")
waitFor("#pay")
click("#pay")
awaitMessage(type = "payment-token", into = "token", timeoutMs = 5_000)
postMessage(type = "ack", payload = Ack(seen = true), id = "ack-1")

Payloads are classes rather than hand-typed envelope strings. id and type stay arguments because they are protocol. The reply arrives in a variable, and the typing picks up again where the values are:

if (event is WorkflowEvent.Completed) use(event.decodePayload<Token>("token"))

The split is deliberate. A workflow block is a builder: it runs once to assemble the step list, and the steps run later, so there is no point in the block at which a reply could be returned to it. When you want a round trip as one expression, you want the host API rather than a workflow. request posts, correlates the answer by replyTo, and gives it back typed:

val token: Token = controller.bridge.request<Ack, Token>("issue-token", Ack(seen = true))

Post-then-await does not race here either: the inbox buffers the reply from a page handler that answers synchronously, which is the normal case, until the wait starts.

The same applies to a script's own result. evaluateJs hands back the JSON encoding of what the expression produced, the one contract every platform was made to agree on, so a value read out of a page can be decoded rather than string-matched:

val ready: Boolean = controller.evaluate("document.readyState==='complete'")
val rows: List<Product> = controller.evaluate("Array.from(document.querySelectorAll('li')).map(toRow)")

A cookie banner, an interstitial, a login form that appears when the session lapsed. waitFor is the wrong tool for all of them — it fails the run when the thing legitimately is not there — and a Kotlin if in the builder cannot help, because it has already finished running by the time the page loads.

runIf appends a WorkflowStep.If the engine evaluates in place. Conditions are values, not JS strings, so a failure can name what went wrong: exists, variableEquals, variableMatches, jsTruthy, combined with and / or / not.

navigate("https://shop.example.com/cart")

// Neither outcome is a failure. The banner is dismissed if it is there, and skipped if it is not.
runIf(exists("#cookie-banner")) {
    click("#cookie-banner .accept")
}

extract("#checkout-status", into = "status")

// Branching on a value the run itself produced — nothing at build time knows what this says.
runIf(variableMatches("status", "session expired"), otherwise = { click("#pay") }) {
    click("#sign-in")
    input("#password", secret)
    waitFor("#checkout-status")
}

Branches nest, and step numbering nests with them: a failure inside one reports a StepPath like 3.then.1 rather than a flat index that would point at the If — the one step that did not fail. Exists is deliberately the one condition where a missing element is an answer instead of an error, including for a stale snapshot handle, which every other step rejects outright.

One WebView per lane, each loading its site as a top-level document, which is what keeps sessions first-party and X-Frame-Options out of the picture. Hand the pool every workflow and it drains them across however many lanes the device could carry.

var pool by remember { mutableStateOf<FramePool?>(null) }

VitreFrameHost(
    laneCount = 4,
    policy = InterceptionPolicy(handlers = shopFixtures),
    onPoolReady = { pool = it },
)

// Six workflows on a two-lane device run three deep — nothing is dropped.
pool?.run(shops.map { it.workflow(query) })?.collect { (taskIndex, laneId, _, event) ->
    if (event is WorkflowEvent.Completed) merge(event.variables["results"])
}

The sample's Price scout does exactly this: four synthetic shops at four distinct origins, merged and ranked by delivered price, which for most of the catalogue is a different shop from the cheapest sticker price.

A RequestHandler answers requests from memory, so a test drives a real WebView against a real origin with no network and no flake. This is how the parallel-lane demo stays a usable smoke test.

val fixtures = RequestHandler { request ->
    when (request.host) {
        "shop-a.test" -> InterceptedResponse(body = shopAHtml.encodeToByteArray())
        else -> null // fall through to the network
    }
}

VitreFrameHost(policy = InterceptionPolicy(handlers = listOf(fixtures)), …)

Interception is real on Android and the desktop, which both let an application answer a request outright: shouldInterceptRequest on one, CEF's resource pipeline on the other. iOS is the exception, since WKURLSchemeHandler refuses to register for https, so fixtures there are served from a private scheme and nothing can rewrite a response header. The failure modes are spelled out in docs/PARALLEL-LANES.md.

Snapshot answers the question a hand-written workflow never has to ask: what is on this page? It returns the interactive and text-bearing elements as an indented outline with a handle each, the same information as the HTML at roughly a third of the tokens.

Vitre fixture — https://fixture.vitre.test/
heading "Vitre fixture" [ref=e1]
textbox value="typed by handle" [ref=e3]
button "Send pong to native" [ref=e4]

From there the agent names no selectors at all:

snapshot(into = "page")
input(handle("e3"), text = "typed by handle")
click(handle("e4"))
extract(handle("e5"), into = "status")

vitre-mcp puts that behind an MCP server: snapshot, navigate, click, type, wait_for, extract, extract_rows, evaluate, send_message, await_message, plus acquire_lease / release_lease for holding a page across several calls. The host registers its WebViews and the server drives exactly those:

val sessions = WebViewSessions().apply { register("main", controller, "the shopping tab") }
val server = McpServer(sessions, scope)
val transport = InProcessMcpTransport(server)

It ships an in-process transport only, on purpose: a loopback socket would expose page automation to anything on the device that can reach the port, and on a WebView signed into the user's accounts that leaks the session, not merely the automation. See docs/MCP.md.

If the agent is written with Koog, vitre-koog hands it the same thirteen tools as Kotlin objects, with typed arguments, no server, and the same names, so a system prompt written for one adapter works against the other:

val driver = PageDriver(sessions, scope)

val agent = AIAgent(
    promptExecutor = executor,
    llmModel = OpenAIModels.Chat.GPT4_1,
    systemPrompt = PageToolDocs.INSTRUCTIONS,
    toolRegistry = ToolRegistry { vitreWebView(driver) },
)

A host that already runs the MCP server can bridge that instead. vitreMcpToolRegistry(server) reads tools/list and translates the schemas, so there is one description of the toolset rather than two. Point the typed tools at server.driver if you want both against one lease registry.

The plugin is VitrePageLease. Ordering already stops two callers corrupting each other's individual steps, but it cannot make a sequence indivisible, and an agent is nothing but sequences. A user tapping "next page" between the agent's wait_for and its extract yields a price read off the wrong page, with every operation correctly serialised. acquire_lease fixes that, at the cost of requiring the model to remember to use it. The plugin takes the lease itself, for the length of the run, and the model never sees it:

val agent = AIAgent(
    promptExecutor = executor,
    llmModel = OpenAIModels.Chat.GPT4_1,
    systemPrompt = PageToolDocs.INSTRUCTIONS,
    // Lease tools off: the run already holds the page, and a model that asks for it again is
    // queueing behind itself until its own lease expires.
    toolRegistry = ToolRegistry { vitreWebView(driver, includeLeaseTools = false) },
) {
    install(VitrePageLease) { driver = pageDriver; ttlMs = 120_000 }
}

The lease is bounded by a TTL, because the page is held while the agent waits on an LLM, and a WebView the user can see is a UI that has stopped responding to its own app. See docs/KOOG.md.

Every element-addressing step takes a Locator: css("#results .item"), xpath("//li[@data-sku]") or handle("e7"). A bare string still means CSS.

Neither CSS nor XPath pierces shadow DOM; that is the page's doing, not the query language's.

A handle is the third kind and behaves differently by design: the other two describe how to search, a handle names an element. It is issued by the page, dies with the document that issued it, and is never recycled, so a handle from the previous page fails loudly instead of resolving against a same-shaped element on the new one.

A WebView owns a thread, the platform main thread, and that is not negotiable: WKWebView is UIKit, and an Android WebView must be used on the thread that built it. So callers do not synchronise with each other, they queue. One object, WebViewSerializer, confines every platform call to that thread and totally orders them, which is what lets the workflow engine, the UI, and an agent drive the same page without special-casing each other. The engine itself runs on Dispatchers.Default and never sees the WebView thread.

See docs/CONCURRENCY.md for the model and the bugs it fixed.

mise install            # ktlint (mise.toml) + the android CLI (mise.local.toml)
mise run test           # library + sample allTests
mise run test:android   # the on-device Koog agent test (needs a device or emulator)
mise run build          # library artifacts + Android sample APK + iOS debug framework
mise run lint           # ktlint
mise run fmt            # ktlint --format

A JDK is not declared in mise.toml. Gradle uses whatever JDK is on your machine, and the toolchain is resolved via foojay. CI installs its own from mise.ci.toml. mise run wrapper regenerates the Gradle wrapper and is the one task that needs a system gradle; you only need it when bumping Gradle.

mise run dev:android    # build + install + launch on a phone, or boot an AVD
mise run dev:ios        # build + install + launch on a simulator
mise run dev:desktop    # build + launch the desktop window

All three build and launch in one step, so none of them needs Android Studio or Xcode open. dev:android prefers a plugged-in device over an emulator and boots the first AVD if nothing is attached; dev:ios reuses a booted simulator, or set VITRE_SIM to pick one by name. They live in mise.local.toml (dev only, and CI never runs them). mise run android:install is the plain fallback: gradlew installDebug onto whatever adb already sees, without the android CLI.

You can also open sample/iosApp/iosApp.xcodeproj in Xcode and hit Run, since the target's "Compile Kotlin Framework" phase builds the KMP framework first either way.

dev:desktop is the odd one out on first run: unlike a WebView, Chromium is not already on the machine, so KCEF downloads and unpacks a bundle of a few hundred megabytes into ~/.vitre/kcef-bundle before the gallery appears. The sample shows that as a progress screen and it happens once per machine; later launches go straight to the window.

The gallery opens on the agent chat: a WebView with a conversation under it, where every action on the page arrives as an MCP tool call and the calls and their results are shown rather than hidden behind the answer. The model is mocked, with no LLM API and no key, but nothing downstream of it is. Each turn issues a real tools/call over JSON-RPC and the answer is computed from what came back. One scripted exchange guesses a CSS selector that does not match, so the transcript shows a tool failure arriving as an isError result the model reads and corrects, rather than as an exception that ends the run.

Below it are two parallel-lane scenarios: Price scout (four synthetic cross-origin shops, ranked by delivered price) and Live pages probe (four real sites, reporting from inside each, including whether a cross-origin fetch got through).

Then the single-page workflows. Two of them drive a page the sample ships, loaded straight into the WebView with no network. Those are the only ones that can demonstrate the bridge at all, since AwaitMessage waits for window.vitre.postMessage and no third-party site will ever call it, and they double as the smoke test: a failure means the library broke rather than that a website was redesigned. The other two hit the live web to show the same steps against real pages, and are the ones that will eventually rot.

VERSION_NAME in gradle.properties is the released version, and the only place it is written down. Changing it on main is the release, so a release is a reviewable pull request like anything else, and what reaches Maven Central is the value someone approved:

 # gradle.properties
-VERSION_NAME=0.1.0
+VERSION_NAME=0.2.0

Merging that fires .github/workflows/release.yml, which runs the full CI suite against the merge commit, publishes all five library modules to Maven Central as one atomic deployment and releases it, with no button to press afterwards. Only then does it tag the commit v0.2.0 and open a GitHub release for it, with notes generated from the pull requests merged since the last one. The tag records a release that happened rather than causing one; pushing one by hand publishes nothing.

A version with a prerelease component (0.3.0-rc.1, 1.0.0-beta.2) publishes to Maven Central like any other, but its GitHub release is flagged so it does not show up as Latest.

gradle.properties also holds Kotlin, Gradle, Android and POM settings, so most edits to it are not releases. The workflow diffs VERSION_NAME against the previous commit and exits quietly when it has not moved. It refuses outright, before Gradle starts, on a malformed version, on the 0.0.0-LOCAL placeholder, and on any version already tagged, because a Maven Central version can be superseded but never withdrawn.

It publishes from macOS because the iOS targets need it: Kotlin/Native only builds Apple klibs on a Mac, and on Linux they are skipped silently rather than reported, which would ship module metadata advertising variants that are not there.

Publishing needs four secrets on the repository's maven-central environment: a Central Portal user token for the verified dev.ggoggam namespace, and a GPG key to sign with.

The public half has to be on a keyserver Central checks (gpg --keyserver keyserver.ubuntu.com --send-keys <key-id>), or validation rejects every signature.

To try the modules from another project without releasing anything, mise run publish:local puts them in ~/.m2 at whatever VERSION_NAME currently says.

Issues and pull requests are welcome.

mise install
mise run fmt:check      # ktlint, same check CI runs
mise run test

The repo uses prek pre-commit hooks (mise run pre-commit), and ktlint with the official Kotlin code style. A few conventions worth knowing before you send a patch:

• Comments explain why. Much of this codebase's value is in the reasoning recorded next to non-obvious decisions: platform seams, ordering guarantees, failure modes. Keep that up.
• Tests come first for anything in vitre-core. The engine, bridge and pool are covered by commonTest against a fake controller; a behaviour change should show up there before it shows up in a WebView.
• Both platforms or neither. If a change can only work on one, say so where a caller will read it, the way interception does.

MIT © 2026 Joon Kwon