handraise ✋

When your agent gets stuck on a Solari cloud browser, let it ask a human — then continue from the exact same session.

0901.2.mp4

npm install handraise

import { Solari } from "@solarisdk/browser"
import { raiseHand } from "handraise"

const browser = await new Solari({ apiKey: process.env.SOLARI_API_KEY }).launch()
const page = await browser.newPage()
await page.goto("https://github.com/login")
// ... the agent fills the credentials, then hits the 2FA wall ...

const result = await raiseHand(page, { reason: "GitHub is asking for a 2FA code" })
// outcome "resolved" → the human typed the code on their phone, the agent is signed in

A QR code appears in the terminal. Scan it, and the live browser session is on your phone — video, taps, typing, scrolling. Tap Hand back and raiseHand returns. Set SOLARI_API_KEY; handraise uses it to create the relay sandbox.

• Same browser session. Same cookies, same page, no restart — the agent resumes exactly where it stopped.
• Works from any phone, nothing to install there. The handoff link is a URL; open it in the mobile browser that is already on the device.
• No server to host. The handoff UI runs on a Solari sandbox that handraise creates and destroys around the call.

Measured against the live API, method and raw data in benchmarks/:

• 19/20 blocked workflows rescued (baseline 0/20).
• 30/30 handoffs resolved in the latency benchmark.
• 3.5 s median from raise to live on the phone.

Approval mode

Sometimes the agent is not stuck — it is about to do something it may not decide alone. That needs no takeover: the human sees a screenshot, the reason and the exact step, and answers.

const answer = await raiseHand(page, {
  mode: "approval",
  reason: "The agent may not move money without a human",
  action: "Submit $12,430 vendor payment to Acme GmbH",
})
if (answer.outcome !== "approved") return   // "denied", "timeout", "disconnected"

One screenshot, no live stream, and nothing is injected into the page: the agent still owns the session and carries out the action itself. (The event's framesSent counts that screenshot once per connection — 1 + reconnects — because a reconnecting agent has to put it back on the wire.) On the phone, Deny is one tap and Approve takes a 700ms hold — the reverse of takeover mode, because here the answer that cannot be taken back is yes. The relay enforces it too: an approval relay routes approve and deny and drops every takeover message, so the restriction is not just a hidden button (docs/adr/0006).

Upgrading from 0.3.0: nothing changes at runtime, and raiseHand(page, { reason }) compiles as it did. Three exported types changed shape, so a TypeScript consumer may have one edit to make even if they never ask for an approval — HandoffOutcome has two new members, HandoffEvent.mode is new and required, and RaiseHandOptions is now a union (extend HandoffOptions or TakeoverOptions instead of it). The CHANGELOG has the detail.

Channels

An approval is a screenshot, a sentence and two answers — which is a chat message. A channel is an object handraise notifies when the handoff starts; in approval mode it also gets the JPEG and can answer in-process, so nobody has to open the link at all.

import { raiseHand } from "handraise"
import { telegram } from "handraise-telegram"

const { TELEGRAM_BOT_TOKEN = "", TELEGRAM_CHAT_ID = "" } = process.env

await raiseHand(page, {
  mode: "approval",
  reason: "The agent may not move money without a human",
  action: "Submit $12,430 vendor payment to Acme GmbH",
  channels: [telegram({ botToken: TELEGRAM_BOT_TOKEN, chatId: TELEGRAM_CHAT_ID })],
})
// The screenshot and two buttons arrive in the chat; the first answer wins,
// whether it comes from there or from the phone.

Write your own in about ten lines: notify(handoff) gets handoffId, url, reason, mode, settled and — in approval mode — action, screenshot (the same bytes the phone shows) and answer("approve" | "deny"), which returns false if somebody was faster. notify is never awaited and whatever it throws is one channel_failed warning: a chat API that is down costs you a notification, not a browser session. Anyone who can see the channel can answer it (docs/adr/0007).

settled is how a channel knows it can stop. It is a promise that resolves with the outcome the moment the handoff ends — however it ended, including on the phone or by timeout — and it never rejects:

const channel = {
  notify: async (handoff) => {
    const message = await post(handoff)
    const outcome = await Promise.race([waitForReply(message), handoff.settled])
    await close(message, outcome)
  },
}

Without it an adapter that waits for a reply can only stop on its own clock, which means holding a connection open — and the process alive — long after raiseHand has returned.

Runnable without writing any code: demo/try.ts raises a hand immediately so you can drive it; demo/approval.ts asks you to approve a payment; demo/github-2fa.ts does the real 2FA wall and keeps the session the handoff earned.

The series

handraise is one of four packages built on the same measurements, each in its own repo with its own benchmark:

Package	What it does
handraise-telegram	The approval arrives in a Telegram chat as the screenshot with Approve/Deny; long-polled, no public endpoint.
handraise-slack	The same in a Slack channel over Socket Mode; no request URL to host.
outlive	Solari browser sessions end after ~600 s. outlive(solari, task) checkpoints, notices the death on the connection, relaunches and re-enters your task. Baseline 0/5, outlive 5/5.

Both adapters are HandoffChannel implementations of the hook above, live-tested with a person pressing the button. The cookbook example is browser-human-handoff-ts.

What this actually is

handraise is a resumable interrupt primitive for autonomous agents — the live view is just the implementation. The product is interrupt → human resolution → resume. A live view shows you a browser; handraise gives the agent a typed outcome it can branch on, and a session that survives the detour.

Give it to your LLM agent

Your agent's loop already knows when it's stuck. Expose handraise as a tool and let the model decide when to call for a human. No extra dependencies; the spec is plain JSON Schema:

import { tool, jsonSchema } from "ai" // Vercel AI SDK
import { createNeedHumanTool, needHumanToolSpec, type NeedHumanInput } from "handraise"

const needHuman = createNeedHumanTool(page)

const tools = {
  needHuman: tool({
    description: needHumanToolSpec.description,
    inputSchema: jsonSchema<NeedHumanInput>(needHumanToolSpec.inputSchema),
    execute: needHuman,
  }),
}

The tool returns { outcome, summary, durationMs }, where summary is a sentence the model can act on ("A human fixed the problem and handed the browser back. Re-read the page and continue.", or "The human refused the action. Do not carry it out and do not ask again for the same step.").

The model also chooses the mode: it passes mode: "approval" plus an action when it could do the step but must not decide alone, and the tool refuses an approval that names no action rather than quietly handing the browser over.

demo/agent.ts is a real agent loop where the model itself decides to call needHuman when it hits the 2FA wall — run it with DEMO_SIM=1 for the scripted version.

Two classes of interrupt, one call. A capability gap — 2FA, a captcha, an unfamiliar UI — is the agent admitting it cannot: mode: "takeover", the default. An authority boundary — a yes before an irreversible step — is the agent not being allowed to: mode: "approval". The model picks, by passing mode and action to the same tool; the tool description says which is which.

How it works

flowchart LR
  A["Agent process<br/>raiseHand(page)"] -- "CDP screencast frames →<br/>← taps & keystrokes" --> R["Relay<br/>(Solari sandbox,<br/>public preview URL)"]
  R <--> P["Your phone<br/>(just a browser tab)"]

Loading

The twist: the handoff UI itself runs on Solari. When the agent raises its hand, handraise boots a Solari sandbox (~3s), deploys a zero-dependency relay into it, and exposes it through Solari's port preview. Frames stream from the browser's CDP screencast through the relay to your phone; your taps and keystrokes stream back and are injected as trusted CDP input events. No tunnel tool, no self-hosted server, no second account — the same API key that runs your browser runs the escape hatch. The phone's end of it is a tokenized URL served from *.preview.getsolari.com, and nothing else.

What the human sees on the phone

The handoff link opens a dark, minimal page: the live browser session on top, an input bar at the bottom. Tap the live view to click, drag to scroll. When the agent reports which field has focus, the view zooms to it so it is readable (remote 16px text renders at ~10px instead of ~5px), draws a ring around it, and the bar names it ("Typing into: Verification code"). Pinch to zoom and pan yourself; double-tap toggles between zoomed and fit. Typing goes straight into the focused field, character by character — and if that field is a one-time code, the phone offers the SMS code it just received.

Under the input, four keys a phone's virtual keyboard cannot be trusted to send, and one that asks the agent a question about the page:

Key	What it does
⌫	Delete one character in the remote field
⇥	Move to the next field
⏎	Submit / press Enter
Clear	Empty the focused field (select-all + backspace; disabled while nothing is focused, and kept well away from ⌫)
Scan QR	Read the QR codes on the page and show what they say

QR codes on the page

Some walls ask for a second device: reCAPTCHA's "scan to verify", a WhatsApp Web login, an authenticator enrolment. The human is holding the phone the site wants — and the code is on that phone's screen, so it cannot be scanned.

Scan QR asks the agent instead. It takes a fresh full-resolution screenshot of the page, decodes it, and sends back what each code said. The phone shows the link in full and offers Open in new tab — so the link is opened on the phone, which is the device the site was asking for. Takeover mode only, one scan per 2 seconds, and a symbol below about 120 CSS pixels will not decode — scroll or zoom the remote page and scan again.

The code came off a page nobody vetted, so:

• Only http, https and mailto: get an Open button. Everything else — javascript:, data:, blob:, content:, and anything carrying an invisible character — is shown as text with a Copy button and no link. The agent classifies it and the phone applies the same rule again, because the handoff URL is a bearer credential and the socket behind it takes messages from anyone holding it.
• tel: and otpauth: are shown and copyable, never opened. A tel: code can carry a dialler control sequence, and an otpauth: code enrols a TOTP secret in your authenticator. Both are one tap and hard to take back, so the sheet names them ("Phone number", "Authenticator secret") and you hand them to the right app yourself.
• An openable link is shown as the address it opens, with the host as the loud part of it. https://аpple.com with a Cyrillic а reads as apple.com and goes to xn--pple-43d.com; the sheet shows the second one and says the code wrote it differently.
• The agent process never fetches any of it, and never decodes on its own event loop: the decode runs on a worker thread, so a handoff stays answerable while it happens.

Measured: docs/measurements/05-qr.md; the decisions are in ADR 0008.

Below that, two ways out. ✋ Hand back ends the handoff as resolved — one tap, the agent continues. I can't do this ends it as aborted — the agent is told a human looked and could not solve it, so it should not retry the same step; it takes a 700ms hold, because it is irreversible and sits next to the primary. The dot in the header shows the connection: white is live, pulsing grey is reconnecting (your input is queued and sent in order once it is back), red means the handoff has ended.

In approval mode the same page has a different job. One screenshot instead of the stream, the action in the largest type on the screen, and no keyboard, key bar or input row — nothing there can reach the remote page. Pinch, drag and double-tap still zoom and pan the screenshot, because an amount you cannot read is an approval you cannot give. Deny is one tap and Hold to approve takes the 700ms; the ending says which one happened.

Getting notified

Four ways, no vendor lock-in:

• QR code in the terminal (default) — scan with the phone camera.
• onUrl callback — do whatever you want with the link.
• webhookUrl — handraise POSTs { url, reason, mode, action?, sessionId } as JSON (action only in approval mode). Point it at Slack, Discord, ntfy, a Telegram bot — anything that accepts a POST.
• channels — the only one that can carry the screenshot and bring an answer back. See Channels.

await raiseHand(page, {
  reason: "Captcha needs a human",
  webhookUrl: process.env.SLACK_WEBHOOK_URL,
  qr: false,
})

API

raiseHand(page, options): Promise<HandoffResult>

Option	Type	Default
reason	string	required	Shown to the human on the handoff page.
mode	"takeover" | "approval"	"takeover"	takeover hands the live browser over; approval shows one screenshot and asks for a yes or a no.
action	string	required in approval mode	The exact step being decided, e.g. "Submit $12,430 vendor payment to Acme GmbH". A type error if mode is "approval" and it is missing.
timeoutMs	number	5 minutes	How long to wait for the human.
humanGoneGraceMs	number	60s	How long to keep waiting after the human's phone disappears. A handoff nobody ever opened is unaffected and waits out timeoutMs. Range 5000–2147483647, and the floor is a floor rather than a safe value — the default is what covers a proxy cut plus the phone's reconnect. Raise it if the human is expected to leave the page: a locked screen can lose the socket.
webhookUrl	string	—	Generic JSON POST when the link is ready.
onUrl	(url) => void	—	Called with the handoff URL.
channels	HandoffChannel[]	—	Where else to announce it. In approval mode a channel also gets the screenshot and can answer. See Channels.
qr	boolean	true	Print a QR code to the terminal.
apiKey	string	$SOLARI_API_KEY	Solari key used to create the relay sandbox.
logger	Logger	warn/error only	Structured logging sink. Pass consoleLogger for full JSON lines incl. the per-handoff wide event.
onEvent	(e: HandoffEvent) => void	—	One wide event per handoff (outcome, timings, ids).
baseUrl	string	—	Solari endpoint/region for the relay sandbox.

HandoffResult

Field
outcome	See below.
durationMs	Wall-clock time the human took.
url	The handoff URL.
storageState	Cookies + localStorage captured right after a successful handback — persist it (e.g. to a Solari profile) and the human's work survives even if the session dies later. Takeover mode only: an approval changes nothing on the page.

outcome	Mode	Means
resolved	takeover	The human handed the browser back.
aborted	takeover	The human looked and could not solve it. Do not retry the same step.
approved	approval	Carry out the action.
denied	approval	Do not carry out the action.
timeout	both	Nobody answered — either timeoutMs ran out, or a human who was there closed the tab and stayed away for humanGoneGraceMs. The wide event's endedEarly and humanSeen say which.
disconnected	both	The browser session died mid-handoff.

scanQrLinks(png): ScannedLink[]

The decoder behind the phone's Scan QR button, exported so an agent can read a code without asking a human. Takes the bytes of a PNG screenshot, returns up to two { text, kind } — kind: "url" only for a scheme in OPENABLE_SCHEMES, which is also exported. It reads and classifies; it never opens anything.

import { scanQrLinks } from "handraise"

const codes = scanQrLinks(await page.screenshot({ type: "png" }))
if (codes[0]?.kind === "url") console.log(codes[0].text)

Errors

raiseHand throws only before the handoff URL exists — while nobody has been asked for anything yet. Everything after that is an outcome, never an exception. What it throws is a HandraiseError with a code: the code is the contract, the message is for whoever reads the log and may be reworded in any release. isHandraiseError narrows a catch binding, and cause keeps the original SDK, CDP or network error whenever there was one — the same class, name, status and code, its own non-enumerable properties, and its own cause chain — with credentials redacted out of every message, stack and response body along it. Every error serialiser prints the whole chain, so a clean outer message on its own would not be worth much.

The first thing raiseHand does is look at your page, before it creates anything: a page you have closed, or a browser you have disconnected, is refused as browser_unusable rather than paid for with a relay sandbox and a person's attention. It reads local state only, so a Solari session that has died server-side while the CDP socket is still open still looks alive — that one arrives as the disconnected outcome, as it always did.

import { isHandraiseError, raiseHand } from "handraise"

try {
  await raiseHand(page, { reason: "GitHub is asking for a 2FA code" })
} catch (error) {
  if (isHandraiseError(error) && error.code === "concurrency_limit") {
    // one Solari session too many: free one, then call again
  }
}

code	Happens when	What to do
missing_api_key	No options.apiKey and no SOLARI_API_KEY.	Set one; handraise needs it to create the relay sandbox.
invalid_mode	mode is neither "takeover" nor "approval".	Fix the call. TypeScript already refuses it; this is for JavaScript callers.
empty_action	mode: "approval" without a non-empty action.	Name the step the human says yes or no to.
invalid_option	An option is present but unusable. Today: humanGoneGraceMs outside 5000–2147483647 ms (below that the phone's own reconnect ends handoffs; above it a Node timer collapses to 1 ms).	Fix the value. Refused rather than clamped, because a grace that is silently something else ends handoffs you did not expect to end.
browser_unusable	The page is closed, or its browser has disconnected — checked before anything is created.	Open a new page or relaunch the session (restore storageState if you kept it) and retry.
relay_start_failed	The relay sandbox could not be created or deployed.	Read cause — it is the Solari SDK's own error, redacted. Retry. Nothing is left behind unless you also see relay_release_failed (below).
concurrency_limit	Your Solari account is at its concurrent session cap (429).	Free a session, or wait and retry. The one relay failure that is purely temporary.
relay_not_ready	The sandbox started but its public URL never answered.	Retry. Persisting means the preview proxy or the region is unhealthy.

There is deliberately no code for the one failure that is not the caller's to catch: a relay sandbox that survives its own teardown. raiseHand logs relay_release_failed and carries on — after a successful handoff it returns the outcome, and on a failed start it still throws relay_start_failed. Either way that sandbox's public URL stays reachable until its idle timeout, so watch for that log line and delete it from the Solari dashboard.

What happens when things die

A handoff tool that loses your session at the worst moment is worse than no tool, so every row here is measured rather than hoped for.

Failure	Outcome
Human never shows	Clean timeout after timeoutMs, relay destroyed
Human opens it, then closes the tab	timeout one grace later, not five minutes later — the relay reports the phone's socket, and the event says endedEarly: true
Browser session dies mid-handoff	disconnected, not an exception
Relay WebSocket drops	20s heartbeats, reconnect, last frame replayed
Agent process killed	Sandbox lifecycle kill, no orphaned URL
Link holder tries the agent role	401, roles are separate credentials

The load-bearing fact is the platform's: Solari browser sessions die ~10 minutes after creation whatever you do, and the sessions API still calls a dead one active (we measured it) — hence the 5-minute default, no keep-alive pinger, and storageState on the result. Every exit path, errors included, destroys the relay sandbox before raiseHand returns: one handoff, one sandbox. The rejected alternatives are in docs/adr/.

How this compares

Human-in-the-loop for cloud browsers isn't new — that's the point, the demand is proven. Browserbase Live View, Cloudflare Browser Run, Scrapfly and AuthLoop are hosted platform features of their own clouds; if you run on one, use theirs. handraise brings the same handoff to Solari browsers, which have no native live view (Solari's VNC is desktop-only), as a portable library instead — less polished, and it works where those don't. What the hosted live views do not have is the second mode: an approval is a yes-or-no on one screenshot, no live session exposed at all, answerable from a chat channel. Nor do they have an answer to a device-change check — a QR code a phone is asked to scan, on the phone's own screen — which handraise reads off the page and hands over as a link. Its scope stops at the handoff, not wall detection (docs/adr/0005).

Security

• The handoff link is a bearer URL scoped to one relay: a preview token for that one sandbox and port, 1-hour lifetime, destroyed with the sandbox.
• The agent role is a separate secret, never in the link. Only its holder can read keystrokes and drive the browser; a foreign Origin is refused.
• Frames and keystrokes are never persisted, and the human side speaks a closed, length- and rate-bounded message set — there is no path from the link to arbitrary browser control. Your API key never leaves the agent process.

Threat model, scope and reporting: SECURITY.md.

Measured

bun run bench: 30 consecutive real handoffs against the live API on the $20 Solari plan, one fresh relay sandbox each, a scripted human on the public WebSocket, measured from Germany against the default (us-west) endpoint. 30/30 resolved, zero reconnects, zero leaked sandboxes.

	p50	p75	worst of 30
Agent raises its hand → the phone shows the live page	3.5s	3.6s	3.7s
— of which: relay sandbox cold start	2.7s	2.7s	2.9s
Input round trip through the relay (150 samples)	186ms	191ms	286ms

The number that matters more than any latency is what handraise does to workflows that would otherwise fail. bun run bench:rescue: 40 runs against a live portal with a real TOTP wall, interleaved arms, one completion test:

	completed	median human time
baseline agent (no human available)	0/20	—
with handraise	19/20	5.5s

The 0/20 baseline is the design fact, not a crippled agent: it tried, and a machine cannot know a TOTP code. The 5.5s is a scripted human — the machine floor of a handoff, not reading speed. The one failure was the platform's ~10min session death landing mid-handoff; handraise reported disconnected instead of claiming success.

Two interrupts do not cost the same, and bun run bench:mixed measures them side by side: 20 workflows against one live portal, interleaved takeover, approval, takeover, on 2026-09-02. A takeover needs the browser driven; an approval needs one decision, and every fourth one here was a denial.

	completed	to visible	frames	bytes	relay sandbox
takeover — the human drives	10/10	4718ms	14	142 KB	10.7s
approval — the human decides	10/10	4896ms	1	25 KB	5.3s

Medians over the completed runs. A denied approval counts as completed: the decision was delivered, and the bench then loads the account page and requires both the session and the absence of a receipt for that run's amount. to visible includes the relay cold start (~3 s) that both modes pay. The relay seconds include the human's time, so they are a floor — a real person takes longer than this scripted one — and only the takeover pays that cost: an approval injects nothing and sends one screenshot however long the human thinks. The approval arm signs itself in with the shared secret, so only the takeover arm is measured against a wall it cannot pass, and the 50/50 mix is the harness's choice rather than a measurement of anyone's traffic: method and caveats.

At N=30 the right-hand column is the worst observation, not a fitted p99 — we say what we measured. The input round trip sits on the network RTT floor from Germany to the us-west edge (pass baseUrl to co-locate the relay with your region), and cold start is ~75% of time-to-visible, which is why a warm relay is the next performance lever. Live view costs 23–80 KB/s while a human solves a 2FA, and each handoff consumes one sandbox, destroyed when it ends.

Verified how

Benchmark method and raw data: benchmarks/. The five platform measurements the design rests on — transport, screencast, input injection, session lifetime, QR decoding — are in docs/measurements/. The e2e test drives the whole loop with no mocks: a Solari browser signs into a TOTP-protected demo app (test-app/, deployed into a sandbox), hits the 2FA wall, raises its hand, a scripted "human" types the code through the real handoff UI, and the test asserts the signed-in page — ~6s end to end. The same run then drives the QR passthrough: the app shows a device-change code, the human asks for a scan, and the link that comes back is fetched from outside the browser to reach the confirmation page. Injected events arrive with isTrusted: true.

Limitations (v1)

• Solari's $20 plan allows 2 concurrent sandboxes; each active handoff uses one. Two simultaneous handoffs is the plan-tier ceiling.
• An approval shows the page as it was when the agent asked. If the page changes underneath (a session expiring, a redirect), the human is deciding on a stale picture — the frame is not refreshed.
• The QR passthrough is untested against reCAPTCHA itself: its demo never served the scan-to-verify variant, which Google shows at its own discretion. The mechanism is proven end to end against a page that behaves the same way (measurement 05 §7). A code drawn below ~120 CSS pixels does not decode, and three or more codes on one screen are not attempted.
• TypeScript/Node only for now.

Contributing

Small, focused PRs welcome. Good first issues: a Python port, a needHuman tool export for more agent frameworks, wall-detection heuristics.

MIT