Put a language model in a web app without running an inference backend. It runs on the user's GPU in the browser tab, or proxies to a cloud model, and the token stream survives reloads and dropped connections

@inbrowser is six composable libraries:

1. @inbrowser/model - Run a language model on the user's GPU or through a cloud provider, and switch between them with a one-line change.
2. @inbrowser/agent - Let the model use your tools and take several steps to finish a task, right in the browser.
3. @inbrowser/resumable - Stream a long answer so a reload, a closed tab, or a dropped connection picks up where it left off instead of starting over.
4. @inbrowser/relay - Add a thin server when you'd rather keep API keys off the client or share one run across devices.
5. @inbrowser/workspace - Run files, shell commands, package installs, git, snapshots, and preview compilation in a browser workspace.
6. @inbrowser/sandbox - Bind workspace capabilities into agent-facing tools, events, checkpoints, and artifacts.

On-device models don't need API keys. Cloud models use BYOK (bring your own key): each user supplies their own, and it stays on the client instead of on a server you run.

import { createEngine, createEngineModelClient, smollm2_360m } from '@inbrowser/model';

const engine = createEngine({
  ...smollm2_360m,
  onLoadProgress: (p) => {
    if (p.phase === 'fetch') {
      const pct = Math.round((p.loadedBytes / p.totalBytes) * 100);
      console.log(`downloading ${p.file}: ${pct}%`);
    } else {
      console.log(p.phase); // 'init' | 'warmup' | 'ready'
    }
  },
});

await engine.ensureReady(); // downloads ~180 MB once, then caches it

const client = createEngineModelClient(engine);

for await (const ev of client.chat({ 
  messages: [{ 
    role: 'user',
    text: 'Explain quantum tunneling in two sentences.' 
  }], 
  tools: [], 
  toolUseEnabled: false 
}, AbortSignal.timeout(60_000))) {
  if (ev.kind === 'text') console.log(ev.text);
}

The weights download once via Transformers.js and run on WebGPU (WASM fallback when there's no GPU). createEngineModelClient wraps the engine as a ModelClient — the same interface every cloud provider implements, so swapping it for geminiModelClient({ apiKey, model }) (or openrouterModelClient, requestyModelClient, ollamaModelClient, …) changes that one line and nothing else.

Cloud models need a key, and there is no server to keep one on, so the user brings it. For OpenRouter that can be a one-click connect instead of a pasted key:

import { beginOpenRouterOAuth, completeOpenRouterOAuth, openrouterModelClient } from '@inbrowser/model';

// 1. Send the user to OpenRouter to authorize (full-page redirect or popup).
const { authUrl, codeVerifier } = await beginOpenRouterOAuth({ callbackUrl: location.href });
sessionStorage.setItem('openrouter_verifier', codeVerifier);
location.href = authUrl;

// 2. Back on your callback page, exchange the ?code for the user's own key.
const code = new URLSearchParams(location.search).get('code')!;
const { key } = await completeOpenRouterOAuth({
  code,
  codeVerifier: sessionStorage.getItem('openrouter_verifier')!,
});

// The key belongs to the user, provisioned in their browser. Nothing on your server.
const client = openrouterModelClient({ apiKey: key, model: 'anthropic/claude-3.5-sonnet' });

The user clicks Connect OpenRouter and authorizes. Your app receives a key tied to their OpenRouter account, so usage is billed to them and they can revoke it whenever they want. PKCE means there is no client secret, so the exchange runs entirely in the browser.

import {
  createAgentSession,
  createReactLoopStrategy,
  createToolRegistry,
  createDispatch,
  createMetricsCollector,
} from '@inbrowser/agent';
import { openrouterModelClient } from '@inbrowser/model';

// A tool is a plain object: name, description, JSON-schema params, and execute().
const getWeather = {
  name: 'get_weather',
  description: 'Current temperature for a city.',
  parameters: { type: 'object', properties: { city: { type: 'string' } }, required: ['city'] },
  async execute({ city }: { city: string }) {
    const r = await fetch(`https://wttr.in/${encodeURIComponent(city)}?format=%t`);
    return { ok: true, summary: `${city}: ${await r.text()}` };
  },
};

const registry = createToolRegistry();
registry.register(getWeather);

const session = createAgentSession({
  strategy: createReactLoopStrategy(),
  llm: openrouterModelClient({ apiKey: '<BYOK>', model: 'z-ai/glm-5.2' }),
  tools: createDispatch(registry),
  toolList: registry.list(),
  toolContext: () => ({ signal: new AbortController().signal }),
  systemPromptBuilder: () => 'Answer using the tools when they help.',
  metrics: createMetricsCollector(),
  history: [],
});

const events = session.submit('Is it jacket weather in Oslo?', new AbortController().signal);
for await (const event of events) {
  if (event.kind === 'text') console.log(event.chunk);
}

A ReAct loop needs a model that can call tools, so this uses a cloud client. Running on-device? Most small presets can't drive a tool loop. Pair smollm2_360m with createRetrievalStrategy() to ground answers in your own documents instead, or step up to qwen2_5_0_5b, the smallest preset that advertises tool calling. The presets table below marks which is which.

import { createJobEngine, createIdbJobStore } from '@inbrowser/resumable';

type Token = { text: string };

// IndexedDB-backed: the event log persists across page reloads.
const engine = createJobEngine<Token>({ 
  store: createIdbJobStore<Token>() 
});

// The producer keeps running even if the tab navigates away.
const { jobId } = await engine.start(async function* () {
  for (const word of ['Durable ', 'by ', 'default.']) yield { text: word };
});

// Reconnect with the last seq you saw — only newer events replay.
for await (const ev of engine.subscribe(jobId, { from: 0 })) {
  if (ev.kind === 'event') console.log(ev.seq, ev.value.text);
  else if (ev.kind === 'terminal') console.log('status:', ev.status);
}

The log is append-only and ordered by seq. A consumer that reconnects after a refresh passes the last seq it saw and gets only what it missed. There are no duplicates, no lost tokens. The store is swappable: createMemoryJobStore() for a single process, createIdbJobStore() for the browser, createRtdbJobStore() to share a job across machines.

The shared ModelClient contract that relay and agent both consume, a set of cloud provider factories, and an on-device engine (Transformers.js + ONNX Runtime Web). Single root entrypoint — import { … } from '@inbrowser/model'.

Cloud providers — each is a factory that returns a ModelClient from { apiKey, model, … }:

openrouterModelClient also exports beginOpenRouterOAuth / completeOpenRouterOAuth for PKCE browser auth (no key on a server).

On-device engine — the lower-level API under createEngineModelClient. Use it directly when you want raw token events rather than the ModelClient contract:

import { createEngine, smollm2_360m } from '@inbrowser/model';

const engine = createEngine({ ...smollm2_360m });
await engine.ensureReady();

for await (const event of engine.generate([{ role: 'user', text: 'Hi' }])) {
  if (event.kind === 'token') console.log(event.text);
}

Bundled presets (all q4f16). "Tools" marks presets that advertise native tool calling. The rest can still retrieve and answer, but can't drive a ReAct loop:

Author your own with definePreset({ model: { modelId }, dtype, backend, capabilities }).

A backend-agnostic resumable streaming-job engine. Producers write typed events into a durable ordered log; subscribers tail it from any offset. Single root entrypoint.

createJobEngine options:

The RTDB store also exports serviceAccountTokenProvider and staticTokenProvider for auth.

Wires @inbrowser/resumable to the ModelClient contract: routes a provider request, streams the model's events into the durable log, and serves SSE. Use it when you do want a server in the loop (server-managed keys, shared jobs). Single root entrypoint.

import { createRelay } from '@inbrowser/relay';
import { openrouterModelClient } from '@inbrowser/model';
import { createMemoryJobStore } from '@inbrowser/resumable';

const relay = createRelay({
  store: createMemoryJobStore(),
  providers: { openrouter: openrouterModelClient },
});

// Start an inference job.
const res = await relay.handleStart(new Request('http://localhost', {
  method: 'POST',
  body: JSON.stringify({
    provider: 'openrouter',
    model: 'anthropic/claude-3.5-sonnet',
    messages: [{ role: 'user', text: 'Write a haiku' }],
    apiKey: '<BYOK>',
  }),
}));
const { jobId } = await res.json();

// Stream as SSE. Reconnect after a drop by passing the last seq you saw.
const stream = await relay.handleStream(new Request('http://localhost'), { jobId, from: 0 });

createRelay options:

A browser-safe agent runtime plus a Node CLI. The runtime (session, strategies, tools, metrics) is the root entrypoint; Node-only and CLI code live behind /node and /cli subpaths so they never reach the browser bundle. See "Give the model tools" above for a full session.

CLI commands (agent):

Any command emits structured output with the global --output json (-o json) flag, or when piped to a non-TTY.

Session events (SessionEvent kind): turn_started, text, thinking, tool_started, tool_finished, workspace_changed, runtime_changed, turn_completed, error, completed, strategy_event.

Packages are published independently. Install what you need:

bun add @inbrowser/resumable    # resumable streaming-job engine
bun add @inbrowser/model        # model contract + providers + on-device engine
bun add @inbrowser/relay        # LLM relay (depends on resumable + model)
bun add @inbrowser/agent        # agent runtime + CLI
bun add @inbrowser/workspace    # browser workspace: files, shell, preview, git
bun add @inbrowser/sandbox      # tools, events, checkpoints, artifacts

The on-device engine needs Transformers.js as a peer dependency:

bun add @huggingface/transformers

bun install
bun run build          # builds all packages in dependency order
bun run typecheck      # type-checks all packages
bun run test           # runs all package tests
bun run check          # biome lint + format

Filter to one workspace:

bun --filter '@inbrowser/agent' run test

@inbrowser/resumable    no internal deps
@inbrowser/model        no internal deps
@inbrowser/relay        depends on resumable + model
@inbrowser/workspace    no internal deps
@inbrowser/sandbox      depends on workspace
@inbrowser/agent        depends on model + sandbox

Pre-1.0. Versions are coordinated manually:

Breaking changes are expected until 1.0.