Runspace

Runspace is an experiment in giving AI agents a concrete working environment for client-specific knowledge.

The core idea is simple: instead of asking an agent to reason through scattered APIs, databases, inboxes, documents, transcripts, notes, and memories one tool call at a time, we materialize a focused workspace for the agent. That workspace contains the relevant information for one client, household, matter, account, or relationship, organized as ordinary files the agent can inspect, search, cite, update, and reason over.

This repository explores that idea.

The Thesis

AI agents become more useful when they have a place to work.

Most agent systems are built around tools: query the database, call the email provider, search the document store, retrieve from a vector index, update the CRM, send a message, create a task. That model is powerful, but it can also make the agent's world fragmented. The agent must continuously ask for slices of context, remember what it has seen, reconcile conflicting results, and maintain working state inside a limited conversation window.

Runspace explores a different layer: a structured, client-specific workspace.

The workspace is not meant to replace a database, CRM, document system, compliance archive, or source-of-truth application. It is meant to be an agent-operating layer: a generated case file that gives the agent a coherent environment for reasoning and work.

In a Runspace-style system, the platform gathers information from canonical systems and materializes it into a folder. The folder might contain:

• client metadata as JSON
• household and relationship data
• meeting transcripts as Markdown
• meeting notes as Markdown
• email threads as Markdown
• document summaries as Markdown
• source manifests and provenance records
• open tasks and unresolved questions
• durable memories about the client
• draft artifacts created by the agent
• audit logs of what the agent read, wrote, and concluded

The agent can then use simple, robust tools such as file reading, search, shell commands, and constrained write operations to inspect and operate on the workspace.

The Experiment

Runspace is an exploration of whether a filesystem-like workspace can become a practical substrate for agentic work over sensitive, client-specific information.

The experiment has several parts.

1. Workspace as Context

Can a folder of well-structured Markdown and JSON files give an agent enough context to perform useful work?

Instead of dumping everything into a prompt or hiding everything behind retrieval APIs, Runspace treats the workspace as a navigable environment. The agent can inspect a manifest, search relevant files, read original sources, and produce answers with citations.

2. Workspace as Memory

Can durable memories be stored as human-readable files?

The agent may extract stable facts, preferences, goals, relationships, unresolved issues, and historical context into memory files. These memories should be useful across future sessions, but they must remain inspectable, correctable, and traceable to sources.

Runspace is especially interested in memory that includes:

• provenance
• confidence
• timestamps
• sensitivity labels
• review status
• conflicts
• expiration or staleness rules

The goal is not to let the agent freely write permanent truth. The goal is to explore how memories can move from raw observation to candidate memory, reviewed memory, and durable memory.

3. Workspace as an Audit Surface

Can agent work become easier to inspect?

If the agent reads files, writes artifacts, updates memories, and logs its actions inside a workspace, then humans can inspect what happened. This matters for debugging, trust, compliance, and evaluation.

Runspace treats explainability as a product requirement. Answers should be sourced. Generated artifacts should show what they relied on. Memory updates should explain why they were created.

4. Workspace as a Sandbox Boundary

Can a workspace become the unit of isolation?

A local folder is useful for early experimentation, but the more powerful version is a remote sandbox: a temporary, isolated runtime where the agent can search files, run analysis, write drafts, and create structured outputs without broad access to production systems.

Runspace explores both shapes:

• local workspace for fast prototyping
• remote sandbox workspace for stronger isolation, auditability, and permission control

5. Workspace as a Bridge Between Systems

Can canonical systems stay canonical while agents still get a rich working environment?

Runspace does not assume that Markdown files should become the system of record. The more realistic architecture is:

Canonical systems -> generated workspace -> agent work -> reviewed outputs -> canonical systems

The workspace is a materialized view for agent work. It can be rebuilt, snapshotted, diffed, archived, or discarded.

The Implementation

This repository contains a working prototype of the experiment, built with Bun, the AI SDK v6 (ToolLoopAgent), and the OpenAI provider.

src/
  agent.ts        ToolLoopAgent wired to a workspace + skills
  tools.ts        readFile, writeFile, listFiles, bash, skill (all sandboxed to the workspace)
  workspace.ts    Workspace sandbox: path resolution that refuses escapes
  skills.ts       SKILL.md loader (frontmatter name/description + body)
  prompt.ts       System prompt with the skills index embedded
  cli.ts          Run the agent against a workspace from the terminal
skills/           Instruction packs the agent loads on demand via the skill tool
  workspace-navigation/     Layout, manifest, source hierarchy, recency rules
  answering-with-citations/ Grounding, [source: path:line] citations, conflicts
  memory-management/        Candidate memory format: provenance, confidence, sensitivity
  drafting-artifacts/       Artifact structure with source manifests
  audit-logging/            Human-readable work log in logs/agent-log.md
workspaces/
  carter-household/         Simulated client workspace (wealth-management household)
evals/
  cases.ts        Five eval cases targeting planted facts in the workspace
  checks.ts       Deterministic checks (regexes, file assertions, source integrity)
  judge.ts        LLM-as-judge scoring (groundedness, correctness, instruction-following)
  run.ts          Runner: pristine workspace copy per case, diff, score, report

Key design points:

• The agent's system prompt only lists skill names and descriptions; full skill bodies are loaded on demand with the skill tool (progressive disclosure).
• Every tool call is automatically appended to logs/tool-calls.jsonl inside the workspace, and the agent is instructed to keep a human-readable logs/agent-log.md — the workspace is the audit surface.
• File tools are sandboxed: paths resolving outside the workspace are rejected, and bash runs with the workspace as its working directory.
• The simulated workspace contains planted dynamics on purpose: a stale client/profile.json risk tolerance that a newer transcript supersedes, a stale durable memory, a contribution figure that evolved from "considering $1,000/month" to "committed $1,500/month", and a 401(k) balance that is genuinely absent — so evals can measure conflict handling and refusal to fabricate.

Getting Started

bun install
cp .env.example .env   # add your OPENAI_API_KEY

Run the agent against the simulated workspace:

bun run agent -- "What is the household's current risk tolerance?"
bun run agent -- --workspace workspaces/carter-household "Prepare a meeting prep brief for the June 24 check-in and save it in artifacts/."

Run the evals:

bun run eval                       # all cases (deterministic checks + LLM judge)
bun run eval -- --case fact-529-contribution
bun run eval -- --skip-judge       # deterministic checks only

Each eval case copies the pristine workspace into .runs/<run-id>/<case>/workspace, runs the agent, diffs the workspace to see what it wrote, runs deterministic checks, and scores the result with an LLM judge. A JSON report is written to .runs/<run-id>/report.json.

The eval cases map to the experiment's hypotheses:

Case	Hypothesis tested
fact-529-contribution	Workspace as context: exact figures with line-level citations
conflict-risk-tolerance	Source hierarchy + recency: stale JSON vs newer transcript
missing-helix-balance	Refusing to fabricate what the workspace does not contain
artifact-meeting-prep	Workspace as audit surface: sourced artifacts + work log
memory-extraction	Workspace as memory: reviewable candidates with provenance

Configuration lives in src/config.ts; override models with RUNSPACE_MODEL and RUNSPACE_JUDGE_MODEL (defaults: gpt-5.2).