Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,22 @@ For prerelease channels, tarball installs, authenticated GitHub Release installs

This dogfood bundle uses VHS as the outer camera for real Codex and Claude interactive TUIs while each agent explores the `agent-tty` skill/CLI, drives `nvim --clean`, writes a file, and exports inner proof artifacts.

| Codex | Claude |
| -------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| [![Codex agent-tty demo](./dogfood/agent-uses-agent-tty/artifacts/codex-thumbnail.png)](./dogfood/agent-uses-agent-tty/artifacts/codex-outer.webm) | [![Claude agent-tty demo](./dogfood/agent-uses-agent-tty/artifacts/claude-thumbnail.png)](./dogfood/agent-uses-agent-tty/artifacts/claude-outer.webm) |
<table>
<tr>
<th width="50%">Codex</th>
<th width="50%">Claude</th>
</tr>
<tr>
<td>
<video src="https://github.com/user-attachments/assets/27cc3b9b-9b91-4cd9-a3a5-1bbb61c33e19" controls width="100%"></video>
</td>
<td>
<video src="https://github.com/user-attachments/assets/36221ef7-97c4-4b06-b673-21ac623a5f0a" controls width="100%"></video>
</td>
</tr>
</table>

GitHub renders these as inline H.264 MP4 video players. See [`VIDEO_PLAYBACK.md`](./dogfood/agent-uses-agent-tty/VIDEO_PLAYBACK.md) for the upload flow that produces the `user-attachments` URLs; the checked-in WebM proof files remain the canonical source of truth.

See [`dogfood/agent-uses-agent-tty/`](./dogfood/agent-uses-agent-tty/) for the Hero Demo reproducer, outer transcripts, inner Neovim recordings, and final file proofs.

Expand Down
26 changes: 22 additions & 4 deletions dogfood/agent-uses-agent-tty/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,27 @@
This bundle is the README-facing **Hero Demo** for real coding-agent TUIs using `agent-tty`.
VHS records the outer Codex and Claude Code TUIs as the presentation layer. The product proof is the inner `agent-tty` artifact set produced while each real agent explores the skill and CLI, drives Neovim, and exports recordings.

| Agent | Outer Hero Demo | Inner proof artifacts | File proof |
| ------ | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------ |
| Codex | [![Codex Hero Demo](./artifacts/codex-thumbnail.png)](./artifacts/codex-outer.webm) | [cast](./artifacts/codex-inner-nvim.cast), [WebM](./artifacts/codex-inner-nvim.webm) | [proof](./artifacts/codex-final-file-proof.txt) |
| Claude | [![Claude Hero Demo](./artifacts/claude-thumbnail.png)](./artifacts/claude-outer.webm) | [cast](./artifacts/claude-inner-nvim.cast), [WebM](./artifacts/claude-inner-nvim.webm) | [proof](./artifacts/claude-final-file-proof.txt) |
The Outer Hero Demo column embeds the uploaded H.264 MP4 recordings as inline GitHub video players; see [VIDEO_PLAYBACK.md](./VIDEO_PLAYBACK.md) for the upload flow. The checked-in WebM files remain the canonical proof artifacts.

<table>
<tr>
<th>Agent</th>
<th>Outer Hero Demo</th>
<th>Inner proof artifacts</th>
<th>File proof</th>
</tr>
<tr>
<td>Codex</td>
<td><video src="https://github.com/user-attachments/assets/27cc3b9b-9b91-4cd9-a3a5-1bbb61c33e19" controls width="320"></video></td>
<td><a href="./artifacts/codex-inner-nvim.cast">cast</a>, <a href="./artifacts/codex-inner-nvim.webm">WebM</a></td>
<td><a href="./artifacts/codex-final-file-proof.txt">proof</a></td>
</tr>
<tr>
<td>Claude</td>
<td><video src="https://github.com/user-attachments/assets/36221ef7-97c4-4b06-b673-21ac623a5f0a" controls width="320"></video></td>
<td><a href="./artifacts/claude-inner-nvim.cast">cast</a>, <a href="./artifacts/claude-inner-nvim.webm">WebM</a></td>
<td><a href="./artifacts/claude-final-file-proof.txt">proof</a></td>
</tr>
</table>

See [promoted-run-summary.md](./promoted-run-summary.md) for the regeneration summary.
104 changes: 104 additions & 0 deletions dogfood/agent-uses-agent-tty/VIDEO_PLAYBACK.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Hero Demo GitHub video playback

The canonical Hero Demo recordings stay checked in as WebM proof artifacts, but
README-facing playback uses GitHub-uploaded H.264 MP4 attachments embedded as
inline `<video>` players.

## Why this shape

GitHub serves `user-attachments` assets only when they are embedded inline in
rendered Markdown (it rewrites them to signed `private-user-images` URLs that
anonymous visitors can stream). Two things follow, both verified against logged-out
GitHub:

1. **A thumbnail _linked_ to a `user-attachments` URL is broken for the public.**
`https://github.com/user-attachments/assets/<uuid>` returns `404` on direct
navigation, so `[![thumb](png)](asset-url)` lands anonymous visitors on a 404.
Use an inline `<video>` element instead.
2. **GitHub strips the `<video poster>` attribute**, so a curated poster image
cannot be supplied as an attribute. Instead the upload MP4 holds the curated
thumbnail as its opening frames, so the player's natural first-frame still
shows the end-state proof rather than a blank startup terminal.

The MP4 copies are derived playback assets, not canonical proof. They live under
`.debug/video-upload/` (git-ignored); the checked-in `*-outer.webm` files remain
the source of truth.

## Prepare upload assets

From the repository root:

```bash
mise run demo:agent-uses-agent-tty:upload-assets
```

The task uses the pinned `ffmpeg`/`ffprobe` from `mise.toml`. For each agent it
prepends ~0.3s of `artifacts/<agent>-thumbnail.png` as the opening frames, encodes
H.264 MP4, writes ffprobe metadata, and writes checksums under `.debug/video-upload/`.

Expected constraints for the promoted 2026-05-21 recordings:

| Agent | Upload file | Expected codec | Expected dimensions | Expected size |
| ------ | ------------------------------------------- | ----------------- | ------------------- | ------------- |
| Codex | `.debug/video-upload/codex-outer-h264.mp4` | H.264 / `yuv420p` | 1600x900 | ~3.5 MB |
| Claude | `.debug/video-upload/claude-outer-h264.mp4` | H.264 / `yuv420p` | 1600x900 | ~3.4 MB |

Both expected sizes are below GitHub's 10 MB video attachment limit for free plans.

## Upload through GitHub

GitHub does not expose a supported PAT-backed API for `user-attachments` uploads:
the endpoint authenticates with a browser `user_session` cookie, not a `gh` OAuth
token. Two working routes:

- **CLI (`gh-image`).** The [`drogers0/gh-image`](https://github.com/drogers0/gh-image)
extension uploads via the same internal endpoints, reading your logged-in browser
`user_session` cookie (treat that cookie like a password):

```bash
gh extension install drogers0/gh-image
gh image --repo coder/agent-tty .debug/video-upload/codex-outer-h264.mp4
gh image --repo coder/agent-tty .debug/video-upload/claude-outer-h264.mp4
```

Copy only the bare `https://github.com/user-attachments/assets/...` URL from each
line (drop the `![](...)` Markdown wrapper).

- **Manual.** Drag each MP4 into any GitHub Markdown text area (a draft issue or PR
comment) with write access to `coder/agent-tty`, wait for the `user-attachments`
URL, and copy it. The draft does not need to be submitted.

## Apply the URLs

```bash
mise run demo:agent-uses-agent-tty:apply-video-urls -- \
--codex-url https://github.com/user-attachments/assets/REPLACE-CODEX \
--claude-url https://github.com/user-attachments/assets/REPLACE-CLAUDE
```

The task rewrites the `src` of the inline `<video>` elements (one per agent, in
Codex/Claude order) in the root README and this bundle README, and refreshes the
bundle manifest entry for `README.md`. Then verify:

```bash
npm run validate-bundle:canonical
```

## Verify in a logged-out browser

This is the step that catches the failure modes above. Open the rendered README on
the branch **while logged out of GitHub** and confirm each `<video>` plays:

- `https://github.com/coder/agent-tty/blob/<branch>/README.md`

Do not test by navigating directly to the `user-attachments/assets/<uuid>` URL — that
returns 404 anonymously by design even when the inline player works. Confirm the
embedded `<video>` element actually streams (it should show the curated thumbnail as
its still, then play).

## Fallback

If GitHub attachment URLs are ever rejected for maintainability, use a GitHub Pages
gallery with `<video controls>` and committed H.264 MP4 playback copies. Do not point
README playback at committed repository videos as the primary path; GitHub may show
those as raw downloads.
14 changes: 10 additions & 4 deletions dogfood/agent-uses-agent-tty/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
"title": "Agents use agent-tty: Codex and Claude Hero Demo",
"description": "README-facing Hero Demo where VHS records real Codex and Claude TUIs while agent-tty produces inner proof artifacts.",
"createdAt": "2026-05-21T15:55:52.138Z",
"scenario": "agent-uses-agent-tty-hero-demo",
"result": "pass",
"commands": [
"mise run demo:agent-uses-agent-tty -- --agent both --runs 3 --record-seconds 180 --promote"
Expand Down Expand Up @@ -114,14 +113,21 @@
{
"path": "README.md",
"description": "Hero Demo bundle README",
"sha256": "dac35e0a5702cd749f726428ececc4905183c89cb4b3580e666c39bcb444fc8b",
"bytes": 1406
"sha256": "5e884d11936bfc38343344b1dd95e4ebafb2b70f2a8e834fdb1fd97a900f6927",
"bytes": 1544
},
{
"path": "VIDEO_PLAYBACK.md",
"description": "GitHub video playback guidance for the Hero Demo",
"sha256": "e62744bd9dac50d97e99fb2d6377b5f09b8a8e1d4e64ada70ae537bb8c4d2f39",
"bytes": 4584
},
{
"path": "reproduce.sh",
"description": "Maintainer-facing reproduction wrapper for the Hero Demo",
"sha256": "92b69748b18cbb64b5d02c25cd98225e8cbc3f793b3d6fc7362d85d3aa8412a9",
"bytes": 276
}
]
],
"scenario": "agent-uses-agent-tty-hero-demo"
}
2 changes: 1 addition & 1 deletion mise.lock
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# @generated - this file is auto-generated by `mise lock` https://mise.jdx.dev/dev-tools/mise-lock.html
# @generated - this file is auto-generated by `mise lock` https://mise.en.dev/dev-tools/mise-lock.html

[[tools.actionlint]]
version = "1.7.12"
Expand Down
34 changes: 32 additions & 2 deletions mise.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
actionlint = "1.7.12"
communique = "1.1.3"
zizmor = "1.24.1"
# Live-demo-only recorder tools are pinned inside src/tools/hero-demo.ts so ordinary CI stays credential- and recorder-tool-free.
# CI installs [tools] with `mise install --locked`; update mise.lock whenever tool versions or supported CI platforms change.
node = "26"
python = "3"
Expand Down Expand Up @@ -37,7 +36,38 @@ run = "npm run smoke:install -- --skip-build"

[tasks."demo:agent-uses-agent-tty"]
description = "Regenerate the real-agent Hero Demo proof bundle"
run = "npx tsx src/tools/hero-demo.ts"
run = '''
PATH="$(mise bin-paths vhs@0.11.0 ttyd@1.7.7 ffmpeg@8.1.1 | paste -sd: -):$PATH" \
npx tsx src/tools/hero-demo.ts
'''
tools.ffmpeg = "8.1.1"
tools.vhs = "0.11.0"
tools.ttyd = "1.7.7"

[tasks."demo:agent-uses-agent-tty:upload-assets"]
description = "Prepare H.264 MP4 upload assets for the Hero Demo"
run = '''
HERO_VIDEO_FFMPEG="$(mise which ffmpeg --tool ffmpeg@8.1.1)" \
HERO_VIDEO_FFPROBE="$(mise which ffprobe --tool ffmpeg@8.1.1)" \
npx tsx src/tools/hero-video-playback.ts prepare-upload-assets
'''
tools.ffmpeg = "8.1.1"
sources = [
"dogfood/agent-uses-agent-tty/artifacts/codex-outer.webm",
"dogfood/agent-uses-agent-tty/artifacts/claude-outer.webm",
"src/tools/hero-video-playback.ts",
]
outputs = ["dogfood/agent-uses-agent-tty/.debug/video-upload/*"]

[tasks."demo:agent-uses-agent-tty:apply-video-urls"]
description = "Apply GitHub user-attachment video URLs to the Hero Demo READMEs"
run = "npx tsx src/tools/hero-video-playback.ts apply-video-urls"
sources = [
"README.md",
"dogfood/agent-uses-agent-tty/README.md",
"dogfood/agent-uses-agent-tty/manifest.json",
"src/tools/hero-video-playback.ts",
]

[tasks.validate-bundles]
description = "Validate canonical proof bundles against the canonical schema"
Expand Down
72 changes: 72 additions & 0 deletions src/tools/canonicalBundleArtifacts.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
import { createHash } from 'node:crypto';
import { createReadStream } from 'node:fs';
import { stat } from 'node:fs/promises';
import { join } from 'node:path';
import { pipeline } from 'node:stream/promises';

import type {
CanonicalBundleArtifact,
CanonicalBundleManifest,
} from './bundleManifestSchema.js';
import { CanonicalBundleManifestSchema } from './bundleManifestSchema.js';
import {
readValidatedJsonFile,
writeValidatedJsonFile,
} from '../storage/manifests.js';
import { invariant } from '../util/assert.js';

export async function sha256File(path: string): Promise<string> {
const hash = createHash('sha256');
await pipeline(createReadStream(path), hash);
return hash.digest('hex');
}

export async function canonicalBundleArtifactEntry(
bundleDir: string,
relativePath: string,
description: string,
): Promise<CanonicalBundleArtifact> {
const fullPath = join(bundleDir, relativePath);
const stats = await stat(fullPath);
return {
path: relativePath,
description,
sha256: await sha256File(fullPath),
bytes: stats.size,
};
}

function validateCanonicalBundleManifest(
_path: string,
data: unknown,
): CanonicalBundleManifest {
return CanonicalBundleManifestSchema.parse(data);
}

export async function readCanonicalBundleManifest(
path: string,
): Promise<CanonicalBundleManifest> {
const manifest = await readValidatedJsonFile({
path,
pathLabel: 'canonical bundle manifest path',
allowMissing: false,
readErrorMessage: `Failed to read canonical bundle manifest at ${path}.`,
invalidJsonMessage: `Canonical bundle manifest contains invalid JSON at ${path}.`,
validate: validateCanonicalBundleManifest,
});
invariant(manifest !== null, 'canonical bundle manifest must exist');
return manifest;
}

export async function writeCanonicalBundleManifest(
path: string,
manifest: CanonicalBundleManifest,
): Promise<void> {
await writeValidatedJsonFile({
path,
pathLabel: 'canonical bundle manifest path',
data: manifest,
writeErrorMessage: `Failed to write canonical bundle manifest at ${path}.`,
validate: validateCanonicalBundleManifest,
});
}
Loading
Loading