Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ gh project item-list 2 --owner traverse-framework --format json --limit 300 \
| `phase2-sidecar-nightly` | Future | Defer — low demo value |
| `embed-trace-explorer` | **In Progress** (Cursor) | Unblocked — Traverse `embedded-trace-api` Done (#802) |
| `registry-ref-starter-process` | **Done** (#224) | Process component uses `registry_ref` |
| `consume-product-wasm-agents` | **Ready** | Unblocked — Traverse `real-wasm-agent-execute` Done (#795/#809) |
| `consume-product-wasm-agents` | **In Progress** (Cursor, #226) | Digest-pin Traverse-published starter agents |

Full gap table + wave notes: [`docs/production-reference-plan.md`](docs/production-reference-plan.md).

Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started-embedded.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ bash scripts/ci/embedded_smoke.sh
- **Web:** sync bundle → `BundleEmbedder` + `NodeFsBundleLoader` → init + workflow invoke
- **CLI:** `phase2_link_traverse.sh` → `cargo run -p traverse-starter-cli -- health --json` → `Embedded` / `Ready`
- **Android / Swift / Windows:** digest-check committed `runtime/runtime.wasm`; full SDK tests when tools exist
- Hard-fails unless validate/process/summarize runtime-owned fields are present (smoke WASI fixtures under `scripts/ci/fixtures/traverse-starter-smoke-agents/`)
- Hard-fails unless validate/process/summarize runtime-owned fields are present (digest-pinned Traverse-published agents under `scripts/ci/fixtures/traverse-starter-smoke-agents/`; see `digests.json` provenance)

CI runs this with `EMBEDDED_SMOKE_EXPECT=linux` on every PR (see `.github/workflows/ci.yml`).

Expand Down
2 changes: 1 addition & 1 deletion docs/production-playbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ bash scripts/ci/embedded_smoke.sh
```

- Merge-blocking on PR CI for **Web + CLI** with **runtime-owned pipeline output** required
- Smoke overlays deterministic WASI fixtures (`scripts/ci/fixtures/traverse-starter-smoke-agents/`) until Traverse example agents ship non-stub WASM
- Smoke overlays digest-pinned Traverse-published agents (`scripts/ci/fixtures/traverse-starter-smoke-agents/`; see `digests.json` provenance)
- Other platforms: skip-with-reason when SDK missing; digest-check always

---
Expand Down
4 changes: 2 additions & 2 deletions docs/production-reference-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Related:
| Repo front door | Embedded-first README + playbook; sidecar appendix-only | Decision B |
| Digest sync | **Shared core + thin per-platform wrappers** — see [`runtime-bundle-sync.md`](runtime-bundle-sync.md) | Decision B |
| Showcase rollout | **Wave 1:** Web + Linux/CLI (+ Android if stable); **Wave 2:** Windows + Apple | Decision C |
| Product agents | Smoke fixtures today; adopt Traverse agents via `consume-product-wasm-agents` | Gap capture |
| Product agents | Digest-pinned Traverse-published starter agents (`consume-product-wasm-agents`) | Gap capture |

### Primary vs secondary (locked)

Expand All @@ -58,7 +58,7 @@ Live status is always on [Project 2](https://github.com/orgs/traverse-framework/
| Trace Explorer embed | `embed-trace-explorer` | **In Progress** | Traverse embedded-trace-api Done (#802); migrate web Trace Explorer | Finish PR |
| Delete sidecar client code | `remove-sidecar-paths` | **Done** (#206) | Dead HTTP paths removed from starter/doc-approval | Shipped |
| Nightly Apple/Windows + Android/GTK | `native-ci-android-gtk-required` | **Done** (#209); nightly green via `fix-nightly-native-required` | Required nightly jobs | Shipped |
| Product WASM agents (Traverse) | `consume-product-wasm-agents` | **Ready** | Traverse real-wasm-agent-execute Done (#795/#809) | Replace App-Refs smoke fixtures |
| Product WASM agents (Traverse) | `consume-product-wasm-agents` | **In Progress** (#226) | Traverse real-wasm-agent-execute Done (#795/#809) | Digest-pin Traverse-published starter agents |
| `registry_ref` adoption | `registry-ref-starter-process` | **In Progress** | Process component uses `registry_ref`; sync materializes for embedders | Finish smoke evidence |
| Phase 2 sidecar nightly | `phase2-sidecar-nightly` | **Future** (defer) | Legacy path; low demo value | Optional; low priority |

Expand Down
2 changes: 1 addition & 1 deletion docs/quality-standards.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ A change must not merge when any of the following are true:
Every PR runs `scripts/ci/embedded_smoke.sh` with `EMBEDDED_SMOKE_EXPECT=linux` (`.github/workflows/ci.yml` job `embedded-smoke`):

- Requires Web (`BundleEmbedder` init + full pipeline → validate/process/summarize fields) and Rust CLI (`health` + `run` with the same output shape)
- Uses smoke WASI fixtures when Traverse example agents are stubs; still public embedder path only
- Uses digest-pinned Traverse-published starter agents for pipeline output; still public embedder path only
- Skips Apple / Windows / Android SDK slices with reason when tools are absent; still digest-checks committed `runtime.wasm`
- Does **not** start `traverse-cli serve`
- Coverage gate for `host/embeddedHost.ts` + `client/traverseOutput.ts` is **100%** line coverage
Expand Down
3 changes: 2 additions & 1 deletion scripts/ci/embedded_smoke.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@
# EMBEDDED_SMOKE_SKIP=<csv> force-skip slices (web,cli,gtk,android,swift,windows)
#
# Web + CLI always require runtime-owned pipeline output (validate/process/summarize)
# via scripts/ci/fixtures/traverse-starter-smoke-agents/ (WASI fixtures).
# via digest-pinned Traverse-published agents under
# scripts/ci/fixtures/traverse-starter-smoke-agents/ (see digests.json provenance).
#
# Exit 0 on pass (skips allowed). Exit 1 on any required-slice failure.
set -euo pipefail
Expand Down
70 changes: 0 additions & 70 deletions scripts/ci/fixtures/build_starter_smoke_agents.mjs

This file was deleted.

21 changes: 16 additions & 5 deletions scripts/ci/fixtures/traverse-starter-smoke-agents/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,20 @@
# traverse-starter smoke agents

Deterministic WASI command modules used only by `scripts/ci/embedded_smoke.sh`.
Each writes fixed JSON matching the public capability contracts (validate / process / summarize).
Digest-pinned **Traverse-published** example agents used by
`scripts/ci/prepare_embedded_smoke_bundle.sh` / `scripts/ci/embedded_smoke.sh`.

Rebuild: `node scripts/ci/fixtures/build_starter_smoke_agents.mjs` (requires `wabt`).
Provenance is recorded in `digests.json` (`provenance.ref` + per-agent
`traverse_source`). These binaries are copies of Traverse
`examples/traverse-starter/{validate,process,summarize}-agent` artifacts — not
App-Refs hand-built WASI stand-ins.

These are **not** UI business logic — they stand in for Traverse example agents until
those ship non-stub WASM. Digests are applied to the smoke bundle at prepare time.
Refresh from a Traverse checkout that contains the real stdout-only agents:

```bash
export TRAVERSE_REPO=/path/to/Traverse # main (post #795/#803/#805/#809)
bash scripts/ci/pin_traverse_starter_smoke_agents.sh
```

Prepare overlays these pinned digests onto `$TRAVERSE_REPO` and the synced web
bundle so CI can stay on an older Traverse tag while still exercising product
agent output shapes.
20 changes: 14 additions & 6 deletions scripts/ci/fixtures/traverse-starter-smoke-agents/digests.json
Original file line number Diff line number Diff line change
@@ -1,17 +1,25 @@
{
"provenance": {
"source": "traverse-framework/Traverse",
"ref": "b00a5c4fb8f93503932afd52eadda5e2fe8d04d0",
"note": "Digest-pinned copies of Traverse-published traverse-starter example agents (stdout-only WASM). Not App-Refs hand-built WASI stand-ins. Refresh with: bash scripts/ci/pin_traverse_starter_smoke_agents.sh"
},
"validate": {
"file": "validate-agent.wasm",
"digest": "sha256:ecc46ac478334adb1dacc2bb36841edf2617fcbfab26e6fecfb3d73440805ce9",
"bytes": 154
"digest": "sha256:6cf96349ff6af05cb7417561384dccd63b6f92f4bfc197bcc53838f2ae5b8016",
"bytes": 588,
"traverse_source": "examples/traverse-starter/validate-agent/artifacts/validate-agent.wasm"
},
"process": {
"file": "process-agent.wasm",
"digest": "sha256:15ae4a7431bb45fca2a4d253d4e24fec4962ce067b012c5d1bfc7b86350c79dd",
"bytes": 265
"digest": "sha256:2c6690fd00432f8bff4366d4852395af22a3c766517020532590bab1fba1ab34",
"bytes": 727,
"traverse_source": "examples/traverse-starter/process-agent/artifacts/process-agent.wasm"
},
"summarize": {
"file": "summarize-agent.wasm",
"digest": "sha256:a2e916443233b3346c6ba9211a341f961def495870b77350d2824bba3667440f",
"bytes": 184
"digest": "sha256:fa5d90f9cbdddc7d81daff0f965650cd34c3fc311d8a312cf79637a1bb2c0b7d",
"bytes": 686,
"traverse_source": "examples/traverse-starter/summarize-agent/artifacts/summarize-agent.wasm"
}
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
76 changes: 76 additions & 0 deletions scripts/ci/pin_traverse_starter_smoke_agents.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#!/usr/bin/env bash
# Refresh digest-pinned Traverse-published traverse-starter agents into
# scripts/ci/fixtures/traverse-starter-smoke-agents/.
#
# Requires TRAVERSE_REPO with real (non-stub) example agents.
set -euo pipefail

REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
TRAVERSE_REPO="${TRAVERSE_REPO:-$REPO_ROOT/../Traverse}"
OUT_DIR="$REPO_ROOT/scripts/ci/fixtures/traverse-starter-smoke-agents"

if [ ! -d "$TRAVERSE_REPO/examples/traverse-starter" ]; then
echo "FAIL: TRAVERSE_REPO examples missing: $TRAVERSE_REPO"
exit 1
fi

export REPO_ROOT TRAVERSE_REPO OUT_DIR

python3 - <<'PY'
import hashlib, json, pathlib, os, subprocess

traverse = pathlib.Path(os.environ["TRAVERSE_REPO"])
out = pathlib.Path(os.environ["OUT_DIR"])
out.mkdir(parents=True, exist_ok=True)

mapping = {
"validate": "validate-agent",
"process": "process-agent",
"summarize": "summarize-agent",
}

try:
ref = subprocess.check_output(
["git", "-C", str(traverse), "rev-parse", "HEAD"],
text=True,
).strip()
except subprocess.CalledProcessError as exc:
raise SystemExit(f"FAIL: cannot resolve Traverse HEAD: {exc}") from exc

agents = {}
for key, stem in mapping.items():
src = traverse / f"examples/traverse-starter/{stem}/artifacts/{stem}.wasm"
if not src.is_file():
raise SystemExit(f"FAIL: missing Traverse agent: {src}")
data = src.read_bytes()
if len(data) < 100:
raise SystemExit(
f"FAIL: {src} looks like a stub ({len(data)} bytes). "
"Checkout Traverse main with real stdout agents (#795/#803/#805/#809)."
)
digest = f"sha256:{hashlib.sha256(data).hexdigest()}"
dest = out / f"{stem}.wasm"
dest.write_bytes(data)
agents[key] = {
"file": dest.name,
"digest": digest,
"bytes": len(data),
"traverse_source": f"examples/traverse-starter/{stem}/artifacts/{stem}.wasm",
}
print(f"OK: pinned {dest.name} {digest} ({len(data)} bytes)")

payload = {
"provenance": {
"source": "traverse-framework/Traverse",
"ref": ref,
"note": (
"Digest-pinned copies of Traverse-published traverse-starter example "
"agents (stdout-only WASM). Not App-Refs hand-built WASI stand-ins. "
"Refresh with: bash scripts/ci/pin_traverse_starter_smoke_agents.sh"
),
},
**agents,
}
(out / "digests.json").write_text(json.dumps(payload, indent=2) + "\n")
print(f"OK: wrote {out / 'digests.json'} (Traverse ref={ref})")
PY
42 changes: 33 additions & 9 deletions scripts/ci/prepare_embedded_smoke_bundle.sh
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
#!/usr/bin/env bash
# Overlay functional smoke agents + patch digests for embedded E2E.
# - Overwrites TRAVERSE_REPO example *.wasm (CI/local Traverse checkout)
# Overlay digest-pinned Traverse-published smoke agents + patch digests for embedded E2E.
# - Overwrites TRAVERSE_REPO example *.wasm (CI/local Traverse checkout) with pinned product agents
# - Writes smoke manifest tree under $SMOKE_MANIFEST_DIR (default /tmp)
# - Syncs + patches web public bundle (gitignored)
# Does NOT mutate committed manifests/ in the git worktree.
#
# Pins live in scripts/ci/fixtures/traverse-starter-smoke-agents/ (see digests.json provenance).
# Refresh: bash scripts/ci/pin_traverse_starter_smoke_agents.sh
set -euo pipefail

REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
Expand All @@ -13,7 +16,7 @@ DIGESTS_JSON="$FIXTURE_DIR/digests.json"
SMOKE_MANIFEST_DIR="${SMOKE_MANIFEST_DIR:-/tmp/app-refs-embedded-smoke-manifests}"

if [ ! -f "$DIGESTS_JSON" ]; then
echo "FAIL: missing $DIGESTS_JSON — run: node scripts/ci/fixtures/build_starter_smoke_agents.mjs"
echo "FAIL: missing $DIGESTS_JSON — run: bash scripts/ci/pin_traverse_starter_smoke_agents.sh"
exit 1
fi
if [ ! -d "$TRAVERSE_REPO/examples/traverse-starter" ]; then
Expand All @@ -24,27 +27,47 @@ fi
export REPO_ROOT TRAVERSE_REPO FIXTURE_DIR DIGESTS_JSON SMOKE_MANIFEST_DIR

python3 - <<'PY'
import json, pathlib, shutil, re, os
import hashlib, json, pathlib, shutil, re, os

repo = pathlib.Path(os.environ["REPO_ROOT"])
traverse = pathlib.Path(os.environ["TRAVERSE_REPO"])
fixture = pathlib.Path(os.environ["FIXTURE_DIR"])
digests = json.loads(pathlib.Path(os.environ["DIGESTS_JSON"]).read_text())
digests_doc = json.loads(pathlib.Path(os.environ["DIGESTS_JSON"]).read_text())
smoke_root = pathlib.Path(os.environ["SMOKE_MANIFEST_DIR"])

provenance = digests_doc.get("provenance") or {}
if provenance:
print(
"OK: agent provenance source=%s ref=%s"
% (provenance.get("source", "?"), provenance.get("ref", "?")[:12])
)

mapping = {
"validate": "validate-agent",
"process": "process-agent",
"summarize": "summarize-agent",
}

digests = {}
for key, artifact_stem in mapping.items():
src = fixture / digests[key]["file"]
digest = digests[key]["digest"]
meta = digests_doc[key]
src = fixture / meta["file"]
if not src.is_file():
raise SystemExit(
f"FAIL: missing pinned agent {src} — run: bash scripts/ci/pin_traverse_starter_smoke_agents.sh"
)
data = src.read_bytes()
actual = f"sha256:{hashlib.sha256(data).hexdigest()}"
expected = meta["digest"]
if actual != expected:
raise SystemExit(f"FAIL: digest mismatch for {src}: expected {expected} actual {actual}")
if len(data) < 100:
raise SystemExit(f"FAIL: pinned agent {src} looks like a stub ({len(data)} bytes)")
digests[key] = meta
dest = traverse / f"examples/traverse-starter/{artifact_stem}/artifacts/{artifact_stem}.wasm"
dest.parent.mkdir(parents=True, exist_ok=True)
shutil.copyfile(src, dest)
print(f"OK: overlay {dest} ({digest})")
print(f"OK: overlay {dest} ({expected})")

# Smoke manifest tree (copy then patch; skip existing _traverse link/dir)
if smoke_root.exists():
Expand Down Expand Up @@ -97,7 +120,8 @@ TRAVERSE_REPO="$TRAVERSE_REPO" bash "$REPO_ROOT/scripts/ci/sync_web_starter_bund
python3 - <<'PY'
import json, pathlib, re, os
repo = pathlib.Path(os.environ["REPO_ROOT"])
digests = json.loads(pathlib.Path(os.environ["DIGESTS_JSON"]).read_text())
digests_doc = json.loads(pathlib.Path(os.environ["DIGESTS_JSON"]).read_text())
digests = {k: digests_doc[k] for k in ("validate", "process", "summarize")}
bundle = repo / "apps/traverse-starter/web-react/public/bundles/traverse-starter"
for key, component in [("validate", "validate"), ("process", "process"), ("summarize", "summarize")]:
digest = digests[key]["digest"]
Expand Down
7 changes: 5 additions & 2 deletions scripts/ci/repository_checks.sh
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,11 @@ check "scripts/ci/phase2_link_traverse.sh" "Phase 2 Traverse link"
check "scripts/ci/embedded_smoke.sh" "Embedded runtime smoke"
check "scripts/ci/embedded_smoke_web.mjs" "Embedded web smoke harness"
check "scripts/ci/prepare_embedded_smoke_bundle.sh" "Embedded smoke prepare"
check "scripts/ci/fixtures/traverse-starter-smoke-agents/digests.json" "Smoke agent digests"
check "scripts/ci/fixtures/traverse-starter-smoke-agents/process-agent.wasm" "Smoke process agent wasm"
check "scripts/ci/pin_traverse_starter_smoke_agents.sh" "Pin Traverse starter smoke agents"
check "scripts/ci/fixtures/traverse-starter-smoke-agents/digests.json" "Smoke agent digests (Traverse provenance)"
check "scripts/ci/fixtures/traverse-starter-smoke-agents/process-agent.wasm" "Pinned Traverse process agent wasm"
check "scripts/ci/fixtures/traverse-starter-smoke-agents/validate-agent.wasm" "Pinned Traverse validate agent wasm"
check "scripts/ci/fixtures/traverse-starter-smoke-agents/summarize-agent.wasm" "Pinned Traverse summarize agent wasm"
check "scripts/ci/sync_bundle_core.sh" "Shared bundle sync core"
check "scripts/ci/sync_web_starter_bundle.sh" "Web starter bundle sync"
check "scripts/ci/sync_web_doc_approval_bundle.sh" "Web doc-approval bundle sync"
Expand Down
Loading