Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -174,8 +174,9 @@ detection/
test-script-mode.md
test-script-mode.lock.yml

# Test metrics patterns workflow (temporary validation file)
.github/workflows/test-metrics-patterns.md
# AW Harness (EXPERIMENTAL) — generated by make aw-harness, do not commit
actions/setup/js/aw_harness.cjs

.github/workflows/test-metrics-patterns.lock.yml
.github/workflows/test-*.md
.github/workflows/test-*.lock.yml
Expand Down
33 changes: 31 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ all: build

# Build the binary, run make deps before this
.PHONY: build
build: sync-action-pins sync-action-scripts
build: sync-action-pins sync-action-scripts aw-harness
go build $(LDFLAGS) -o $(BINARY_NAME) ./cmd/gh-aw

# Build for all platforms
Expand Down Expand Up @@ -233,6 +233,35 @@ bundle-js:
@echo "✓ bundle-js tool built"
@echo "To bundle a JavaScript file: ./bundle-js <input-file> [output-file]"

# ── AW Harness (EXPERIMENTAL) ─────────────────────────────────────────────────

# Install aw-harness dependencies
.PHONY: deps-aw-harness
deps-aw-harness: check-node-version
@echo "Installing aw-harness dependencies..."
cd aw-harness && npm ci
@echo "✓ aw-harness dependencies installed"

# Build the aw-harness TypeScript project and copy bundle to actions/setup/js/
# EXPERIMENTAL: engine: aw is experimental and subject to breaking changes
.PHONY: aw-harness
aw-harness: deps-aw-harness
@echo "Building aw-harness (EXPERIMENTAL)..."
cd aw-harness && npm run typecheck && node --experimental-strip-types build.ts
@echo "✓ aw-harness built → actions/setup/js/aw_harness.cjs"

# Run aw-harness tests
.PHONY: test-aw-harness
test-aw-harness: deps-aw-harness
cd aw-harness && npx vitest run

# Lint aw-harness TypeScript sources (typecheck + prettier)
.PHONY: lint-aw-harness
lint-aw-harness: deps-aw-harness
@echo "Linting aw-harness..."
cd aw-harness && npm run typecheck
@echo "✓ aw-harness lint passed"

# Test all code (Go, JavaScript, and wasm golden)
.PHONY: test-all
test-all: test test-js test-wasm-golden
Expand Down Expand Up @@ -609,7 +638,7 @@ check-validator-sizes:

# Validate all project files
.PHONY: lint
lint: fmt-check fmt-check-json lint-cjs golint
lint: fmt-check fmt-check-json lint-cjs lint-aw-harness golint
@echo "✓ All validations passed"

# Install the binary locally
Expand Down
96 changes: 96 additions & 0 deletions aw-harness/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# AW Harness

> ⚠️ **EXPERIMENTAL** — This package is experimental and subject to breaking changes without notice. Do not use in production workflows.

The AW Harness (`aw_harness.cjs`) is a Node.js execution engine for the `engine: aw` mode of GitHub Agentic Workflows (gh-aw). It runs a single Pi `AgentSession` with budget management, steering, and observability.

## Architecture

The harness is a Pi SDK application: it creates a single `AgentSession` and runs a compiled prompt through it, with gh-aw Pi extensions providing budget gating, steering, and observability. See [`specs/aw-harness.md`](../specs/aw-harness.md) for the full specification.

## Status

| Feature | Status |
|---------|--------|
| Project scaffolding | ✅ |
| Provider setup extension | ✅ |
| Cost tracker extension | ✅ |
| Steering extension | ✅ |
| Repair extension | ✅ |
| Observability extension | ✅ |
| Loader (config.json + prompt.txt) | ✅ |
| User extension loading | ✅ |
| Context assembly | ✅ |
| Entry point | ✅ |
| gh-aw compiler integration | 🔲 |

## Development

```bash
# Install dependencies
make deps-aw-harness

# Build (bundles to actions/setup/js/aw_harness.cjs)
make aw-harness

# Run tests
cd aw-harness && npm test

# Typecheck only
cd aw-harness && npm run typecheck

# Lint
make lint-aw-harness
```

## Project Structure

```
aw-harness/
├── package.json # Pinned dependencies
├── tsconfig.json # TypeScript config (ES2024, noEmit)
├── vitest.config.ts # Vitest test runner config
├── build.ts # esbuild → dist/aw_harness.cjs
├── src/
│ ├── index.ts # Entry point (EXPERIMENTAL)
│ ├── types.ts # Shared config types
│ ├── loader.ts # config.json + prompt.txt reader
│ ├── context.ts # Prompt assembly with imports
│ ├── user-extensions.ts # Load user-declared extensions
│ └── extensions/
│ ├── provider-setup.ts # Register LLM providers from env vars
│ ├── cost-tracker.ts # Budget gates via turn_end events
│ ├── steering.ts # Time/budget pressure messages
│ ├── repair.ts # Broken session recovery
│ └── observability.ts # JSONL events + OTel + step summary
├── test/
│ ├── loader.test.ts
│ ├── context.test.ts
│ ├── user-extensions.test.ts
│ └── extensions/
│ ├── provider-setup.test.ts
│ ├── cost-tracker.test.ts
│ ├── steering.test.ts
│ ├── repair.test.ts
│ └── observability.test.ts
└── dist/
└── aw_harness.cjs # Copied to actions/setup/js/aw_harness.cjs
```

## Invocation

The gh-aw compiler pre-processes the workflow markdown and provides the harness with:
- `config.json` — parsed harness configuration
- `prompt.txt` — extracted prompt body

```
node aw_harness.cjs --config <config-path> --prompt <prompt-path>
```

## Exit Codes

| Code | Meaning |
|------|---------|
| `0` | Prompt completed successfully |
| `1` | Session failed or budget exceeded |
| `2` | Invocation error (missing arguments, unreadable files) |
36 changes: 36 additions & 0 deletions aw-harness/build.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/**
* esbuild bundle script for aw_harness.
*
* Compiles src/index.ts → dist/aw_harness.cjs and copies to actions/setup/js/.
* Run with: node --experimental-strip-types build.ts
*/

import { build } from "esbuild";
import { copyFileSync, mkdirSync } from "node:fs";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";

const __dirname = dirname(fileURLToPath(import.meta.url));

const outfile = resolve(__dirname, "dist/aw_harness.cjs");
const dest = resolve(__dirname, "../actions/setup/js/aw_harness.cjs");

await build({
entryPoints: [resolve(__dirname, "src/index.ts")],
bundle: true,
platform: "node",
target: "node24",
format: "cjs",
outfile,
// Bundle all dependencies into the single .cjs (no runtime npm install in container)
external: [],
// Keep readable in Actions logs; inline sourcemap for CI debugging
minify: false,
sourcemap: "inline",
});

mkdirSync(dirname(dest), { recursive: true });
copyFileSync(outfile, dest);

console.log(`✓ Bundled: ${outfile}`);
console.log(`✓ Copied: ${dest}`);
Loading