Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,6 @@ tmp/

# Local agent state
.claude/

# Local-only audit notes (not part of the upstream repo)
requested improvements.md
16 changes: 1 addition & 15 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This project manages **Vapi voice agent configurations** as code. All resources

**Template-safe first run:** In a fresh clone, prefer `npm run pull -- <org> --bootstrap` to refresh `.vapi-state.<org>.json` and credential mappings without materializing the target org's resources into `resources/<org>/`. `npm run push -- <org>` will auto-run the same bootstrap sync when it detects empty or stale state for the resources being applied.

**Excluding resources from sync (`.vapi-ignore`):** To prevent specific resources from being pulled at all (e.g. assistants owned by another team or legacy resources you don't want to manage), create `resources/<env>/.vapi-ignore` with gitignore-style patterns. See `resources/<env>/.vapi-ignore.example` for syntax and examples. Ignored resources are silently skipped on every pull and never tracked in state — distinct from "locally deleted" which keeps an entry in state.
**Excluding resources from sync (`.vapi-ignore`):** To prevent specific resources from being pulled at all (e.g. assistants owned by another team or legacy resources you don't want to manage), create `resources/<org>/.vapi-ignore` with gitignore-style patterns. See `resources/.vapi-ignore.example` for syntax and examples. Ignored resources are silently skipped on every pull and never tracked in state — distinct from "locally deleted" which keeps an entry in state.

**Learnings & recipes:** Before configuring resources or debugging issues, read the relevant file in **`docs/learnings/`**. Load only what you need:

Expand Down Expand Up @@ -46,7 +46,6 @@ This project manages **Vapi voice agent configurations** as code. All resources
| Add post-call analysis | Create `resources/<org>/structuredOutputs/<name>.yml` |
| Write test simulations | Create files under `resources/<org>/simulations/` |
| Promote resources across orgs | Copy files between `resources/<org-a>/` and `resources/<org-b>/` |
| Test webhook event delivery locally | Run `npm run mock:webhook` and tunnel with ngrok |
| Push changes to Vapi | `npm run push -- <org>` |
| Pull latest from Vapi | `npm run pull -- <org>`, `--force`, or `--bootstrap` |
| Pull one known remote resource | `npm run pull -- <org> --type assistants --id <uuid>` |
Expand Down Expand Up @@ -88,9 +87,6 @@ resources/
│ └── simulations/
└── <another-org>/ # Another org (each is isolated)
└── (same structure)

scripts/
└── mock-vapi-webhook-server.ts # Local webhook receiver for server message testing
```

---
Expand Down Expand Up @@ -755,7 +751,6 @@ npm run call -- <org> -a <assistant-name> # Call an assistant via WebSo
npm run call -- <org> -s <squad-name> # Call a squad via WebSocket
npm run eval -- <org> -s <squad-name> # Run evals against a squad
npm run eval -- <org> -a <assistant-name> # Run evals against an assistant
npm run mock:webhook # Run local webhook receiver for server message testing

# Maintenance
npm run cleanup -- <org> # Dry-run: show orphaned remote resources
Expand Down Expand Up @@ -844,12 +839,3 @@ When transferring to human:
4. Create suites (batch simulations together)
5. Run via Vapi dashboard or API

### Mock Server Testing (Webhook/Message Receipt)

If you need a local mock server to validate webhook payloads or message delivery behavior, you can add scripts under `/scripts` (for example: `scripts/mock-vapi-webhook-server.ts`) and run them locally during testing.

- Default expectation: no provider API key is needed for local receive-only mock testing.
- If a provider-specific key is required, refer to the Vapi monorepo secrets workflow and use `dotenvx` to decrypt the needed values.
- Assume decryption only works when the corresponding private keys are already available in your zsh environment.
- For local webhook validation, prioritize core `serverMessages` event types such as `speech-update`, `status-update`, and `end-of-call-report`.
- To test callbacks from Vapi into your local machine, expose the mock server with a tunnel like `ngrok` and use that public HTTPS URL in `assistant.server.url`.
112 changes: 91 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,10 @@ Every command works in two modes:
| `npm run push` | ✅ | `npm run push -- <org> [flags]` | Push local resources to Vapi |
| `npm run apply` | ✅ | `npm run apply -- <org> [--force]` | Pull → Merge → Push in one shot |
| `npm run call` | ✅ | `npm run call -- <org> -a <name>` | Start a WebSocket call |
| `npm run cleanup` | ✅ | `npm run cleanup -- <org> [--force]` | Delete orphaned remote resources |
| `npm run cleanup` | ✅ | `npm run cleanup -- <org> [--force --confirm <org>]` | Delete orphaned remote resources (destructive run requires `--confirm <org>`) |
| `npm run eval` | — | `npm run eval -- <org> -s <squad>` | Run evals against an assistant/squad |
| `npm run mock:webhook` | — | — | Local webhook receiver for testing |
| `npm run build` | — | — | Type-check the codebase |
| `npm test` | — | — | Run regression tests (`node:test`) |

### Interactive Mode

Expand All @@ -95,15 +95,24 @@ npm run pull
# → Select org
# → All resources / Let me pick…
# → Shows which resources are already local (✔)
# → "Overwrite locally modified files?" — defaults to NO (local-first)
# → Confirm and execute

npm run cleanup
# → Select org
# → Dry-run preview of what would be deleted
# → "Proceed with actual deletion?" — defaults to NO
# → Destructive run is gated by both your confirm AND --confirm <org>
```

Navigation:
- **Type** to search/filter resources
- **Space** to toggle selection
- **Ctrl+A** to select/deselect all visible
- **Space** to toggle the focused row (or toggle the whole group when the cursor is on a header)
- **Ctrl+A** to select/deselect all currently-visible rows
- **Ctrl+G** to toggle every item in the focused group
- **→ / ←** (right / left arrow) to expand or collapse the focused group
- **Enter** to confirm
- **Esc** to go back to the previous step
- **Esc** to clear the search; press again to step back to the previous prompt

### Direct Mode

Expand Down Expand Up @@ -221,7 +230,20 @@ npm run pull -- my-org
# ✨ new-tool -> resources/my-org/tools/new-tool.yml
```

Use `--force` to overwrite everything with the platform version.
Detection works in two layers, so it covers both day-to-day and fresh-clone
workflows:

1. **Git-tracked changes** — files that show up in `git status` (modified,
deleted, or individually untracked) are preserved.
2. **mtime fallback** — if git can't help (no commits yet, the resource tree
isn't tracked at all, or git just had nothing to say), files that are
newer than `.vapi-state.<org>.json` are still preserved. This is the safety
net for the "fresh clone, edit a file, run pull again" case.

Interactive `npm run pull` defaults to local-first too — it asks
`Overwrite locally modified files?` (default `No`) before forwarding the
pull. Pass `--force` directly (or answer `Yes` to that prompt) to overwrite
everything with the platform version.

### Selective Push

Expand All @@ -232,13 +254,23 @@ Push only specific resources instead of everything:
npm run push -- my-org assistants
npm run push -- my-org tools

# By specific file
# By specific file (long form)
npm run push -- my-org resources/my-org/assistants/my-assistant.md

# By specific file (short form — folder/filename)
npm run push -- my-org assistants/my-assistant.md
npm run push -- my-org simulations/personalities/skeptical-sam.yml

# Multiple files
npm run push -- my-org resources/my-org/assistants/a.md resources/my-org/tools/b.yml
```

> A bare resource id like `npm run push -- my-org my-assistant` (no folder,
> no extension) is **rejected explicitly**. The CLI prints
> `Unrecognized argument: my-assistant` and exits with a non-zero code rather
> than silently falling through to a full apply. Pass either a type
> (`assistants`) or a path (`assistants/my-assistant.md`).

### Auto-Dependency Resolution

When pushing a single squad or assistant, missing dependencies (tools, structured outputs, etc.) are automatically created first:
Expand Down Expand Up @@ -273,18 +305,6 @@ npm run eval -- my-org -s my-squad -v eval-variables.json

Evals must be pushed first (`npm run push -- my-org evals`). Eval definitions live in `resources/<org>/evals/*.yml`.

### Webhook Local Testing

```bash
# 1) Run local receiver
npm run mock:webhook

# 2) Expose localhost
ngrok http 8787
```

Set your assistant's `server.url` to the ngrok HTTPS URL.

---

## File Formats
Expand Down Expand Up @@ -497,7 +517,17 @@ Tracks resource ID ↔ Vapi UUID mappings per org:
vapi-gitops/
├── docs/
│ ├── Vapi Prompt Optimization Guide.md
<<<<<<< HEAD
│ └── changelog.md
=======
│ ├── changelog.md
│ └── learnings/ # Gotchas, recipes, troubleshooting per area
│ ├── assistants.md
│ ├── tools.md
│ ├── squads.md
│ ├── simulations.md
│ └── ...
>>>>>>> e280ea5 (docs: align README and AGENTS with org-slug model and P0 fixes)
├── src/
│ ├── setup.ts # Interactive setup wizard
│ ├── interactive.ts # Interactive pull/push/apply/call/cleanup flows
Expand Down Expand Up @@ -533,8 +563,12 @@ vapi-gitops/
│ ├── scenarios/
│ ├── tests/
│ └── suites/
├── scripts/
│ └── mock-vapi-webhook-server.ts
├── tests/
│ ├── credentials.test.ts # Credential walker scoping (P0-1 regression suite)
│ ├── clean-resource.test.ts # null-preservation in pull (P0-3 regression suite)
│ ├── path-matching.test.ts # Short-form path matching (P0-7 regression suite)
│ ├── cleanup-safety.test.ts # --confirm + empty-state gates (P0-4 regression suite)
│ └── cli-arg-parsing.test.ts # Bare-id refusal, --confirm pass-through (P0-7)
├── .env.<org> # API token per org (gitignored)
└── .vapi-state.<org>.json # State file per org
```
Expand Down Expand Up @@ -592,6 +626,42 @@ The credential UUID doesn't exist in the target org. Fix:

Some properties can't be updated after creation. Add them to `UPDATE_EXCLUDED_KEYS` in `src/config.ts`.

### "Refusing to run destructive cleanup" errors

`npm run cleanup` is intentionally double-gated for destructive runs:

- `--force` alone is not enough — you also have to name the org with
`--confirm <org>`. This catches the common mistake of copy-pasting `--force`
from another command where it had a different meaning.
- An empty state file (zero tracked resources) is refused even with both
flags. This prevents a fresh clone or a corrupted state from being misread
as "all remote resources are orphaned" and wiping the org.

```bash
# Wrong — refused
npm run cleanup -- my-org --force

# Right — destructive run
npm run cleanup -- my-org --force --confirm my-org

# Bootstrapping into an empty state? Pull first.
npm run pull -- my-org --bootstrap
```

The interactive `npm run cleanup` flow handles both gates for you (it shows
the dry-run preview, asks you to confirm, and forwards `--force --confirm
<org>` automatically when you say yes).

### "Unrecognized argument" / push appears to do nothing

If you typed `npm run push -- my-org foo` (a bare resource id with no folder
or extension), the CLI now refuses with `Unrecognized argument: foo` rather
than silently running a full apply. Pass either:

- a resource type — `npm run push -- my-org assistants`, or
- a path — `npm run push -- my-org assistants/foo.yml` (short form)
or `npm run push -- my-org resources/my-org/assistants/foo.yml` (long form).

---

## API Reference
Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@
"call": "tsx src/call-cmd.ts",
"cleanup": "tsx src/cleanup-cmd.ts",
"eval": "tsx src/eval.ts",
"mock:webhook": "tsx scripts/mock-vapi-webhook-server.ts",
"build": "tsc --noEmit"
"build": "tsc --noEmit",
"test": "node --import tsx --test tests/*.test.ts"
},
"devDependencies": {
"@types/node": "^22.0.0",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
# .vapi-ignore — explicit opt-out for resources this repo does NOT manage.
#
# Resources matching any pattern below are skipped during `pull:<env>` —
# never written to disk, never tracked in state. Use this for:
# Resources matching any pattern below are skipped during `npm run pull --
# <org>` — never written to disk, never tracked in state. Use this for:
# - Resources owned by another team or repo
# - Legacy/broken resources you don't want to touch
# - Resource types you don't care about (e.g. an entire `simulations/**`)
#
# To activate this file, copy it to `.vapi-ignore` (no `.example` suffix).
# To activate this file, copy it into your org's resource directory and drop
# the `.example` suffix:
#
# cp resources/.vapi-ignore.example resources/<org>/.vapi-ignore
#
# Each org gets its own file — the engine reads
# `resources/<org>/.vapi-ignore` on every pull for that org.
#
# ─────────────────────────────────────────────────────────────────────
# Pattern syntax
Expand All @@ -16,6 +22,7 @@
# - tools/...
# - squads/...
# - structuredOutputs/...
# - evals/...
# - simulations/personalities/...
# - simulations/scenarios/...
# - simulations/tests/...
Expand Down
45 changes: 0 additions & 45 deletions resources/prod/.vapi-ignore.example

This file was deleted.

45 changes: 0 additions & 45 deletions resources/stg/.vapi-ignore.example

This file was deleted.

Loading