Support Investigator

An AI agent that investigates VTEX tickets — a deterministic gate has the final say.

It gathers evidence through a purpose-built MCP server and VTEX's own docs, then proposes a diagnosis. A pure-function policy gate audits that proposal against the evidence and decides the actual verdict — resolving a ticket only at high confidence, escalating everything else to a human.

Built to study how far a policy gate can constrain an LLM's verdict. Independent project, not affiliated with VTEX.

_{Scripted in docs/demo-script.md · Full video ↗}

Quickstart

pnpm install && pnpm dev

pnpm dev alone prints usage. No VTEX credentials needed — the whole agent chain (Strands Agent → McpClient → Support MCP → fixture VtexGateway) runs against fixtures; the only credential required is AWS access for Bedrock.

pnpm dev order-context ORDER-1001                                        # one Support MCP tool call
pnpm dev simulate-fulfillment SKU-2002 1 1 01310-100                     # try SKU-2001 (healthy) vs SKU-2002 (escalation case)
pnpm dev search-knowledge "no delivery option"                           # search_support_knowledge
pnpm dev investigate "SKU-2002 qty 1 seller 1 postal 01310-100 shows in stock but no delivery option shows up"
pnpm dev serve                                                           # starts POST /investigate locally on :8787

See docs/demo-script.md for the full scripted walkthrough, with verbatim ticket text and expected outcomes.

Frontend (local)

cd web && pnpm install && pnpm dev

Vite proxies /investigate to the local Agent API on :8787 — run pnpm dev serve first.

Architecture

React/Vite SPA (evidence timeline, local dev only — see docs/demo-script.md)
   → Agent API (Lambda, Node 22, Hono)
        → Strands Agent (TS) → ModelProvider (Bedrock default | GLM alt.)
        → McpClient (Streamable HTTP)
              → Support MCP (Lambda, Node 22, MCP SDK 1.x stable)
                    → VtexGateway (fixture default | live)
                    → Knowledge Index (frozen JSON + embeddings)
   → Policy Gate (pure TS function — final authority on `status`)

_{Same walkthrough lives in the app itself — the "How it works" tab of the onboarding Drawer · Full video ↗}

3 read-only Support MCP tools: get_order_context, simulate_fulfillment, search_support_knowledge. The model proposes an InvestigationResult; the policy gate decides:

• status: resolved only at confidence: high
• everything else escalates — needs_more_data (missing input, before investigation) or needs_human (insufficient evidence, after investigation)
• see CONTEXT.md's proposed/final entry for why the two can diverge even when they agree on status

Opt-in alternatives — live VTEX API, alternate model provider

Both default to the zero-credential path above; set these only to exercise the alternative.

Env	Values	Default	What it does
VTEX_MODE	fixture / live	fixture	Swaps VtexGateway's fixture implementation for real VTEX API calls — requires VTEX_ACCOUNT_NAME, VTEX_APP_KEY, VTEX_APP_TOKEN. VTEX sandbox access is gated behind partner approval, so this stays opt-in.
MODEL_PROVIDER	bedrock / glm	bedrock	MODEL_PROVIDER=glm switches the CLI's model to GLM (glm-4.7-flash via z.ai) — free local iteration, requires MODEL_API_KEY (MODEL_ID/MODEL_BASE_URL optional overrides). CLI-only: the deployed stack always runs Bedrock.

.env.example documents the Bedrock and VTEX_MODE=live variables; copy it to .env for local overrides (pnpm dev loads it automatically, gitignored).

Testing, evals, insights

pnpm typecheck   # tsc --noEmit
pnpm lint        # biome lint .
pnpm test        # vitest run — unit + integration tests
pnpm evals       # 15 golden tickets + the sabotage regression — the CI gate, fixture VtexGateway + fake model, no credentials
pnpm evals:live  # same 15 tickets against the real configured MODEL_PROVIDER — informational only, never gates CI
pnpm cdk synth   # CDK stack synthesizes cleanly, no AWS credentials required
pnpm insights    # aggregates diagnosis/escalation patterns over whichever of reports/evals*.json already exist — run pnpm evals first

CI (.github/workflows/ci.yml) runs the first five of these, in order, on every push to main and every PR — the same commands, in the same order, a developer runs locally. The badge above reflects main's latest run.

Live AWS deployment

Deployed for real — not just IaC that synths cleanly: SupportInvestigatorStack (us-east-2; account id intentionally not named here — see infra/main.ts, CDK_ACCOUNT_ID), 2 application Lambdas (Agent API, Support MCP) plus a demo-key Lambda authorizer, behind one API Gateway HTTP API.

• Both POST /investigate and /mcp require the x-demo-key header (SSM SecureString, never committed). /mcp was originally left open for third-party MCP clients, until a live, unauthenticated, billable Bedrock call was found reachable by anyone with the URL — that use case is satisfied instead by running this repo's own Support MCP, or by handing out a deployment's key out of band.
• No public Lambda Function URL anywhere in the stack.

The stack is only up during active demos. pnpm cdk destroy — docs/runbooks/cdk-destroy.md — is a required step every time, documented in full in docs/demo-script.md's closing section.

Security

• Credentials (GLM key, VTEX appKey/appToken) live only in SSM SecureString or a local, gitignored .env — never in code, never committed
• No public Lambda Function URL; /investigate is gated by API Gateway + a shared-secret header, not obscurity
• The Agent Lambda's IAM role grants exactly one bedrock:InvokeModel statement, scoped to the configured inference profile plus its routable regions — nothing broader
• Any write/mutating VTEX operation is out of scope — the agent and Support MCP are strictly read-only

Learn more

• CONTEXT.md — the domain glossary (Status, Confidence, proposed/final, NO_DELIVERY_OPTIONS, etc.)
• docs/adr/ — every architecture decision, in order
• GitHub issue #1 (gh issue view 1) — the full spec: user stories, contracts, testing decisions, out-of-scope list