diff --git a/CHANGELOG.md b/CHANGELOG.md index b9d7dc87..49975ea7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,12 @@ ## [Unreleased] +## [1.36.0] - 2026-07-13 + +### Added + +- On a fixture-miss passthrough (record/proxy mode), aimock can inject its own configured upstream provider key instead of forwarding a caller's dummy placeholder key. Every static-key provider aimock proxies is wired end-to-end, each with an independent `AIMOCK_PROVIDER__KEY` env var applied with the provider-correct wire scheme: `Authorization: Bearer` for OpenAI (`_OPENAI_KEY`), OpenRouter (`_OPENROUTER_KEY`), Cohere (`_COHERE_KEY`), Grok/xAI (`_GROK_KEY`), and Ollama (`_OLLAMA_KEY`); `x-api-key` for Anthropic (`_ANTHROPIC_KEY`); `x-goog-api-key` for Gemini (`_GEMINI_KEY`, also used for Gemini Interactions) and Veo (`_VEO_KEY`); `api-key` for Azure OpenAI (`_AZURE_KEY`); `xi-api-key` for ElevenLabs (`_ELEVENLABS_KEY`); and `Authorization: Key` for fal.ai (`_FAL_KEY`). Injection fires only when the caller credential is absent or dummy-prefixed (`sk-aimock-`, overridable via `AIMOCK_DUMMY_KEY_MARKER`); a real caller key is always forwarded unchanged, an empty-string env var is treated as unset, and with no built-in key configured the feature is inert. Signed/exchanged credentials — AWS Bedrock (SigV4) and Vertex AI (OAuth) — are never rewritten (#293) + ## [1.35.1] - 2026-07-06 ### Fixed diff --git a/README.md b/README.md index 0e3a29b7..410cd936 100644 --- a/README.md +++ b/README.md @@ -127,6 +127,28 @@ Private and link-local addresses (loopback, RFC1918, CGNAT, cloud metadata, ULA, On replay, `turnIndex` is a non-fatal disambiguator, not a hard reject gate: a content-matching fixture is served even when its scripted `turnIndex` differs from the request's assistant-message count. This kills false "no fixture matched" misses for multi-bubble agent runs (multi-step agents emit several assistant bubbles per logical turn). When a served fixture diverges from its scripted `turnIndex`, the match diagnostic carries `turnIndexRelaxed: true` and aimock logs a one-shot warning (at the `warn` log level — silent by default). To restore the legacy strict behavior where a defined `turnIndex` must equal the assistant count exactly, set `AIMOCK_STRICT_TURN_INDEX=1`. The record path is always strict regardless of this flag. +### aimock-owned upstream keys — `AIMOCK_PROVIDER_*_KEY` + +In record or `--proxy-only` mode, aimock forwards the caller's auth header to the real provider unchanged. If your tests can only send a dummy placeholder key (e.g. an SDK that refuses to start without a non-empty API key), aimock can inject its own configured upstream key on a fixture-miss passthrough so the proxied call actually authenticates. Each provider has an independent env var, and the key is applied with the provider-correct wire scheme: + +| Env var | Provider | Injected header | +| -------------------------------- | -------------------------------- | ----------------------------- | +| `AIMOCK_PROVIDER_OPENAI_KEY` | OpenAI | `Authorization: Bearer ` | +| `AIMOCK_PROVIDER_OPENROUTER_KEY` | OpenRouter | `Authorization: Bearer ` | +| `AIMOCK_PROVIDER_COHERE_KEY` | Cohere | `Authorization: Bearer ` | +| `AIMOCK_PROVIDER_GROK_KEY` | Grok (xAI) | `Authorization: Bearer ` | +| `AIMOCK_PROVIDER_OLLAMA_KEY` | Ollama (Cloud / bearer-gated) | `Authorization: Bearer ` | +| `AIMOCK_PROVIDER_ANTHROPIC_KEY` | Anthropic | `x-api-key: ` | +| `AIMOCK_PROVIDER_GEMINI_KEY` | Gemini (and Gemini Interactions) | `x-goog-api-key: ` | +| `AIMOCK_PROVIDER_VEO_KEY` | Veo | `x-goog-api-key: ` | +| `AIMOCK_PROVIDER_AZURE_KEY` | Azure OpenAI | `api-key: ` | +| `AIMOCK_PROVIDER_ELEVENLABS_KEY` | ElevenLabs | `xi-api-key: ` | +| `AIMOCK_PROVIDER_FAL_KEY` | fal.ai | `Authorization: Key ` | + +`gemini-interactions` reuses `AIMOCK_PROVIDER_GEMINI_KEY` (same upstream API as Gemini). An empty-string value is treated as unset. + +This is opt-in and backward-compatible: with no key configured the feature is inert and the caller's header passes through as-is. Injection fires only when the caller sent no credential **or** a dummy credential prefixed with `sk-aimock-` (overridable via `AIMOCK_DUMMY_KEY_MARKER`); a real caller key never starting with that marker is always forwarded verbatim, so the caller overrides aimock. Signed and exchanged credentials — AWS Bedrock (SigV4) and Vertex AI (OAuth) — are never rewritten and always forwarded unchanged. (Azure's static `api-key` is injected; a real Microsoft Entra ID `Authorization: Bearer` token from the caller is never dummy-prefixed, so it too passes through verbatim.) + ## Framework Guides Test your AI agents with aimock — no API keys, no network calls: [LangChain](https://aimock.copilotkit.dev/integrate-langchain) · [CrewAI](https://aimock.copilotkit.dev/integrate-crewai) · [PydanticAI](https://aimock.copilotkit.dev/integrate-pydanticai) · [LlamaIndex](https://aimock.copilotkit.dev/integrate-llamaindex) · [Mastra](https://aimock.copilotkit.dev/integrate-mastra) · [Google ADK](https://aimock.copilotkit.dev/integrate-adk) · [Microsoft Agent Framework](https://aimock.copilotkit.dev/integrate-maf) diff --git a/docs/record-replay/index.html b/docs/record-replay/index.html index 062ab46c..6d49dfd4 100644 --- a/docs/record-replay/index.html +++ b/docs/record-replay/index.html @@ -489,6 +489,114 @@

Header Forwarding

recordedTimings block (streaming frame timings) — never the request headers.

+

aimock-Owned Upstream Keys

+

+ By default, aimock forwards the caller's auth header to the upstream provider unchanged. + If your tests can only send a dummy placeholder key — for example, an SDK that refuses to + start without a non-empty API key — aimock can inject its own configured upstream + key on a fixture-miss passthrough so the recorded/proxied call actually authenticates. + This is opt-in and backward-compatible: with no key configured the + feature is fully inert and the caller's header is forwarded as-is. +

+

+ Set one or more of these env vars to aimock's real upstream keys. Each is independent — + configure only the providers you record against: +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Env varProviderInjected header
AIMOCK_PROVIDER_OPENAI_KEYOpenAIAuthorization: Bearer <key>
AIMOCK_PROVIDER_OPENROUTER_KEYOpenRouterAuthorization: Bearer <key>
AIMOCK_PROVIDER_COHERE_KEYCohereAuthorization: Bearer <key>
AIMOCK_PROVIDER_GROK_KEYGrok (xAI)Authorization: Bearer <key>
AIMOCK_PROVIDER_OLLAMA_KEYOllama (Cloud / bearer-gated)Authorization: Bearer <key>
AIMOCK_PROVIDER_ANTHROPIC_KEYAnthropicx-api-key: <key>
AIMOCK_PROVIDER_GEMINI_KEYGemini (and Gemini Interactions)x-goog-api-key: <key>
AIMOCK_PROVIDER_VEO_KEYVeox-goog-api-key: <key>
AIMOCK_PROVIDER_AZURE_KEYAzure OpenAIapi-key: <key>
AIMOCK_PROVIDER_ELEVENLABS_KEYElevenLabsxi-api-key: <key>
AIMOCK_PROVIDER_FAL_KEYfal.aiAuthorization: Key <key>
+

+ gemini-interactions reuses AIMOCK_PROVIDER_GEMINI_KEY (same + upstream API as Gemini). An empty-string value is treated as unset. +

+

Injection fires only when both of these hold:

+
    +
  • + The request is a fixture-miss passthrough (record mode or + --proxy-only). +
    • +
  • + The caller sent no credential for that provider, + or sent a dummy credential prefixed with + sk-aimock-. A caller credential that does not start with the marker is + treated as a real key and forwarded verbatim — the caller always overrides aimock. +
    • +
+

+ The dummy marker prefix is overridable via AIMOCK_DUMMY_KEY_MARKER for setups + that mint placeholder keys under a different prefix. Gemini is injected as an + x-goog-api-key header only (no query-param ?key= rewrite). +

+

+ Signed and exchanged credentials are never rewritten. AWS Bedrock + (SigV4), Vertex AI, and Azure AD carry OAuth/signed auth rather than a simple bearer or + api-key header, so aimock always forwards their credentials unchanged regardless of these + env vars. +

+

Strict Mode

When --strict is enabled, an unmatched request returns diff --git a/package.json b/package.json index 8f417070..d4212f81 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@copilotkit/aimock", - "version": "1.35.1", + "version": "1.36.0", "description": "Mock infrastructure for AI application testing — LLM APIs, image generation, image editing, text-to-speech, transcription, audio translation, audio generation, video generation, embeddings, MCP tools, A2A agents, AG-UI event streams, vector databases, search, rerank, and moderation. One package, one port, zero dependencies.", "license": "MIT", "keywords": [ diff --git a/src/__tests__/provider-auth.test.ts b/src/__tests__/provider-auth.test.ts new file mode 100644 index 00000000..b93755ca --- /dev/null +++ b/src/__tests__/provider-auth.test.ts @@ -0,0 +1,435 @@ +import { describe, it, expect, afterEach, beforeEach } from "vitest"; +import * as http from "node:http"; +import type { RecordProviderKey } from "../types.js"; +import { createServer, type ServerInstance } from "../server.js"; + +// --------------------------------------------------------------------------- +// This suite exercises the REAL forwarded-header surface: a fake upstream HTTP +// server records the auth headers aimock forwards to it on a fixture-miss +// passthrough (proxy-only). We assert on what the fake upstream actually saw — +// not on a mock of aimock's internals. +// --------------------------------------------------------------------------- + +function post( + url: string, + body: unknown, + headers?: Record, +): Promise<{ status: number; body: string }> { + return new Promise((resolve, reject) => { + const data = JSON.stringify(body); + const parsed = new URL(url); + const req = http.request( + { + hostname: parsed.hostname, + port: parsed.port, + path: parsed.pathname, + method: "POST", + headers: { + "Content-Type": "application/json", + "Content-Length": Buffer.byteLength(data), + ...headers, + }, + }, + (res) => { + const chunks: Buffer[] = []; + res.on("data", (c: Buffer) => chunks.push(c)); + res.on("end", () => + resolve({ status: res.statusCode ?? 0, body: Buffer.concat(chunks).toString() }), + ); + }, + ); + req.on("error", reject); + req.write(data); + req.end(); + }); +} + +interface FakeUpstream { + server: http.Server; + url: string; + /** Auth-relevant headers observed on the most recent request. */ + last: () => http.IncomingHttpHeaders; + requestCount: () => number; +} + +/** + * A fake upstream that records the headers it receives and returns a minimal + * OpenAI-shaped chat completion so aimock's collapse/record path is happy. + */ +function createFakeUpstream(): Promise { + return new Promise((resolve) => { + let seen: http.IncomingHttpHeaders = {}; + let count = 0; + const server = http.createServer((req, res) => { + count++; + seen = req.headers; + // Drain the request body before responding. + req.on("data", () => {}); + req.on("end", () => { + res.writeHead(200, { "Content-Type": "application/json" }); + res.end( + JSON.stringify({ + id: "chatcmpl-fake", + object: "chat.completion", + created: Date.now(), + model: "gpt-4", + choices: [ + { + index: 0, + message: { role: "assistant", content: "hi" }, + finish_reason: "stop", + }, + ], + usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 }, + }), + ); + }); + }); + server.listen(0, "127.0.0.1", () => { + const addr = server.address(); + const port = typeof addr === "object" && addr ? addr.port : 0; + resolve({ + server, + url: `http://127.0.0.1:${port}`, + last: () => seen, + requestCount: () => count, + }); + }); + }); +} + +const DUMMY = "sk-aimock-dev-ci-only"; +const REAL = "sk-real-test-123"; + +let upstream: FakeUpstream | undefined; +let recorder: ServerInstance | undefined; + +afterEach(async () => { + if (recorder) { + await new Promise((r) => recorder!.server.close(() => r())); + recorder = undefined; + } + if (upstream) { + await new Promise((r) => upstream!.server.close(() => r())); + upstream = undefined; + } +}); + +/** + * Stand up the fake upstream + an aimock proxy-only recorder pointed at it for a + * single provider, optionally with a built-in key configured for that provider. + */ +async function setup( + provider: RecordProviderKey, + providerKey?: string, +): Promise<{ recorderUrl: string }> { + upstream = await createFakeUpstream(); + recorder = await createServer([], { + port: 0, + record: { + providers: { [provider]: upstream.url }, + providerKeys: providerKey ? { [provider]: providerKey } : undefined, + proxyOnly: true, + }, + }); + return { recorderUrl: recorder.url }; +} + +const CHAT_PATH = "/v1/chat/completions"; +const CHAT_BODY = { model: "gpt-4", messages: [{ role: "user", content: "unmatched-miss" }] }; + +describe("applyProviderAuth — aimock owns the upstream key", () => { + describe("OpenAI (bearer)", () => { + it("RED baseline: with NO built-in key, a dummy caller key is forwarded verbatim", async () => { + const { recorderUrl } = await setup("openai"); // feature OFF + const res = await post(recorderUrl + CHAT_PATH, CHAT_BODY, { + Authorization: `Bearer ${DUMMY}`, + }); + expect(res.status).toBe(200); + // Today's behavior (and case (a)): dummy forwarded unchanged. + expect(upstream!.last().authorization).toBe(`Bearer ${DUMMY}`); + }); + + it("GREEN: built-in key set + caller sends dummy → aimock injects its own key", async () => { + const { recorderUrl } = await setup("openai", REAL); + const res = await post(recorderUrl + CHAT_PATH, CHAT_BODY, { + Authorization: `Bearer ${DUMMY}`, + }); + expect(res.status).toBe(200); + expect(upstream!.last().authorization).toBe(`Bearer ${REAL}`); + }); + + it("case (b): built-in key set + caller sends a REAL key → caller overrides", async () => { + const callerReal = "sk-caller-owns-this-999"; + const { recorderUrl } = await setup("openai", REAL); + await post(recorderUrl + CHAT_PATH, CHAT_BODY, { + Authorization: `Bearer ${callerReal}`, + }); + expect(upstream!.last().authorization).toBe(`Bearer ${callerReal}`); + }); + + it("no caller credential + built-in key set → aimock injects its own key", async () => { + const { recorderUrl } = await setup("openai", REAL); + await post(recorderUrl + CHAT_PATH, CHAT_BODY); + expect(upstream!.last().authorization).toBe(`Bearer ${REAL}`); + }); + }); + + describe("Anthropic (x-api-key)", () => { + it("built-in key set + caller sends dummy → injects x-api-key", async () => { + const { recorderUrl } = await setup("anthropic", REAL); + await post(recorderUrl + "/v1/messages", CHAT_BODY, { "x-api-key": DUMMY }); + expect(upstream!.last()["x-api-key"]).toBe(REAL); + }); + + it("caller sends a real x-api-key → forwarded unchanged", async () => { + const { recorderUrl } = await setup("anthropic", REAL); + await post(recorderUrl + "/v1/messages", CHAT_BODY, { "x-api-key": "real-anthropic-key" }); + expect(upstream!.last()["x-api-key"]).toBe("real-anthropic-key"); + }); + }); + + describe("Gemini (x-goog-api-key)", () => { + it("built-in key set + caller sends dummy → injects x-goog-api-key", async () => { + const { recorderUrl } = await setup("gemini", REAL); + await post(recorderUrl + "/v1beta/models/gemini-pro:generateContent", CHAT_BODY, { + "x-goog-api-key": DUMMY, + }); + expect(upstream!.last()["x-goog-api-key"]).toBe(REAL); + }); + }); + + describe("Cohere (bearer, via /v2/chat)", () => { + const COHERE_PATH = "/v2/chat"; + const COHERE_BODY = { + model: "command-r", + messages: [{ role: "user", content: "unmatched-miss" }], + }; + + it("built-in key set + caller sends dummy → injects Bearer", async () => { + const { recorderUrl } = await setup("cohere", REAL); + await post(recorderUrl + COHERE_PATH, COHERE_BODY, { + Authorization: `Bearer ${DUMMY}`, + }); + expect(upstream!.last().authorization).toBe(`Bearer ${REAL}`); + }); + + it("caller sends a real Bearer key → forwarded unchanged", async () => { + const callerReal = "co-caller-real-abc"; + const { recorderUrl } = await setup("cohere", REAL); + await post(recorderUrl + COHERE_PATH, COHERE_BODY, { + Authorization: `Bearer ${callerReal}`, + }); + expect(upstream!.last().authorization).toBe(`Bearer ${callerReal}`); + }); + + it("inert when no built-in key set → dummy forwarded verbatim", async () => { + const { recorderUrl } = await setup("cohere"); // feature OFF + await post(recorderUrl + COHERE_PATH, COHERE_BODY, { + Authorization: `Bearer ${DUMMY}`, + }); + expect(upstream!.last().authorization).toBe(`Bearer ${DUMMY}`); + }); + }); + + describe("Ollama (bearer, via /api/chat)", () => { + const OLLAMA_PATH = "/api/chat"; + const OLLAMA_BODY = { + model: "llama3", + messages: [{ role: "user", content: "unmatched-miss" }], + }; + + it("built-in key set + caller sends dummy → injects Bearer", async () => { + const { recorderUrl } = await setup("ollama", REAL); + await post(recorderUrl + OLLAMA_PATH, OLLAMA_BODY, { + Authorization: `Bearer ${DUMMY}`, + }); + expect(upstream!.last().authorization).toBe(`Bearer ${REAL}`); + }); + + it("inert when no built-in key set → dummy forwarded verbatim", async () => { + const { recorderUrl } = await setup("ollama"); // feature OFF + await post(recorderUrl + OLLAMA_PATH, OLLAMA_BODY, { + Authorization: `Bearer ${DUMMY}`, + }); + expect(upstream!.last().authorization).toBe(`Bearer ${DUMMY}`); + }); + }); + + describe("Azure OpenAI (api-key, via /openai/deployments/{id}/chat/completions)", () => { + const AZURE_PATH = "/openai/deployments/my-deploy/chat/completions"; + + it("built-in key set + caller sends dummy → injects api-key", async () => { + const { recorderUrl } = await setup("azure", REAL); + await post(recorderUrl + AZURE_PATH, CHAT_BODY, { "api-key": DUMMY }); + expect(upstream!.last()["api-key"]).toBe(REAL); + }); + + it("caller sends a real api-key → forwarded unchanged", async () => { + const callerReal = "azure-caller-real-xyz"; + const { recorderUrl } = await setup("azure", REAL); + await post(recorderUrl + AZURE_PATH, CHAT_BODY, { "api-key": callerReal }); + expect(upstream!.last()["api-key"]).toBe(callerReal); + }); + + it("inert when no built-in key set → dummy forwarded verbatim", async () => { + const { recorderUrl } = await setup("azure"); // feature OFF + await post(recorderUrl + AZURE_PATH, CHAT_BODY, { "api-key": DUMMY }); + expect(upstream!.last()["api-key"]).toBe(DUMMY); + }); + }); + + describe("ElevenLabs (xi-api-key, via /v1/text-to-speech/{voice})", () => { + const EL_PATH = "/v1/text-to-speech/voice-123"; + const EL_BODY = { text: "unmatched-miss speak this" }; + + it("built-in key set + caller sends dummy → injects xi-api-key", async () => { + const { recorderUrl } = await setup("elevenlabs", REAL); + await post(recorderUrl + EL_PATH, EL_BODY, { "xi-api-key": DUMMY }); + expect(upstream!.last()["xi-api-key"]).toBe(REAL); + }); + + it("caller sends a real xi-api-key → forwarded unchanged", async () => { + const callerReal = "el-caller-real-777"; + const { recorderUrl } = await setup("elevenlabs", REAL); + await post(recorderUrl + EL_PATH, EL_BODY, { "xi-api-key": callerReal }); + expect(upstream!.last()["xi-api-key"]).toBe(callerReal); + }); + + it("inert when no built-in key set → dummy forwarded verbatim", async () => { + const { recorderUrl } = await setup("elevenlabs"); // feature OFF + await post(recorderUrl + EL_PATH, EL_BODY, { "xi-api-key": DUMMY }); + expect(upstream!.last()["xi-api-key"]).toBe(DUMMY); + }); + }); + + describe("fal.ai (Authorization: Key, via /fal/queue/submit)", () => { + const FAL_PATH = "/fal/queue/submit/fal-ai/some-model"; + const FAL_BODY = { prompt: "unmatched-miss render this" }; + + it("built-in key set + caller sends dummy → injects Authorization: Key", async () => { + const { recorderUrl } = await setup("fal", REAL); + await post(recorderUrl + FAL_PATH, FAL_BODY, { + Authorization: `Key ${DUMMY}`, + }); + expect(upstream!.last().authorization).toBe(`Key ${REAL}`); + }); + + it("caller sends a real Key credential → forwarded unchanged", async () => { + const callerReal = "fal-caller-real-555"; + const { recorderUrl } = await setup("fal", REAL); + await post(recorderUrl + FAL_PATH, FAL_BODY, { + Authorization: `Key ${callerReal}`, + }); + expect(upstream!.last().authorization).toBe(`Key ${callerReal}`); + }); + + it("inert when no built-in key set → dummy forwarded verbatim", async () => { + const { recorderUrl } = await setup("fal"); // feature OFF + await post(recorderUrl + FAL_PATH, FAL_BODY, { + Authorization: `Key ${DUMMY}`, + }); + expect(upstream!.last().authorization).toBe(`Key ${DUMMY}`); + }); + }); + + describe("fixture MATCH short-circuits injection", () => { + it("a matched request never reaches the proxy, so no injection happens", async () => { + upstream = await createFakeUpstream(); + // Serve a fixture that matches the request so proxyAndRecord never runs. + recorder = await createServer( + [ + { + match: { userMessage: "matched-hit" }, + response: { content: "served from fixture" }, + }, + ], + { + port: 0, + record: { + providers: { openai: upstream.url }, + providerKeys: { openai: REAL }, + proxyOnly: true, + }, + }, + ); + const res = await post( + recorder.url + CHAT_PATH, + { model: "gpt-4", messages: [{ role: "user", content: "matched-hit" }] }, + { Authorization: `Bearer ${DUMMY}` }, + ); + expect(res.status).toBe(200); + expect(res.body).toContain("served from fixture"); + // The fake upstream was never contacted → injection path never ran. + expect(upstream!.requestCount()).toBe(0); + }); + }); +}); + +describe("readProviderKeysFromEnv", () => { + const ALL_ENV_VARS = [ + "AIMOCK_PROVIDER_OPENAI_KEY", + "AIMOCK_PROVIDER_ANTHROPIC_KEY", + "AIMOCK_PROVIDER_GEMINI_KEY", + "AIMOCK_PROVIDER_OPENROUTER_KEY", + "AIMOCK_PROVIDER_COHERE_KEY", + "AIMOCK_PROVIDER_GROK_KEY", + "AIMOCK_PROVIDER_OLLAMA_KEY", + "AIMOCK_PROVIDER_VEO_KEY", + "AIMOCK_PROVIDER_AZURE_KEY", + "AIMOCK_PROVIDER_ELEVENLABS_KEY", + "AIMOCK_PROVIDER_FAL_KEY", + ] as const; + const saved = { ...process.env }; + beforeEach(() => { + for (const name of ALL_ENV_VARS) delete process.env[name]; + }); + afterEach(() => { + process.env = { ...saved }; + }); + + it("returns undefined when no provider key env vars are set", async () => { + const { readProviderKeysFromEnv } = await import("../provider-auth.js"); + expect(readProviderKeysFromEnv({})).toBeUndefined(); + }); + + it("reads every wired provider key from its env var", async () => { + const { readProviderKeysFromEnv } = await import("../provider-auth.js"); + const keys = readProviderKeysFromEnv({ + AIMOCK_PROVIDER_OPENAI_KEY: "o", + AIMOCK_PROVIDER_ANTHROPIC_KEY: "a", + AIMOCK_PROVIDER_GEMINI_KEY: "g", + AIMOCK_PROVIDER_OPENROUTER_KEY: "or", + AIMOCK_PROVIDER_COHERE_KEY: "co", + AIMOCK_PROVIDER_GROK_KEY: "gr", + AIMOCK_PROVIDER_OLLAMA_KEY: "ol", + AIMOCK_PROVIDER_VEO_KEY: "ve", + AIMOCK_PROVIDER_AZURE_KEY: "az", + AIMOCK_PROVIDER_ELEVENLABS_KEY: "el", + AIMOCK_PROVIDER_FAL_KEY: "fa", + } as NodeJS.ProcessEnv); + expect(keys).toEqual({ + openai: "o", + anthropic: "a", + gemini: "g", + openrouter: "or", + cohere: "co", + grok: "gr", + ollama: "ol", + veo: "ve", + azure: "az", + elevenlabs: "el", + fal: "fa", + }); + }); + + it("treats an empty-string env var as unset (feature stays inert)", async () => { + const { readProviderKeysFromEnv } = await import("../provider-auth.js"); + const keys = readProviderKeysFromEnv({ + AIMOCK_PROVIDER_OPENAI_KEY: "o", + AIMOCK_PROVIDER_COHERE_KEY: "", + AIMOCK_PROVIDER_FAL_KEY: "", + } as NodeJS.ProcessEnv); + expect(keys).toEqual({ openai: "o" }); + }); +}); diff --git a/src/cli.ts b/src/cli.ts index 4e366e49..7056472d 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -8,6 +8,7 @@ import { Logger, type LogLevel } from "./logger.js"; import { watchFixtures } from "./watcher.js"; import { AGUIMock } from "./agui-mock.js"; import { resolveFixturesValue } from "./fixtures-remote.js"; +import { readProviderKeysFromEnv } from "./provider-auth.js"; import type { Fixture, ChaosConfig, RecordConfig } from "./types.js"; const HELP = ` @@ -276,6 +277,10 @@ if (values.record || values["proxy-only"]) { } record = { providers, + // aimock's own upstream keys, sourced from AIMOCK_PROVIDER_*_KEY env vars + // (not CLI flags — secrets must not appear in `ps`). Injected on a + // fixture-miss passthrough when the caller sent no/dummy credential. + providerKeys: readProviderKeysFromEnv(), // In proxy-only mode with only URL sources, fixturePath is never consumed // (recorder.ts skips disk writes when proxyOnly is set). Leave it undefined // rather than resolving a URL string as a filesystem path. diff --git a/src/fal-audio.ts b/src/fal-audio.ts index 62aa6a12..f79a8a64 100644 --- a/src/fal-audio.ts +++ b/src/fal-audio.ts @@ -577,6 +577,9 @@ async function tryRecordAudioQueueWalk(args: { submitPath: pathname, body, headers: buildForwardHeaders(req), + // aimock's built-in fal key (Authorization: Key), injected by the walk on + // a no/dummy caller credential; a real caller key overrides. + builtinKey: record.providerKeys?.fal, pollIntervalMs: record.fal?.pollIntervalMs, timeoutMs: record.fal?.timeoutMs, upstreamTimeoutMs: record.upstreamTimeoutMs, diff --git a/src/fal.ts b/src/fal.ts index 9fb16ba9..268aace6 100644 --- a/src/fal.ts +++ b/src/fal.ts @@ -36,6 +36,7 @@ import { sanitizeHeaderValue, } from "./recorder.js"; import { resolveUpstreamUrl } from "./url.js"; +import { applyProviderAuth } from "./provider-auth.js"; import type { Journal } from "./journal.js"; import { audioToFalFile } from "./fal-audio.js"; @@ -885,6 +886,14 @@ export async function walkFalQueue(args: { fallbackResultPath: (requestId: string) => string; /** Warn sink for the same-origin envelope-URL gate (omitting it only mutes the warns). */ logger?: Logger; + /** + * aimock's built-in fal key, when configured. Injected as `Authorization: Key + * ` on every walk fetch (submit/status/result) if the caller sent no + * credential or a dummy placeholder; a real caller credential overrides. This + * is the single chokepoint for all fal queue walks — the queue path doesn't + * route through `proxyAndRecord`, so own-key injection lives here. + */ + builtinKey?: string; }): Promise { const { upstreamBase, @@ -897,8 +906,14 @@ export async function walkFalQueue(args: { fallbackStatusPath, fallbackResultPath, logger, + builtinKey, } = args; + // Inject aimock's own fal credential onto the walk headers up front so every + // fetch below (submit + polls) carries it. fal's scheme ignores the target + // URL, so any resolvable URL suffices. + applyProviderAuth(headers, resolveUpstreamUrl(upstreamBase, submitPath), "fal", builtinKey); + const deadline = Date.now() + timeoutMs; const perFetchTimeoutMs = clampTimeout(upstreamTimeoutMs, DEFAULT_FAL_FETCH_TIMEOUT_MS); // Bound every upstream fetch: the per-fetch timeout, additionally clamped @@ -1065,6 +1080,9 @@ async function proxyAndRecordFalQueueSubmit(args: { submitPath: strippedPath, body, headers: buildForwardHeaders(req), + // aimock's built-in fal key (Authorization: Key), injected by the walk on + // a no/dummy caller credential; a real caller key overrides. + builtinKey: record.providerKeys?.fal, pollIntervalMs: record.fal?.pollIntervalMs, timeoutMs: record.fal?.timeoutMs, upstreamTimeoutMs: record.upstreamTimeoutMs, diff --git a/src/provider-auth.ts b/src/provider-auth.ts new file mode 100644 index 00000000..b00f436e --- /dev/null +++ b/src/provider-auth.ts @@ -0,0 +1,238 @@ +import type { RecordProviderKey } from "./types.js"; + +/** + * Default marker prefix identifying a "dummy" credential that a caller sends + * only to satisfy an SDK's non-empty API-key requirement (e.g. `sk-aimock-...`). + * A caller credential that is absent OR begins with this prefix is treated as a + * placeholder that aimock is free to override with its own built-in upstream key. + * A caller credential that does NOT begin with this prefix is treated as a real + * key and forwarded verbatim (the caller overrides aimock). + * + * Overridable via the `AIMOCK_DUMMY_KEY_MARKER` env var for setups that mint + * placeholder keys under a different prefix. + */ +export const DEFAULT_DUMMY_KEY_MARKER = "sk-aimock-"; + +/** Resolve the active dummy-key marker, honoring the env override. */ +export function getDummyKeyMarker(): string { + const override = process.env.AIMOCK_DUMMY_KEY_MARKER; + return override && override.length > 0 ? override : DEFAULT_DUMMY_KEY_MARKER; +} + +/** + * How a given provider carries its API credential on the wire. aimock injects + * its built-in key using the provider's native scheme so the upstream accepts + * it. Providers whose auth is a signed/exchanged credential (Bedrock SigV4, + * Vertex AI OAuth) are deliberately absent — those are NOT simple static-key + * schemes, so aimock never rewrites their auth and always forwards the caller's + * credential unchanged. + * + * Scheme kinds (all static long-lived keys): + * - bearer `Authorization: Bearer ` (OpenAI, OpenRouter, Cohere, Grok/xAI, Ollama) + * - fal-key `Authorization: Key ` (fal.ai — note the `Key ` prefix, NOT `Bearer`) + * - x-api-key `x-api-key: ` (Anthropic) + * - x-goog-api-key `x-goog-api-key: ` (Gemini, Gemini Interactions, Veo) + * - api-key `api-key: ` (Azure OpenAI static-key auth) + * - xi-api-key `xi-api-key: ` (ElevenLabs) + * + * Note on Azure: Azure OpenAI also supports Microsoft Entra ID `Authorization: + * Bearer ` OAuth. aimock only ever injects the static `api-key` header, + * and a real Entra bearer token from the caller is never dummy-prefixed, so it + * is always forwarded verbatim (the caller overrides aimock). + */ +type AuthScheme = + | { kind: "bearer" } // Authorization: Bearer + | { kind: "fal-key" } // Authorization: Key + | { kind: "x-api-key" } // x-api-key: + | { kind: "x-goog-api-key" } // x-goog-api-key: + | { kind: "api-key" } // api-key: + | { kind: "xi-api-key" }; // xi-api-key: + +const PROVIDER_AUTH_SCHEMES: Partial> = { + // Bearer-token providers. + openai: { kind: "bearer" }, + openrouter: { kind: "bearer" }, + cohere: { kind: "bearer" }, + grok: { kind: "bearer" }, // xAI + ollama: { kind: "bearer" }, // Ollama Cloud / bearer-gated Ollama servers + // Anthropic. + anthropic: { kind: "x-api-key" }, + // Google (Gemini + Veo) use the x-goog-api-key header scheme. + gemini: { kind: "x-goog-api-key" }, + "gemini-interactions": { kind: "x-goog-api-key" }, + veo: { kind: "x-goog-api-key" }, + // Azure OpenAI static-key auth. + azure: { kind: "api-key" }, + // ElevenLabs. + elevenlabs: { kind: "xi-api-key" }, + // fal.ai — uses the `Key ` prefix on Authorization, NOT `Bearer `. + fal: { kind: "fal-key" }, +}; + +/** + * Case-insensitively delete a header from a forward-header map, returning the + * value that was present (if any). Header maps preserve the original casing of + * incoming headers, so we can't assume a canonical key. + */ +function takeHeader(headers: Record, name: string): string | undefined { + const lowerName = name.toLowerCase(); + let found: string | undefined; + for (const key of Object.keys(headers)) { + if (key.toLowerCase() === lowerName) { + found = headers[key]; + delete headers[key]; + } + } + return found; +} + +/** + * Extract the caller's credential value for a given auth scheme from the + * forwarded headers, if present. For bearer, strips the `Bearer ` prefix. + */ +function readCallerCredential( + headers: Record, + scheme: AuthScheme, +): string | undefined { + if (scheme.kind === "bearer" || scheme.kind === "fal-key") { + const raw = scanHeader(headers, "authorization"); + if (raw === undefined) return undefined; + // Strip the scheme prefix (`Bearer ` for bearer, `Key ` for fal) if present, + // otherwise treat the whole value as the credential. + const prefix = scheme.kind === "fal-key" ? "Key" : "Bearer"; + const match = new RegExp(`^\\s*${prefix}\\s+(.+)$`, "i").exec(raw); + return match ? match[1].trim() : raw.trim(); + } + return scanHeader(headers, authHeaderName(scheme)); +} + +/** + * The wire header name a non-Authorization scheme carries its credential in. + * (bearer/fal-key both use `Authorization` and are handled separately.) + */ +function authHeaderName(scheme: AuthScheme): string { + switch (scheme.kind) { + case "x-api-key": + return "x-api-key"; + case "x-goog-api-key": + return "x-goog-api-key"; + case "api-key": + return "api-key"; + case "xi-api-key": + return "xi-api-key"; + case "bearer": + case "fal-key": + return "authorization"; + } +} + +/** Case-insensitive header lookup that does not mutate the map. */ +function scanHeader(headers: Record, name: string): string | undefined { + const lowerName = name.toLowerCase(); + for (const key of Object.keys(headers)) { + if (key.toLowerCase() === lowerName) return headers[key]; + } + return undefined; +} + +/** + * Decide whether aimock should override the caller's credential with its own + * built-in key. Override when the caller sent no credential OR sent a dummy + * placeholder (prefixed with the active marker). A real caller key is left + * untouched so the caller can always override aimock. + */ +function shouldOverride(callerCredential: string | undefined, marker: string): boolean { + if (callerCredential === undefined || callerCredential.length === 0) return true; + return callerCredential.startsWith(marker); +} + +/** + * Inject aimock's built-in upstream credential into the forwarded headers when + * appropriate (opt-in, backward-compatible). Mutates `forwardHeaders` in place + * and may mutate `target` (query-param schemes; none currently). + * + * Precedence: + * - No built-in key configured for this provider → no-op (forward verbatim). + * - Provider not a static-key scheme → no-op (Bedrock SigV4 / Vertex AI OAuth). + * - Caller credential absent or dummy-prefixed → inject built-in key. + * - Caller credential is a real key → forward it unchanged. + * + * @param forwardHeaders headers already built by buildForwardHeaders(req) + * @param target upstream URL (reserved for query-param auth schemes) + * @param providerKey which provider this request is routed to + * @param builtinKey aimock's configured key for this provider (if any) + */ +export function applyProviderAuth( + forwardHeaders: Record, + target: URL, + providerKey: RecordProviderKey, + builtinKey: string | undefined, +): void { + void target; // reserved for future query-param schemes (e.g. Gemini ?key=) + if (!builtinKey) return; // feature inert for this provider + + const scheme = PROVIDER_AUTH_SCHEMES[providerKey]; + if (!scheme) return; // signed/OAuth provider — never rewrite auth + + const marker = getDummyKeyMarker(); + const callerCredential = readCallerCredential(forwardHeaders, scheme); + if (!shouldOverride(callerCredential, marker)) return; // caller sent a real key + + // Strip any existing auth headers for this scheme (across casings) before + // setting aimock's key, so a dummy caller key can't linger alongside ours. + switch (scheme.kind) { + case "bearer": + takeHeader(forwardHeaders, "authorization"); + forwardHeaders["Authorization"] = `Bearer ${builtinKey}`; + break; + case "fal-key": + takeHeader(forwardHeaders, "authorization"); + forwardHeaders["Authorization"] = `Key ${builtinKey}`; + break; + case "x-api-key": + takeHeader(forwardHeaders, "x-api-key"); + forwardHeaders["x-api-key"] = builtinKey; + break; + case "x-goog-api-key": + takeHeader(forwardHeaders, "x-goog-api-key"); + forwardHeaders["x-goog-api-key"] = builtinKey; + break; + case "api-key": + takeHeader(forwardHeaders, "api-key"); + forwardHeaders["api-key"] = builtinKey; + break; + case "xi-api-key": + takeHeader(forwardHeaders, "xi-api-key"); + forwardHeaders["xi-api-key"] = builtinKey; + break; + } +} + +/** + * Read per-provider built-in keys from the environment. Returns undefined when + * no provider key is configured so the feature stays fully inert by default. + * + * Each wired static-key provider reads `AIMOCK_PROVIDER__KEY`. An + * empty-string value is treated as unset (existing truthiness pattern), keeping + * the feature inert. `gemini-interactions` is not read here: it reuses the + * Gemini key via the lookup-key remap in `proxyAndRecord`. Signed/OAuth + * providers (`bedrock`, `vertexai`) have no env var — aimock never rewrites + * their auth. + */ +export function readProviderKeysFromEnv( + env: NodeJS.ProcessEnv = process.env, +): Partial> | undefined { + const keys: Partial> = {}; + if (env.AIMOCK_PROVIDER_OPENAI_KEY) keys.openai = env.AIMOCK_PROVIDER_OPENAI_KEY; + if (env.AIMOCK_PROVIDER_ANTHROPIC_KEY) keys.anthropic = env.AIMOCK_PROVIDER_ANTHROPIC_KEY; + if (env.AIMOCK_PROVIDER_GEMINI_KEY) keys.gemini = env.AIMOCK_PROVIDER_GEMINI_KEY; + if (env.AIMOCK_PROVIDER_OPENROUTER_KEY) keys.openrouter = env.AIMOCK_PROVIDER_OPENROUTER_KEY; + if (env.AIMOCK_PROVIDER_COHERE_KEY) keys.cohere = env.AIMOCK_PROVIDER_COHERE_KEY; + if (env.AIMOCK_PROVIDER_GROK_KEY) keys.grok = env.AIMOCK_PROVIDER_GROK_KEY; + if (env.AIMOCK_PROVIDER_OLLAMA_KEY) keys.ollama = env.AIMOCK_PROVIDER_OLLAMA_KEY; + if (env.AIMOCK_PROVIDER_VEO_KEY) keys.veo = env.AIMOCK_PROVIDER_VEO_KEY; + if (env.AIMOCK_PROVIDER_AZURE_KEY) keys.azure = env.AIMOCK_PROVIDER_AZURE_KEY; + if (env.AIMOCK_PROVIDER_ELEVENLABS_KEY) keys.elevenlabs = env.AIMOCK_PROVIDER_ELEVENLABS_KEY; + if (env.AIMOCK_PROVIDER_FAL_KEY) keys.fal = env.AIMOCK_PROVIDER_FAL_KEY; + return Object.keys(keys).length > 0 ? keys : undefined; +} diff --git a/src/recorder.ts b/src/recorder.ts index eb299688..7b518271 100644 --- a/src/recorder.ts +++ b/src/recorder.ts @@ -20,6 +20,7 @@ import type { Logger } from "./logger.js"; import { collapseStreamingResponse, capturedRedactedData } from "./stream-collapse.js"; import { writeErrorResponse } from "./sse-writer.js"; import { resolveUpstreamUrl } from "./url.js"; +import { applyProviderAuth } from "./provider-auth.js"; import { getTestId, slugifyTestId, slugifyContext } from "./helpers.js"; import { DEFAULT_TEST_ID } from "./constants.js"; @@ -431,6 +432,12 @@ export async function proxyAndRecord( // Forward all request headers except hop-by-hop and client-set ones. const forwardHeaders = buildForwardHeaders(req); + // If aimock owns a built-in upstream key for this provider, inject it now + // (opt-in, backward-compatible). A real caller credential overrides it; an + // absent or dummy-prefixed caller credential is replaced. gemini-interactions + // reuses the Gemini key, mirroring the upstream-URL remap above. + applyProviderAuth(forwardHeaders, target, providerKey, record.providerKeys?.[lookupKey]); + const requestBody = rawBody ?? JSON.stringify(request); // Make upstream request diff --git a/src/types.ts b/src/types.ts index ab8f88ca..b7a0aebe 100644 --- a/src/types.ts +++ b/src/types.ts @@ -694,6 +694,19 @@ export type RecordProviderKey = export interface RecordConfig { providers: Partial>; + /** + * aimock's own built-in upstream API keys, keyed by provider. Opt-in and + * backward-compatible: when set for a provider, aimock injects the key on a + * fixture-miss passthrough IF the caller sent no credential or a dummy + * placeholder (see `applyProviderAuth`); a real caller key always overrides. + * Only static-key providers are eligible: OpenAI/OpenRouter/Cohere/Grok/Ollama + * (bearer), Anthropic (x-api-key), Gemini/Gemini-Interactions/Veo + * (x-goog-api-key), Azure OpenAI (api-key), ElevenLabs (xi-api-key), and fal + * (Authorization: Key). Signed/OAuth providers (Bedrock SigV4, Vertex AI + * OAuth) are never rewritten. Sourced from AIMOCK_PROVIDER_*_KEY env vars by + * the CLI (secrets stay out of `ps`). + */ + providerKeys?: Partial>; fixturePath?: string; /** Proxy unmatched requests without saving fixtures or caching in memory. */ proxyOnly?: boolean;