feat(adapters): caching interceptor

fern/versions/latest/pages/model-server/adapters-caching.mdx

@@ -0,0 +1,26 @@
+---
+title: "Caching Interceptor"
+description: "Disk-backed cache that short-circuits the chain on cache hit."
+position: 5
+---
+
+Disk-backed cache keyed by a SHA-256 hash of the canonicalized request body (+ optional session prefix). On cache hit, the interceptor short-circuits the chain and returns the stored response without invoking the upstream.
+
+| Name | Stage | Purpose |
+|------|-------|---------|
+| `caching` | request → response | Disk-backed cache (sqlite). Hits short-circuit upstream. |
+
+### `caching`
+
+```yaml
+- name: caching
+  config:
+    cache_dir: /var/cache/gym-adapter
+    bypass: false
+```
+
+Cache keys include the session prefix when present (via `ctx.extra["session_id"]`), so the same body in different sessions does not collide.
+
+<Note>
+Per-replica proxy or server instance means per-replica disk cache: if multiple replicas share the same `cache_dir`, sqlite write contention is real. Configure a per-replica path (e.g. `cache_dir: /tmp/adapter_cache_${pod_id}`) or accept the race.
+</Note>

fern/versions/latest/pages/model-server/adapters.mdx

@@ -0,0 +1,68 @@
+---
+title: "Adapter Middleware"
+description: "Interceptor-based middleware framework for Model, Agent, and Resources servers."
+position: 3
+---
+
+Adapter middleware adds an interceptor chain to a server's request/response path. Each interceptor can observe or mutate the payload without the host server changing. Use it to inject system prompts, drop unsupported params, cache responses, count turns, normalize reasoning fields, log tokens, and so on.
+
+`adapters` is opt-in. It is declared on `BaseResponsesAPIModelConfig`, `BaseResponsesAPIAgentConfig`, and `BaseResourcesServerConfig`, so every in-tree server inheriting from `SimpleResponsesAPIModel` / `SimpleResponsesAPIAgent` / `SimpleResourcesServer` accepts an `adapters` block automatically. Omitting it leaves behavior identical to the base server.
+
+## Quickstart
+
+Add an `adapters` list to any server config:
+
+```yaml
+policy_model:
+  responses_api_models:
+    openai_model:
+      openai_base_url: https://api.openai.com/v1
+      openai_api_key: ...
+      openai_model: gpt-4.1
+      adapters:                       # ← toggle: omit or null = OFF
+        - {name: logging, config: {}}
+```
+
+This PR ships the **framework** (`AdapterPipeline`, `InterceptorRegistry`, `install_middleware`, `start_adapter_proxy`) plus two built-in interceptors:
+
+| Name | Stage | Purpose |
+|------|-------|---------|
+| `logging` | request + response | Log request body keys and response status/latency. Canonical "did the chain fire?" probe. |
+| `endpoint` | request → response | Drive the upstream HTTP call directly. **Only used by `start_adapter_proxy`** (standalone host mode); forbidden inside `install_middleware`. |
+
+Additional interceptor families (observability, caching, request rewriting) ship in follow-on PRs.
+
+## Host modes
+
+The same `AdapterPipeline` runs in either of two hosting modes:
+
+**Middleware mode** — `install_middleware(app, adapters)` attaches the pipeline to an existing FastAPI app via `app.middleware("http")`. The host server's own routing performs the upstream call via `call_next`. Used by every Model/Agent/Resources server when `adapters` is set on its config.
+
+**Proxy mode** — `start_adapter_proxy(upstream_url, adapters)` launches a localhost uvicorn that hosts the pipeline with its own forwarding logic. Used by agents that bring their own SDK client (e.g. `claude_code_agent` with `anthropic_base_url`); the agent points its SDK's `*_BASE_URL` at the proxy URL via the `adapter_proxy` field on `BaseResponsesAPIAgentConfig`.
+
+## Path-Based Session Scoping
+
+Posts to `/s/<hex-id>/<path>` have the prefix stripped before forwarding, and `<hex-id>` is recorded as `ctx.extra["session_id"]`. Follow-on interceptors that key per-session (e.g. `turn_counter`, `caching`) use this id.
+
+## Custom Interceptors
+
+Register a class at runtime via `InterceptorRegistry.register`:
+
+```python
+from nemo_gym.adapters import InterceptorRegistry
+
+InterceptorRegistry.register("my_interceptor", "myproject.adapters.my_interceptor")
+```
+
+The target module must expose a class named `Interceptor` that subclasses one of `RequestInterceptor`, `RequestToResponseInterceptor`, or `ResponseInterceptor` from `nemo_gym.adapters.types`. The class is instantiated with the YAML `config` dict as kwargs.
+
+## Configuration Reference
+
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `adapters` | `list[dict] \| null` | `null` | Ordered interceptor specs on each server config. Each entry is `{name: <str>, config: <dict>}`. `null` or `[]` disables the middleware. |
+| `adapter_proxy` | `AdapterProxyConfig \| null` | `null` | On `BaseResponsesAPIAgentConfig` only. Configures a localhost proxy in front of an external inference upstream. Fields: `upstream_url`, `adapters`, `host` (default `127.0.0.1`), `port` (default `0` → kernel-assigned), `request_timeout`, `unsafe_allow_remote`. |
+
+<Warning>
+`start_adapter_proxy` refuses any `host` other than `127.0.0.1`/`localhost` unless you pass `unsafe_allow_remote=True`. The proxy forwards the client's `Authorization` header verbatim to the upstream, so binding to `0.0.0.0` would leak the upstream API key to any caller on the network.
+</Warning>

fern/versions/latest/pages/model-server/index.mdx

@@ -32,3 +32,9 @@ Self-hosted inference with vLLM for maximum control.
 [Model Server Fields](/reference/configuration#model-server-fields) for server configuration syntax and fields.
 
 </Note>
+
+## Middleware
+
+Model servers can attach an interceptor chain that observes or mutates request and response payloads — useful for logging, caching, system-prompt injection, turn budgeting, reasoning-field normalization, and similar cross-cutting hooks.
+
+See [Adapter Middleware](/model-server/adapters) for the framework, built-in interceptors, and configuration syntax.

nemo_gym/adapters/__init__.py

@@ -0,0 +1,58 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Gym adapter framework — interceptor-based middleware for responses_api_models.
+
+Public API:
+    install_middleware(app, interceptor_specs)  — attach pipeline to a FastAPI app
+    start_adapter_proxy(upstream_url, adapters) — host pipeline as a localhost uvicorn
+    AdapterPipeline                             — the async interceptor chain
+    InterceptorRegistry                         — name → class resolution
+"""
+
+from nemo_gym.adapters.middleware import install_middleware
+from nemo_gym.adapters.pipeline import AdapterPipeline
+from nemo_gym.adapters.proxy import ProxyHandle, start_adapter_proxy
+from nemo_gym.adapters.registry import InterceptorRegistry
+from nemo_gym.adapters.types import (
+    AdapterProxyConfig,
+    AdapterRequest,
+    AdapterResponse,
+    GracefulError,
+    InterceptorContext,
+    InterceptorSpec,
+    RequestInterceptor,
+    RequestToResponseInterceptor,
+    ResponseInterceptor,
+    Stage,
+)
+
+
+__all__ = [
+    "AdapterPipeline",
+    "AdapterProxyConfig",
+    "AdapterRequest",
+    "AdapterResponse",
+    "GracefulError",
+    "InterceptorContext",
+    "InterceptorRegistry",
+    "InterceptorSpec",
+    "ProxyHandle",
+    "RequestInterceptor",
+    "RequestToResponseInterceptor",
+    "ResponseInterceptor",
+    "Stage",
+    "install_middleware",
+    "start_adapter_proxy",
+]

nemo_gym/adapters/cache/__init__.py

@@ -0,0 +1,14 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

nemo_gym/adapters/cache/disk_cache.py

@@ -0,0 +1,112 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import annotations
+
+import asyncio
+import hashlib
+import json
+import logging
+import os
+import sqlite3
+import threading
+import time
+from typing import Any
+
+
+logger = logging.getLogger(__name__)
+
+_RELEVANT_KEYS = ("model", "messages", "tools", "temperature", "max_tokens", "top_p", "seed")
+
+_SCHEMA = """
+CREATE TABLE IF NOT EXISTS cache_v1 (
+    key        TEXT PRIMARY KEY,
+    value      TEXT NOT NULL,
+    created_at REAL NOT NULL
+);
+"""
+
+_UPSERT = """
+INSERT INTO cache_v1 (key, value, created_at) VALUES (?, ?, ?)
+ON CONFLICT(key) DO UPDATE SET value=excluded.value, created_at=excluded.created_at;
+"""
+
+
+class DiskCache:
+    def __init__(self, cache_dir: str) -> None:
+        os.makedirs(cache_dir, exist_ok=True)
+        self._db_path = os.path.join(cache_dir, "adapter_cache.db")
+        self._lock = threading.Lock()
+        self._init_db()
+
+    def _init_db(self) -> None:
+        try:
+            with self._lock:
+                conn = sqlite3.connect(self._db_path)
+                try:
+                    conn.execute("PRAGMA journal_mode=WAL")
+                    conn.execute("PRAGMA busy_timeout=5000")
+                    conn.execute(_SCHEMA)
+                    conn.commit()
+                finally:
+                    conn.close()
+        except sqlite3.Error:
+            logger.warning("Failed to initialize cache DB at %s", self._db_path, exc_info=True)
+
+    @staticmethod
+    def cache_key(body: dict[str, Any], *, session_prefix: str = "") -> str:
+        canonical: dict[str, Any] = {}
+        for k in _RELEVANT_KEYS:
+            if k in body:
+                canonical[k] = body[k]
+        if "extra_body" in body and isinstance(body["extra_body"], dict):
+            canonical["extra_body"] = body["extra_body"]
+        raw = json.dumps(canonical, sort_keys=True, ensure_ascii=False)
+        if session_prefix:
+            raw = session_prefix + "|" + raw
+        return hashlib.sha256(raw.encode()).hexdigest()
+
+    def _get_sync(self, key: str) -> dict | None:
+        try:
+            with self._lock:
+                conn = sqlite3.connect(self._db_path)
+                try:
+                    row = conn.execute("SELECT value FROM cache_v1 WHERE key = ?", (key,)).fetchone()
+                finally:
+                    conn.close()
+            if row is None:
+                return None
+            return json.loads(row[0])
+        except (sqlite3.Error, json.JSONDecodeError):
+            logger.warning("Cache get failed for key=%s", key[:16], exc_info=True)
+            return None
+
+    def _set_sync(self, key: str, value: dict) -> None:
+        try:
+            serialized = json.dumps(value, ensure_ascii=False)
+            with self._lock:
+                conn = sqlite3.connect(self._db_path)
+                try:
+                    conn.execute(_UPSERT, (key, serialized, time.time()))
+                    conn.commit()
+                finally:
+                    conn.close()
+        except sqlite3.Error:
+            logger.warning("Cache set failed for key=%s", key[:16], exc_info=True)
+
+    async def get(self, key: str) -> dict | None:
+        return await asyncio.to_thread(self._get_sync, key)
+
+    async def set(self, key: str, value: dict) -> None:
+        await asyncio.to_thread(self._set_sync, key, value)

nemo_gym/adapters/interceptors/__init__.py

@@ -0,0 +1,15 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Built-in interceptors for the adapter pipeline."""

nemo_gym/adapters/interceptors/caching.py

@@ -0,0 +1,67 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import annotations
+
+import logging
+
+from nemo_gym.adapters.cache.disk_cache import DiskCache
+from nemo_gym.adapters.types import (
+    AdapterRequest,
+    AdapterResponse,
+    RequestToResponseInterceptor,
+    ResponseInterceptor,
+)
+
+
+logger = logging.getLogger(__name__)
+
+
+class Interceptor(RequestToResponseInterceptor, ResponseInterceptor):
+    stream_safe = False
+
+    def __init__(self, cache_dir: str, *, bypass: bool = False) -> None:
+        self._bypass = bypass
+        self._cache = DiskCache(cache_dir)
+
+    async def intercept_request(
+        self,
+        req: AdapterRequest,
+    ) -> AdapterRequest | AdapterResponse:
+        if self._bypass:
+            return req
+        session_prefix = req.ctx.extra.get("session_id", "")
+        key = DiskCache.cache_key(req.body, session_prefix=session_prefix)
+        hit = await self._cache.get(key)
+        if hit is not None:
+            logger.debug("cache hit key=%s", key[:16])
+            req.ctx.extra["cache_hit"] = True
+            return AdapterResponse(
+                status_code=200,
+                headers={},
+                body=hit,
+                latency_ms=0.0,
+                ctx=req.ctx,
+            )
+        req.ctx.extra["cache_key"] = key
+        return req
+
+    async def intercept_response(self, resp: AdapterResponse) -> AdapterResponse:
+        key = resp.ctx.extra.get("cache_key")
+        if key is None or not resp.ok:
+            return resp
+        if isinstance(resp.body, dict):
+            await self._cache.set(key, resp.body)
+        resp.ctx.extra.pop("cache_key", None)
+        return resp