Skip to content

0.0.80

Choose a tag to compare

View all tags
@github-actions github-actions released this 30 Jun 13:42
0.0.80
4d1e635

Otoroshi LLM Extension v0.0.80

This release turns Otoroshi into a governed control plane for the agentic web: full Agent-to-Agent (A2A) support both ways, a complete MCP governance layer — reusable Virtual Servers, a standard MCP Registry, Zero-Trust controls, OAuth hardening and an OpenAPI-to-MCP connector — plus Rampart, a local (no-LLM, no-network) PII guardrail, and real-time MCP & LLM metrics.

What's New

A2A — Agent-to-Agent support — [#179]

Otoroshi now speaks the A2A protocol (started by Google, now under the Linux Foundation) both ways, with zero glue code:

  • A2A Servers (new entity) — publish any Otoroshi AI Agent or workflow as a standards-compliant A2A agent. Otoroshi generates the Agent Card (/.well-known/agent-card.json), handles discovery, streams responses and tracks stateful tasks over JSON-RPC. The agent keeps everything it already has: provider/model, tools & MCP connectors, guardrails, memory.
  • A2A Connectors (new entity) — point Otoroshi at any remote A2A agent and its skills instantly become tools your local agents can call, exactly like MCP connectors but for whole agents. Implemented across every tool-capable provider (OpenAI, Azure OpenAI, Anthropic, Mistral, Groq, Cohere, Ollama, xAI).
  • Because every exchange flows through the gateway, your security (API keys, JWT, OAuth2, mTLS, IP rules), outgoing auth (Bearer / API key / Basic / OAuth2 client-credentials, secrets from the vault), guardrails, budgets/cost tracking and observability apply to every exposed and consumed agent.

MCP Virtual Servers (new entity)

The whole MCP exposition complexity now lives in one reusable, persisted entity that the exposition plugins reference with server_ref — define it once, reuse it across routes, override fields inline when needed. A virtual server bundles:

  • the tool functions (refs) and MCP connectors (mcp_refs) to aggregate;
  • filtering (include/exclude patterns, allow/disallow JsonPath rules);
  • OAuth 2.0 security, scope → tool authorization (RBAC), audience binding;
  • Meta mode (tool virtualization) with optional semantic search;
  • per-tool rate limiting & result caching;
  • managed resources & prompts (inline), and item overlays;
  • Zero-Trust controls and registry publication (see below);
  • opt-in audit events.

MCP Registry

Publish your governed virtual servers to the standard MCP registry format, so registry-aware MCP clients discover the sanctioned servers automatically — and connect through the gateway (OAuth and Zero-Trust apply end-to-end).

  • Each virtual server projects 1:1 to an official server.json (schema 2025-12-11) exposed via GET /v0/servers, with cursor pagination.
  • Publication is a single published switch governed by Otoroshi RBAC — that switch is your approval gate; deprecated keeps an entry listed but flagged.
  • Stand up as many registries as you like, each a curated slice: all published servers, selected refs, by tag, or by route's tenant.
  • "Detect from route" assistant auto-discovers the public streamable-HTTP URL and the detected auth (OAuth / apikey / mTLS).

Zero-Trust MCP controls

New defense-in-depth layer on virtual servers (zero_trust):

  • Anti-rug-pull pinning — pin tool definitions and detect/refuse silent changes after approval.
  • Tool-poisoning / prompt-injection scanning of tool descriptions and results.
  • PII / secrets redaction on MCP traffic.

MCP OAuth hardening

  • Audience validation (RFC 8707) — bind tokens to the intended MCP resource.
  • Opaque token introspection (RFC 7662) — validate non-JWT access tokens against an introspection endpoint.
  • Scope → tool mapping — fine-grained, per-tool authorization from token scopes.

OpenAPI → MCP connector (no MCP server needed)

A new connector turns any OpenAPI / Swagger spec into MCP tools and calls the target REST API directly — no MCP server to run. It joins the existing stdio / SSE / Streamable HTTP / WebSocket / meta transports.

Rampart — local PII guardrail (no LLM, no network) — [#183]

A new guardrail that detects and redacts Personally Identifiable Information locally, in-process, using the bundled open-source Rampart ONNX model (a fine-tuned MiniLM token classifier) plus a deterministic recognizer layer — a fast, private, deterministic alternative to the LLM-based pif guardrail (~a few ms on CPU).

  • Three layers: high-precision regex recognizers (EMAIL, URL, IP, SSN, Luhn-checked CREDIT_CARD) → the 35-label BIO model (names, addresses, phones, gov IDs, bank info…) → de-dup & action.
  • Actions: redact (rewrites the prompt downstream), deny, or flag.
  • Stable placeholders ([GIVEN_NAME_1], [SSN_1], …) keep multi-turn context coherent; optional re-inflation restores real values in the (non-streamed) response while the LLM never sees them.
  • Also shipped as two HTTP body-redaction plugins (request & response) usable on any Otoroshi route, and as a workflow function. Plugins fail open on error.

Real-time MCP & LLM metrics

MCP calls — and the LLM gateway internals (decorators, provider fallback, circuit breaker, guardrails, OCR, search, workflows) — now feed Otoroshi's live metrics subsystem, exported over Prometheus / JSON. Unlike audit events, metrics are always on when Otoroshi metrics are enabled (no opt-in flag required).

  • New / enhanced audit events: McpAudit (servers Otoroshi exposes) and McpClientAudit (upstream calls through connectors), both correlated with LLMUsageAudit via a shared request_id.

Search Engines — Exa provider & RAG compatibility — [#177]

  • New Exa search-engine provider, alongside the providers shipped in 0.0.79.
  • Search Engines are now compatible with EmbeddingStore + EmbeddingModel, so a Search Engine can be used as a RAG source.

OCR budgets & auditing

  • Budget support for OCR calls (counted against defined budgets, with dashboard wiring).
  • Audit events for OCR operations.

Improvements & fixes

  • Moderation models: support a base_url override ([#181]) and array input ([#182]).
  • Fix Scaleway provider reporting 0/0/0 token usage on streamed responses ([#178]).
  • Updated Mistral base URL handling.
  • More tolerant input parsing for workflow functions / model inputs.
  • MCP: resource bug fix, metadata pass-through, per-function MCP metadata, audit-event coverage.
  • Dependency: bump Otoroshi 17.16.1 → 17.17.0.

Release Infos

  • the documentation is available here
  • release is available here

Contributors

Contributors

mathieuancelin
Assets 4
Loading