[RFC] Native Payload Gating Layer: In-Process Architecture & Performance Baseline

[Rehanguards]

1. Overview & Architecture Strategy

To maintain sub-millisecond execution budgets while preventing agent payload manipulation, we propose an In-Process Inspection Layer within the gateway pipeline.

• Deployment Model (In-Process vs. Sidecar):

  • In-Process (Recommended): Evaluates payload buffers directly inside the request execution pipeline before stream consumption. Eliminates extra IPC/network hop latency inherent to sidecar proxies.
  • Sidecar (Optional): Can be deployed as an independent proxy container for language-agnostic topologies.

• Pipeline Placement:
  Sits directly after request authentication and prior to LLM/Tool payload dispatch:
  Client Request → Auth → Cloakwall Inspection Gate → LLM / Tool Execution Engine

• Configuration & Execution Modes:

  • BLOCK (Strict Mode): Immediately short-circuits execution and returns 403 Forbidden with an audit receipt payload upon argument drift or rule violation.
  • WARN (Audit Mode): Injects execution receipts into response headers (X-Guardrail-Receipt-ID) without blocking upstream execution.

---

2. Benchmark Setup & Baseline Analysis

Environment & Hardware

• Runner Environment: Windows x64 (Node.js harness over IPv4 loopback 127.0.0.1)
• Tooling: autocannon (50 concurrent connections, 10s duration, 14,000+ total requests)
• Payload Shape: Deeply nested JSON tool calls (3–5 levels of nested JSON arguments mimicking complex multi-agent parameters).

Comparative Overhead Analysis

• Bare Baseline Gateway Routing (No Extra Inspection):
  • Avg Latency: ~5.1 ms
  • Throughput: ~1,420 req/sec
• With In-Process Payload Inspection (Nested Gating Active):
  • Avg Latency: 7.52 ms (P50: 7 ms | P99: 20 ms)
  • Throughput: 1,246.1 req/sec
  • Overhead Added: ~2.4 ms processing delay under heavy payload depth.

---

Next Steps

Looking forward to feedback from the maintainers and community on refining the config schemas and middleware boundary hooks!

[rivamarco]

Hi @Rehanguards , thanks for the RFC and for the effort you've put into this.

Since the proposal points at Cloakwall as the inspection layer, we read through
that codebase before replying.

The blocker for us is licensing, and it isn't something we can engineer around.
Cloakwall is AGPL-3.0 and this gateway is Apache-2.0. Taking it as a dependency,
linking it in-process, or porting parts of it into our code would all pull AGPL
obligations onto the gateway and onto everyone deploying it.

On the technical side: the gateway already does PII detection and redaction
in-process. guardrails/guardrail_redact.py runs Presidio with spaCy models for
en/it, plus an optional Azure Health De-identification backend for PHI, with no
sidecar involved. So the deployment simplification the RFC is aimed at is
something we already have, and Presidio's NLP models cover cases regex can't:
names and conditions in free text, mainly.

The part of your RFC we find more interesting is the integrity angle, what you
called argument drift and payload manipulation. That's a genuinely different
problem from PII detection: not "is there sensitive data in this text" but "is
this tool call legitimate, given what the client actually declared". A gateway is
arguably the right place for it, since we see the declared tool definitions and
the calls the model emits in the same request, where a client only sees its own
half.

If you want to sketch something in that direction, a small standalone PoC would
be easier for us to look at than a full integration, validating tool calls
against the schema the client declared is the part we'd be most curious about.
It also sidesteps the licence issue, since it wouldn't be based on Cloakwall.

Thanks again for the interest in the project.

[Rehanguards]

Hi @rivamarco,

That makes complete sense. Licensing integrity is critical, and I fully respect keeping the gateway pure Apache-2.0 without pulling in AGPL obligations from Cloakwall.

You hit the nail on the head regarding the distinction: text-level PII redaction is a different layer than tool call schema validation and argument drift prevention. Catching malicious or mismatched tool calls at the gateway level—where both declared schemas and model outputs intersect—is the most effective boundary.

I will assemble a lightweight, standalone, Apache-2.0 compliant PoC specifically for Tool Call Schema & State Validation (completely independent of Cloakwall).

It will focus purely on:

1. Validating model-emitted tool parameters against the client's declared JSON schema/signature.
2. Detecting structural anomalies/argument drift before payload execution.

I'll share the PoC repository link here once ready for you to review!