F5 integration: what exact governance claim can an input/output rail support?

[ljefford2-cmyk]

I would like to raise a system-boundary question prompted by PR #2105, “add F5 Guardrails integration”, and F5’s subsequent description of the integration as providing centralized governance across AI applications.

This is not a criticism of the contribution. The implementation appears to provide a useful and reasonably well-bounded function: sending input and output text to the F5 AI Guardrails API, receiving a scanning result, and blocking or allowing the corresponding interaction according to configuration.

The code and documentation actually make the immediate scope clearer than the broader product language does.

The integration scans:

• The full user message through an input rail.
• The full bot response through an output rail.

It also allows an operator to enable fail_open. When enabled, a service error may return an outcome of cleared together with a fail_open: true marker so that the mapping does not block the content.

That may be a legitimate availability option for some applications. But it raises an important claim-licensing question: “allowed because inspection failed” and “inspected and cleared” are not equivalent evidence, particularly for an execution agent.

The larger question is what we mean when we say AI governance.

My working definition is:

AI orchestration governance is the governance of how an organization turns mission, authority, professional expertise, authorized information, technology, and evidence into action—and how it independently determines whether that action worked and what should improve next.

The agent is not the system. The guardrail is not the system. The model is not the organization.

If the claim is only that identified input and output text was submitted to F5 under a particular configuration and received a particular result, the evaluated item may reasonably be limited to that pathway.

If the claim is that an enterprise AI application or agent was governed, the evaluated item must expand to include every component whose failure or unmodeled behavior could change the meaning of that claim. That may include:

• Application and model.
• NeMo and F5 runtime controls.
• Identity, session, purpose, and delegated authority.
• RAG, memory, files, instruction sources, and inter-agent messages.
• Routing, approvals, exceptions, and human handoffs.
• Tool selection, APIs, credentials, targets, and material action parameters.
• Downstream execution and the resulting external effect.
• Observation, evidence custody, replay, evaluation, and controlled change.

Nothing consequential is excluded by silence. Silence is reliance without declaration.

This distinction is especially important when moving from exploratory systems to execution systems.

An exploratory assistant may use content inspection as one useful safety layer while a human interprets its output. An execution agent requires substantially more. Before consequential dispatch, the system should be able to verify and bind:

• Actor and session identity.
• Approved purpose and work type.
• Authority source, scope, limits, expiration, and revocation state.
• Exact action, target, and material parameters.
• Applicable policy and evidence versions.
• Required approval or bounded delegation.
• Credential release and consumption receipt.
• Mutation and replay protection.
• Actual downstream effect and recovery state.

A scanner verdict may be evidence used by a gate. It is not, by itself, a grant of operational authority.

The F5 and NeMo controllers also remain inside the system under test. Their separation from one another or from the application is useful modularity, but it is not independent evaluation.

Control, observation, and evaluation perform different functions:

• Control constrains operation before the consequence becomes irreversible.
• Observation preserves what actually happened.
• Evaluation determines what claim the evidence permits.

The controller may produce evidence, but it cannot be the final authority over evidence proving its own coverage or correctness. Engineering independence is claim-relative, not merely organizational.

I would appreciate clarification from the maintainers and from @ciarancourtney on the following questions:

1. What precise engineering claim is this integration intended to support, and what is the complete evaluated item for that claim?
2. What independently proves that every relevant prompt, response, retrieval, memory operation, agent-to-agent exchange, tool call, API action, and external effect traverses the declared control path?
3. Which components or routes are outside the evaluated boundary, what assumptions justify their exclusion, and how are those assumptions monitored at runtime?
4. For execution agents, does the evidence bind identity, purpose, authority, policy version, controller result, approval, tool invocation, and actual downstream effect—or does it stop at the input/output scan?
5. When fail_open is enabled and a failed scan is represented with outcome: cleared plus a marker, what prevents downstream systems or audit summaries from treating that event as equivalent to a successful inspection? Does the supported governance claim automatically narrow when this occurs?
6. What happens during timeout, rate limiting, classifier uncertainty, malformed content, policy conflict, configuration mismatch, alternate routing, or partial failure? Can consequential execution be stopped before the consequence window closes?
7. Can either controller suppress, alter, delay, or selectively disclose the telemetry used to evaluate its own behavior? What independent observation or replay path can falsify its account?
8. Who independently determines both:
   • Whether the deployed operation conformed to its declared model.
   • Whether that declared model remains complete and appropriate as the operation changes.

A useful evaluation record might identify:

• Licensed claim.
• Evaluated item.
• Explicit assumptions.
• Measurand.
• Agent class and authority type.
• Model, rail, policy, and configuration versions.
• Policy hash.
• Expected and observed behavior.
• Fail-open or fail-closed status.
• Scope limitation.
• Independent evidence.
• Last adversarial rerun.

The narrow defensible claim appears to be:

The configured F5 policies were applied to identified input and output text that was shown to traverse the F5 rail under the recorded configuration.

That is useful runtime security evidence. It does not, without additional system-level evidence, establish governance of the complete enterprise operation in which that text participated.

I am not asking NeMo or F5 to solve the entire enterprise problem inside one integration. I am asking that the governance claim remain bounded by the system and evidence actually evaluated.

Lawrence Jeffords
End-User and Operator

[Rehanguards]

Spot-on observation. Pure input/output text rails can't guarantee execution safety for agentic tools because intent drift and prompt injection often happen inside nested tool payload arguments. True governance claims require a pre-execution deterministic policy layer sitting between the model output parser and the tool execution interface to validate actual function payloads before invocation.

[ljefford2-cmyk]

Thank you @Rehanguards for continuing the discussion. Your work on execution receipts and testing the policy engine contributes concrete implementation ideas to the broader evaluation problem.

The principle I am developing is that the entire organizational operating system—including the models, people, information, authority, procedures, controllers, tools, and evidence processes—must be part of the evaluation appropriate to the operation being governed. We need to establish whether those parts work correctly together, whether the work remains authorized, and whether the intended result actually occurs. The evaluation machinery itself belongs within that examination.

Anthropic’s recent disclosure provides a current example of why this matters: the reported problems involved the operating environment, evidence review, and stopping mechanism as well as model behavior. Its relevance here is the need to evaluate that connected operation.

I would welcome your perspective on the broader approach documented in my repository, including where your receipts and policy-testing work could contribute. The purpose is to help people and organizations use advanced AI effectively, with independent evidence of how the operation performs and what needs improvement.

[Rehanguards]

Thanks @ljefford2-cmyk — I spent some time reviewing the repository and the 3-Volume engineering corpus.

The core controlling rule—governance language must never outrun implementation evidence—is spot-on and desperately needed in the agentic AI space.

Looking at Volume 2 (Evaluate Reality), there is a clean architectu
ral alignment with the FastAPI/ASGI execution layer:

• Bridging [S] to [P]: Your specification for tamper-evident receipts and bounded execution gates maps directly to sub-millisecond pre-execution inspection at the gateway level.
• Deterministic Telemetry: By intercepting tool call payloads before invocation, we can emit structured execution receipts (capturing input hashes, policy verdicts, and routing flags) to feed your downstream evaluation machinery with verifiable proof.

I’m aligning our middleware’s audit schema with this Volume 2 evidence model. Looking forward to continuing the discussion as these implementation primitives mature!

[Rehanguards]

Hey Lawrence, tried sending over an email regarding DRNT Vol 2 implementation partnership, but your domain mail server bounced it with a temporary 451 rate-limit error. Let me know if there's an alternative email or if we can connect here!

[ljefford2-cmyk]

Thank you @Rehanguards for your consideration and for taking the time to review the work.

I am an old man approaching the end of a long working life spent with systems, organizations, and the people who make them work. I am simply sharing lessons handed down to me by many people much smarter than I am.

Working with multiple frontier AI models in an orchestrated process, under my direction, has helped me put those lessons into words that others can understand, examine, and learn from.

My partnership is with the entire AI community. My hope is that this work helps people see the possibilities of these amazing tools as multipliers of human intelligence, augmenting our abilities rather than replacing them, and improving productivity, quality of life, and prosperity for those who put in the effort.

That is why I share the work freely under MIT. I must honor my existing organizational obligations and cannot undertake proprietary work or accept constraints on my ability to share this work openly.

I welcome your ideas and comments in my repository, where they can contribute to that shared purpose.

Thanks’

Lawrence Jeffords

[Rehanguards]

Thank you so much for this inspiring response, Lawrence.

I deeply respect and salute your commitment to open-source, as well as your dedication to staying true to principles, organizational rules, and the broader AI community after a long and distinguished career in tech. It takes true integrity to share decades of wisdom freely under MIT for the benefit of everyone.

I am a young developer just starting out in the AI and software engineering space, building systems around AI security and deterministic guardrails. Seeing how thoughtfully you approach system architecture and human-AI collaboration is a huge learning experience for me.

Since you have decades of hands-on experience navigating systems, organizations, and tech evolutions:

1. What fundamental advice would you give to a young builder trying to gain credibility, find their first real clients, and build something sustainable in today's fast-moving AI landscape?
2. How can young engineers like me best contribute to your repository and learn from your architecture choices?

I will actively follow your repository, review the discussions, and look forward to contributing through pull requests and open discussions. It is an honor to learn from veterans like you.

Respect and warm regards,
Rehan