Proxy-sidecar: cooperative egress mode opt-out for platforms without iptables (ROSA HCP)

[akram]

Problem

On ROSA HCP (and likely other managed OpenShift platforms), agent pods in proxy-sidecar mode are stuck in Init:CrashLoopBackOff because the proxy-init init container cannot initialize iptables. Neither iptables-legacy nor iptables (nft) works on managed OpenShift nodes (SELinux, missing iptable_nat module, no MachineConfig access).

proxy-sidecar is the rhoai-3.5 default (kagenti#1428, kagenti-operator#361). This blocks agent deployments on ROSA HCP.

Agreed direction

Per Slack discussion with Hai Huang (thread): add explicit cooperative mode opt-out for platforms where iptables is unavailable. Keep enforce-redirect as the default. Document security limitations.

Upstream config/runtime changes tracked in kagenti-extensions#519.

Proposed solution

Add egressEnforcement to AgentRuntime (CRD field and/or annotation):

Value	Behavior
enforce-redirect (default)	Current — mutating webhook injects proxy-init, iptables REDIRECT
none	Skip proxy-init; set HTTP_PROXY / HTTPS_PROXY only (cooperative)

Operator behavior when none

1. Mutating webhook does not inject proxy-init init container
2. Authbridge sidecar still injected with forward/reverse proxy listeners
3. HTTP_PROXY env vars point agent traffic at authbridge
4. Cooperative agents → authenticated egress via token exchange
5. Non-cooperative agents → direct egress → destination inbound AuthBridge rejects (401)

Security model (must be documented)

What cooperative mode provides:

• HTTP_PROXY path for agents that respect proxy settings
• Inbound AuthBridge JWT validation on destination services
• Egress NetworkPolicy can restrict L4 destinations (operator already supports agent NetworkPolicy)

Known limitations:

• NetworkPolicy is L4 — cannot distinguish authbridge vs agent traffic within the same pod network namespace
• Non-cooperative agents that target allowed destinations can exfiltrate without token exchange
• Admin must explicitly opt in and acknowledge the trade-off

Scope (this repo)

• Add egressEnforcement field to AgentRuntime API (CRD + webhook validation)
• Mutating webhook: conditional proxy-init injection (skip when none)
• Helm/ConfigMap default: enforce-redirect
• E2E or integration test: pod spec without proxy-init when opted out
• Document platform guidance (ROSA HCP / managed OpenShift) and security limitations

Acceptance criteria

1. egressEnforcement: none on AgentRuntime → agent pods reach Running on ROSA HCP (no proxy-init)
2. Default unchanged (enforce-redirect) — no behavior change on iptables-capable platforms
3. Security limitations documented at opt-in point
4. Validated on ROSA HCP: cooperative egress works; non-cooperative fails at destination inbound AuthBridge

References

• Red Hat Jira: RHAIENG-5702
• Upstream: kagenti-extensions#519
• Related: kagenti-extensions#502, kagenti-extensions#407
• Slack: enforce-redirect thread