Optional VM egress MITM proxy with mock-secret header rewriting by sjmiller609 · Pull Request #134 · kernel/hypeman

sjmiller609 · 2026-03-08T17:41:32Z

Summary

This PR adds an optional, default-off egress MITM proxy mode for Hypeman VMs so workloads can run with mock secrets in-VM while real secrets stay on the host.

When enabled per instance, Hypeman now:

Starts/uses a host-side HTTP/HTTPS MITM proxy on the VM bridge gateway.
Injects proxy env vars into the guest (HTTP_PROXY / HTTPS_PROXY, lower-case variants).
Installs proxy CA material in guest init so TLS MITM can be trusted by system components.
Rewrites outbound HTTP header values by replacing configured mock literals with real values from host env vars.
Enforces egress path on Linux using per-instance iptables FORWARD rules for TCP ports 80/443 so direct internet egress bypass is rejected.

Why

This enables safer execution of untrusted or lower-trust workloads in VMs without placing real secrets inside guest env/config, while still allowing authenticated outbound API traffic.

API / Config changes

CreateInstanceRequest now supports top-level egress_proxy:
- enabled: bool
- mock_to_real_env_var: map[string]string (mock literal -> host env var name)
OpenAPI schema updated accordingly.
Instance domain model now persists optional EgressProxyConfig.

Implementation details

New module: lib/egressproxy/
- Host CA generation/load and on-the-fly cert signing.
- HTTP proxying and HTTPS CONNECT MITM handling.
- Header replacement policy resolved per source VM IP.
- Linux enforcement helpers for egress gating.
- Behavior-focused README.
Lifecycle wiring in instances manager:
- Register proxy policy/enforcement on create/start/restore when enabled.
- Remove policy/enforcement on stop/standby/delete and rollback paths.
Guest init support:
- New guest config section for egress proxy settings and CA PEM.
- CA installation call added in both exec and systemd modes.

Tests

Added integration test:

TestEgressProxyRewritesHTTPSHeaders
- Boots a VM with egress proxy mode enabled.
- Sends HTTPS request with mock secret in Authorization header.
- Verifies upstream receives rewritten real secret value.

Validation run

Executed on deft-kernel-dev as root:

sudo -n /usr/local/go/bin/go test ./cmd/api/api -run TestDoesNotExist -count=1
sudo -n /usr/local/go/bin/go test ./lib/instances -run TestEgressProxyRewritesHTTPSHeaders -count=1 -v
sudo -n /usr/local/go/bin/go test ./... -run TestDoesNotExist -count=1

All passed for this change set.

Notes

Enforcement currently targets default HTTP/HTTPS ports (80/443) by design.
Header replacement applies to HTTP headers only (not bodies).

Note

High Risk
Introduces a host-side MITM proxy and Linux iptables enforcement that alters VM outbound networking and handles real secret material, so misconfiguration or bugs could impact security and connectivity.

Overview
Adds an optional, default-off egress MITM proxy mode for instances, including new API fields network.egress (with enforcement mode) and credentials to keep real secrets host-side while the guest receives mock-* placeholders.

Wires the feature through instance lifecycle (create/start/restore and cleanup on stop/standby/delete): registers per-instance proxy + enforcement, regenerates config disks to inject proxy settings and mock env vars, and persists new metadata (NetworkEgress, Credentials).

Introduces lib/egressproxy (CA generation + per-host cert signing, CONNECT MITM, HTTPS-only header rewriting with domain allowlists, and Linux iptables egress blocking) plus guest-init support to install the proxy CA, expands test coverage (unit + VM integration), and updates prewarm/e2e scripts and tests to support registry mirroring and new OpenAPI fields.

^{Written by Cursor Bugbot for commit a25a59b. This will update automatically on new commits. Configure here.}

Add a new host-side egress proxy module that supports HTTP/HTTPS interception and per-instance header secret substitution from mock values to real host environment secrets. Wire proxy lifecycle into instance create/start/restore/stop/standby/delete flows, inject guest proxy settings via config disk, and install proxy CA material in guest init. Add Linux egress enforcement rules to require proxy path for outbound 80/443 traffic, document behavior in lib/egressproxy/README.md, and add an integration test validating HTTPS header rewrite end to end.

github-actions · 2026-03-08T17:41:52Z

✱ Stainless preview builds

This PR will update the hypeman SDKs with the following commit message.

feat: add optional VM egress MITM proxy with mock-secret header rewriting

⚠️

hypeman-typescript studio · code

Your SDK build had at least one "error" diagnostic.
generate ❗ → build ✅ → lint ✅ → test ✅
npm install https://pkg.stainless.com/s/hypeman-typescript/99c818225c466f8ca53ceb64f11a4c631082e2a9/dist.tar.gz

✅ hypeman-go studio · code

Your SDK build had at least one "note" diagnostic.
generate ✅ → build ⏭️ → lint ✅ → test ✅
go get github.com/stainless-sdks/hypeman-go@e8b721cbc709af739827929839e443981d636233

✅ hypeman-openapi studio · code

Your SDK build had at least one "note" diagnostic.
generate ✅

This comment is auto-generated by GitHub Actions and is automatically kept up to date as you push.
If you push custom code to the preview branch, re-run this workflow to update the comment.
Last updated: 2026-03-19 17:56:25 UTC

…n test Switch the new egress proxy integration test away from curlimages/curl:8.12.1 so it works with CI strict prewarm registry mirror. Use docker.io/library/nginx:alpine (already mirrored in CI) while keeping HTTPS header rewrite validation via curl.

cursor

Automated risk triage result: High risk.

Why this is high risk (from code diff evidence):

Introduces a new host-side HTTP/HTTPS MITM subsystem (lib/egressproxy/*) including dynamic cert signing, CONNECT interception, and header rewriting logic.
Adds Linux egress enforcement via host iptables FORWARD rules (lib/egressproxy/enforce_linux.go), which affects networking behavior and operational safety.
Wires proxy/enforcement into core VM lifecycle flows (create, start, restore, stop, standby, delete) across lib/instances/*.
Expands external API surface (openapi.yaml, lib/oapi/oapi.go, API request mapping) and guest init trust/bootstrap behavior (lib/system/init/*, lib/vmconfig/config.go).

Decision:

Code review is required.
No auto-approval (high-risk PRs are not approved by automation).

Reviewer assignment:

PR already has 2 reviewers requested, so no additional reviewers were added.

…roxy-secret-rewrite # Conflicts: # lib/instances/create.go # lib/instances/fork.go # lib/instances/types.go # lib/oapi/oapi.go # openapi.yaml

…roxy-secret-rewrite # Conflicts: # lib/instances/create.go # lib/instances/manager.go # lib/oapi/oapi.go

…roxy-secret-rewrite

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

…roxy-secret-rewrite

rgarcia

still seems like there's no hard api/control-plane error when mitm ca cert setup fails in the guest... seems not ideal to have to inspect guest logs to find that this failure. fine merging as is to keep things moving though

sjmiller609 · 2026-03-18T21:32:14Z

still seems like there's no hard api/control-plane error when mitm ca cert setup fails in the guest... seems not ideal to have to inspect guest logs to find that this failure. fine merging as is to keep things moving though

I'll take a look. I still need to review the changes it decided to do since the previous review, and I'll see if I can handle that too. thanks for the careful reviews.

Change installEgressProxyCA to return an error instead of silently logging and continuing. On failure, the guest init now emits a HYPEMAN-EXIT sentinel (code 78 / EX_CONFIG) with the failure reason and powers off the VM. This means: - The instance never reaches Running — it stays Initializing then transitions to Stopped - The error is visible via the API through the existing ExitCode / ExitMessage fields (parsed from the HYPEMAN-EXIT sentinel) - No new API fields or sentinel types needed — reuses the existing exit sentinel parsing in query.go The CA cert setup runs before HYPEMAN-PROGRAM-START is emitted, so the guest program starting is now gated on successful CA installation when egress proxy is enabled. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

sjmiller609 added 4 commits March 7, 2026 17:13

Stabilize integration network and ingress test flakes

e28a8f6

Speed up fork-from-running integration tests

e66e759

Reduce fork test agent wait latency

4433590

sjmiller609 commented Mar 8, 2026

View reviewed changes

Comment thread lib/egressproxy/service.go

sjmiller609 added 2 commits March 8, 2026 14:14

feat(egress-proxy): source secret rewrites from instance env

e3beda6

chore: remove agents notes from this PR

d56d8b5

sjmiller609 commented Mar 8, 2026

View reviewed changes

Comment thread lib/ingress/binaries_linux.go

sjmiller609 added 3 commits March 8, 2026 14:26

test(ci): mirror API/integration image refs and prewarm missing images

cbe7082

Add egress proxy enforcement modes with strict default

c58e231

Fix CI image pull flakes in API and e2e install tests

2d76140

sjmiller609 marked this pull request as ready for review March 8, 2026 21:54

sjmiller609 requested review from hiroTamada and rgarcia March 8, 2026 21:54

cursor Bot reviewed Mar 8, 2026

View reviewed changes

Comment thread lib/egressproxy/service.go

Comment thread lib/egressproxy/service.go

Comment thread cmd/api/api/test_prewarm_test.go

cursor Bot reviewed Mar 8, 2026

View reviewed changes

sjmiller609 added 2 commits March 8, 2026 22:34

Add per-env domain-gated HTTPS secret substitution

63fa947

Fix review issues in cert cache and prewarm logging

a9231cd

cursor Bot reviewed Mar 9, 2026

View reviewed changes

Comment thread lib/instances/fork.go

Comment thread lib/egressproxy/service.go Outdated

Comment thread lib/egressproxy/enforce_linux.go

Fix follow-up proxy review findings

26c4a29

cursor Bot reviewed Mar 9, 2026

View reviewed changes

Comment thread cmd/api/api/api_test.go

Comment thread cmd/api/api/api_test.go

Comment thread lib/egressproxy/service.go Outdated

Comment thread lib/egressproxy/service.go

Comment thread cmd/api/api/test_prewarm_test.go

Comment thread lib/egressproxy/enforce_linux.go

sjmiller609 added 2 commits March 8, 2026 22:48

Address latest bugbot follow-ups

3266c10

Harden HTTP proxy replacement destination handling

060e1d1

sjmiller609 enabled auto-merge (squash) March 9, 2026 02:58

sjmiller609 changed the title ~~feat: add optional VM egress MITM proxy with mock-secret header rewriting~~ Add optional VM egress MITM proxy with mock-secret header rewriting Mar 9, 2026

sjmiller609 changed the title ~~Add optional VM egress MITM proxy with mock-secret header rewriting~~ Optional VM egress MITM proxy with mock-secret header rewriting Mar 9, 2026

Merge remote-tracking branch 'origin/main' into feature/egress-mitm-p…

e648719

…roxy-secret-rewrite # Conflicts: # lib/instances/create.go # lib/instances/fork.go # lib/instances/types.go # lib/oapi/oapi.go # openapi.yaml

cursor Bot reviewed Mar 10, 2026

View reviewed changes

Comment thread cmd/api/api/images_test.go

Comment thread lib/instances/restore.go

Comment thread lib/egressproxy/service.go

Merge remote-tracking branch 'origin/main' into feature/egress-mitm-p…

2ad0511

…roxy-secret-rewrite # Conflicts: # lib/instances/create.go # lib/instances/manager.go # lib/oapi/oapi.go

cursor Bot reviewed Mar 10, 2026

View reviewed changes

Comment thread lib/egressproxy/cert.go Outdated

sjmiller609 added 3 commits March 10, 2026 17:21

Fix restore proxy config refresh and async image queue coverage

717e16f

Remove unused egress proxy host normalizer

3ef9157

Format instances types after merge

8c3775f

cursor Bot reviewed Mar 10, 2026

View reviewed changes

Comment thread lib/instances/restore.go Outdated

Comment thread lib/instances/restore.go

rgarcia reviewed Mar 11, 2026

View reviewed changes

Comment thread openapi.yaml Outdated

sjmiller609 added 2 commits March 16, 2026 14:54

Merge remote-tracking branch 'origin/main' into feature/egress-mitm-p…

e6d841e

…roxy-secret-rewrite

refactor egress API to policy + credentials and fix restore regressions

a25a59b

cursor Bot reviewed Mar 16, 2026

View reviewed changes

Comment thread lib/instances/create.go

Comment thread lib/egressproxy/service.go

sjmiller609 removed the request for review from hiroTamada March 17, 2026 17:58

sjmiller609 marked this pull request as draft March 17, 2026 17:58

auto-merge was automatically disabled March 17, 2026 17:58
Pull request was converted to draft

sjmiller609 requested review from rgarcia and removed request for rgarcia March 17, 2026 17:58

Fix open review findings for create validation and proxy errors

865b50c

sjmiller609 requested a review from rgarcia March 17, 2026 19:17

sjmiller609 marked this pull request as ready for review March 17, 2026 19:17

rgarcia reviewed Mar 17, 2026

View reviewed changes

Comment thread openapi.yaml

Comment thread openapi.yaml

Comment thread openapi.yaml

Comment thread lib/instances/create.go Outdated

Comment thread lib/instances/egress_proxy.go Outdated

Comment thread lib/system/init/egress_proxy.go Outdated

sjmiller609 marked this pull request as draft March 18, 2026 15:23

sjmiller609 added 3 commits March 18, 2026 11:25

Align egress credential injection with policy model

2dd5cfc

Merge remote-tracking branch 'origin/main' into feature/egress-mitm-p…

2f6d425

…roxy-secret-rewrite

Fix API instance test after merging main

f342efb

rgarcia approved these changes Mar 18, 2026

View reviewed changes

sjmiller609 and others added 2 commits March 19, 2026 11:44

Merge branch 'main' into feature/egress-mitm-proxy-secret-rewrite

3ba6d54

sjmiller609 marked this pull request as ready for review March 19, 2026 17:54

sjmiller609 merged commit f985341 into main Mar 19, 2026
5 checks passed

sjmiller609 deleted the feature/egress-mitm-proxy-secret-rewrite branch March 19, 2026 17:54

Conversation

sjmiller609 commented Mar 8, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

API / Config changes

Implementation details

Tests

Validation run

Notes

Uh oh!

github-actions Bot commented Mar 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✱ Stainless preview builds

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rgarcia left a comment

Choose a reason for hiding this comment

Uh oh!

sjmiller609 commented Mar 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sjmiller609 commented Mar 8, 2026 •

edited by cursor Bot

Loading

github-actions Bot commented Mar 8, 2026 •

edited

Loading