Proposal: live EC2 port publishing with security-group enforcement

[CoreOxide]

This proposes a separate design discussion for host port publishing while EC2 security-group enforcement is enabled.

Related work:

• Original request: [FEAT] EC2/ECS: enforce security group rules on container traffic #2564
• Implementation and review context: feat(ec2)!: complete security group enforcement #3771
• Hector's request to separate port publishing from the ENI and IPv6 work: feat(ec2)!: complete security group enforcement #3771 (comment)

Problem

The current socat publisher supports opening application ports on an already-running EC2 instance. That behavior matters: a security-group rule can authorize a new port after launch.

The launch-time Docker binding approach explored in #3771 does not preserve that behavior. Docker port bindings are fixed when a container is created, so putting those bindings on the protected namespace at startup leaves later-authorized ports unpublished. Recreating the workload to add a port would be a regression.

A plain TCP relay also creates a new connection to the application. Filtering that connection by the relay's address cannot establish whether the original peer was permitted.

Proposed direction

Keep the existing publisher when enforcement is disabled. Enforcement remains opt-in through FLOCI_NETWORK_SECURITY_GROUP_ENFORCEMENT_ENABLED=false by default.

For enforcement mode, investigate a separately managed, replaceable publishing helper for each published port, reusing existing port allocation and lifecycle patterns. Adding or removing a publication should affect that helper, not restart the EC2 workload or its protected network namespace.

The helper would use packet forwarding rather than terminate and recreate application connections. Every forwarded connection must cross the protected endpoint's security-group policy before reaching the application. Docker would still own host port bindings; Floci would manage only its own helper namespaces, rules, containers, and labels.

This is a direction to prototype, not a claim that a simple DNAT rule is sufficient. The prototype must establish the return path, source-address visibility, and required namespace connections on both rootful Linux Docker and Linux containers on Docker Desktop. If preserving the necessary identity requires a different topology, that result should come back to this discussion before implementation expands.

Required behavior

• Authorizing a new application port after launch can publish it without restarting the instance. A published listener is only transport availability; ingress rules must still decide which sources can connect.
• Managed peers are evaluated using logical ENI addresses and security-group membership, then translated to transport addresses. A publishing helper must not collapse distinct managed peers into one trusted identity or provide a bypass.
• New connections require the applicable ingress and egress permissions. Policy updates must not accidentally discard permitted tracked return traffic.
• Revocation immediately prevents newly forbidden connections. Listener removal and helper cleanup must account explicitly for established flows, rather than silently breaking connection tracking.
• A failed helper setup or policy update must not expose an unrestricted path. Install policy before making a publication reachable, and reconcile or quarantine failures.
• Stop/start, reboot, termination, restart recovery, and port-allocation cleanup must preserve the intended mapping without leaking helpers or reserving ports indefinitely.
• Workload containers receive no networking privileges. No changes to Docker-owned firewall rules.

Boundaries and alternatives

The initial scope is EC2 application host publishing. ECS bridge/host networking and new ECS publishing behavior are outside this proposal. No VPC routing, NACLs, NAT gateways, or peering are introduced.

Host-published ingress can only filter the source visible inside Docker. Original external-client IP fidelity on Docker Desktop is excluded and must be documented. Managed-peer identity preservation is a separate requirement and must be demonstrated.

Launch-only bindings do not meet the live-addition requirement. Recreating the workload to change bindings also does not meet it. Keeping an ordinary socat relay in the enforced path would require a separate, demonstrated way to enforce original-peer policy before identity is lost.

Until a design is agreed and validated, document that application host publishing, including ports authorized after launch, is unavailable with enforcement enabled. Preserve existing publishing behavior with enforcement disabled.

Validation before landing

Use SDK-driven management operations and real traffic on Linux and Docker Desktop to prove live additions, wrong-CIDR denial followed by correct-CIDR success, independent ingress/egress decisions, SG references, and IPv4/IPv6 TCP and UDP behavior. Exercise tracked connections during revocation, restart recovery, cleanup, and injected helper/policy failures. Verify that both direct and host-published access cannot bypass filtering.

The main design question is whether replaceable packet-forwarding helpers can meet these identity and lifecycle requirements with acceptable complexity. Feedback on that topology, return-path handling, and established-flow behavior would help determine the smallest viable implementation.

[CoreOxide]

@hectorvent is there a "waiting for community review" tag or something? 😆