Architecture

Architecture

This page describes how Rosemary works internally, from the transport layer to kernel-level traffic interception.

---

Overview

Rosemary has two components:

• Server runs on the operator machine with root/admin privileges. It manages agent connections, installs interception rules, and exposes the dashboard and REST API.
• Agent runs on the remote host with no special privileges. It connects back to the server, registers its subnets, and forwards traffic on demand.

Once an agent registers, the server installs kernel rules so that any packet your machine sends to the agent's subnets is silently intercepted and tunneled through the agent. No application changes, no proxy settings, no TUN device.

---

Transport Stack

Application traffic
       |
       v
  iptables / WinDivert / pf   (kernel intercept)
       |
       v
  Transparent proxy listener   (server, userspace)
       |
       v
  WebSocket connection         (gorilla/websocket)
       |
       v
  yamux multiplexer            (hashicorp/yamux)
       |
       v
  AES-256-GCM messages         (encrypted JSON + binary)
       |
       v
  Agent                        (dials target, bridges data)

WebSocket

All agent-to-server communication runs over a single persistent WebSocket connection. The wsNetConn adapter converts the WebSocket into a standard net.Conn so that yamux can layer on top of it.

yamux

yamux multiplexes many independent streams over the single WebSocket connection. Each TCP connection being proxied gets its own yamux stream. This avoids head-of-line blocking and allows concurrent flows without opening multiple WebSocket connections.

Encryption

Every message is encrypted with AES-256-GCM before sending:

• Key: 32-byte random value shared between server and agent, base64url-encoded
• Nonce: random per message
• Framing: 4-byte big-endian length prefix followed by the encrypted payload

Compression

When a data payload exceeds 256 bytes and DEFLATE compression reduces its size, the payload is compressed before encryption. The receiver detects and decompresses automatically.

---

Authentication

When an agent connects over WebSocket, a challenge-response handshake runs before any data flows:

1. Server generates a random nonce and encrypts it with the shared key
2. Server sends the encrypted nonce to the agent
3. Agent decrypts the nonce and encrypts it back
4. Server verifies the response matches the original nonce

If the response is wrong, the connection is closed immediately. This ensures only agents that know the shared key can register.

---

Agent Registration

After authentication the agent:

1. Enumerates local network interfaces
2. Runs ip route show (Linux) or route print -4 (Windows) to discover subnets
3. Probes internet access by dialing an external address
4. Reads hostname and current username
5. Sends a register message with all of this metadata

The server assigns the agent an ID and stores its subnet list.

---

Kernel-Level Interception

Linux

The server uses iptables to intercept traffic destined for the agent subnets:

Protocol	Mechanism
TCP	OUTPUT REDIRECT to the TCP proxy port (1080)
UDP	MARK + TPROXY to the UDP proxy port (1081)
ICMP	MARK on echo-request packets, caught by raw socket
DNS	OUTPUT REDIRECT port 53 to the DNS proxy port (5300)

The TCP proxy reads the original destination using SO_ORIGINAL_DST. The UDP proxy uses IP_RECVORIGDSTADDR to recover the original destination from TPROXY-intercepted packets.

When a default egress agent is set, a catch-all rule routes all non-subnet traffic through that agent as well.

Windows

The server embeds WinDivert.dll and WinDivert64.sys and extracts them at startup. WinDivert intercepts packets at the network layer before they leave the machine, allowing the server to redirect them to the appropriate agent.

BSD / macOS

The server configures pf rules and loopback routing to redirect traffic for agent subnets.

---

Proxy Listeners

The server runs four listeners that handle intercepted traffic:

TCP Transparent Proxy (default :1080)

• Accepts connections redirected by iptables
• Reads original destination via SO_ORIGINAL_DST
• Looks up which agent owns that destination subnet
• Opens a yamux stream to the agent and sends a connect message
• Bridges data bidirectionally

UDP TPROXY (default :1081)

• Binds with IP_TRANSPARENT and IP_RECVORIGDSTADDR
• Reads original destination from ancillary socket data
• Forwards UDP datagrams to the owning agent
• Spoofs UDP responses back using a raw socket bound to the original destination

DNS Proxy (default :5300)

• Intercepts all DNS queries redirected from port 53
• Fans the query out to all connected agents in parallel
• Returns the first successful answer
• Falls back to saved system DNS servers and public resolvers (8.8.8.8, 1.1.1.1)

ICMP Interceptor

• Opens a raw ip4:icmp socket
• Intercepts echo-request packets marked by iptables
• Forwards them to the owning agent via icmp_proxy messages
• Returns echo replies to the original sender

---

Multi-Agent Routing

The server maintains a routing table mapping each CIDR subnet to the agent that owns it. When a connection arrives at a proxy listener, the server does a longest-prefix match against this table to find the correct agent.

Messages include original_agent_id and target_agent_id fields, enabling the server to relay messages between agents for multi-hop pivoting scenarios.

---

Agent Message Loop

After registration the agent enters a loop processing server commands:

Message	Action
connect	Dial the target host and bridge data back
data	Write payload to an open connection
icmp_proxy	Send an ICMP echo request and return the result
dns_request	Resolve a hostname and return the answer
ping-sweep-request	Probe a subnet for live hosts via TCP
port-scan-request	Scan TCP/UDP ports on a target
start-agent-listener	Open a local port forward
stop-agent-listener	Close a port forward
agent_fwd_open	Initiate a reverse forward
reconnect	Close and reconnect to the server
disconnect	Exit

The agent also sends a heartbeat message every 10 seconds. If the connection drops, the agent reconnects with exponential backoff.

---

Bind Mode

In standard mode the agent connects outbound to the server. In bind mode the roles are reversed:

1. Agent listens on a local TCP port
2. Operator runs connect <ip:port> from the server REPL
3. Server dials the agent and runs the same authentication and registration flow over the raw TCP connection with yamux on top

This is useful when the agent host cannot reach the server directly but the server can reach the agent.