Skip to content

Architecture

Jump to bottom
katana edited this page May 15, 2026 · 2 revisions

Architecture

This page describes how Rosemary works internally, from the transport layer to kernel-level traffic interception.

Overview

Rosemary has two components:

  • Server runs on the operator machine with root/admin privileges. It manages agent connections, installs interception rules, and exposes the dashboard and REST API.
  • Agent runs on the remote host with no special privileges. It connects back to the server, registers its subnets, and forwards traffic on demand.

Once an agent registers, the server installs kernel rules so that any packet your machine sends to the agent's subnets is silently intercepted and tunneled through the agent. No application changes, no proxy settings, no TUN device.

Transport Stack

Application traffic
       |
       v
  iptables / WinDivert / pf   (kernel intercept)
       |
       v
  Transparent proxy listener   (server, userspace)
       |
       v
  QUIC/UDP agent transport     (default :2048/udp)
       |
       v
  RTN framed messages          (control + data envelopes)
       |
       v
  AES-256-GCM payloads         (encrypted JSON + data)
       |
       v
  Agent                        (dials target, bridges data)

QUIC Transport

In standard mode the agent connects outbound to the server over QUIC/UDP. The default agent transport port is 2048, configured with -agent-port on the server and -s <server>:2048 on the agent.

QUIC provides stream multiplexing and transport keepalive. Rosemary carries control messages and proxied data over transport streams.

Bind Transport

In bind mode the agent listens on a TCP address, default 0.0.0.0:9001, and the server connects with connect <ip:port>. Bind mode uses length-prefixed AES-GCM frames over TCP and is intended for networks where the server can reach the agent but the agent cannot connect outbound to the server.

Encryption

Rosemary message payloads use AES-256-GCM:

  • Key: 32-byte random value shared between server and agent, base64url-encoded
  • Nonce: random per encrypted message
  • Bind framing: 4-byte big-endian length prefix followed by encrypted payload
  • QUIC authentication: key-derived certificate material and HMAC token

Compression

When a data payload exceeds 256 bytes and DEFLATE compression reduces its size, the payload is compressed before encryption. The receiver detects and decompresses automatically.

Agent Registration

After authentication the agent:

  1. Enumerates local network interfaces
  2. Runs ip route show (Linux) or route print -4 (Windows) to discover subnets
  3. Probes internet access by dialing an external address
  4. Reads hostname and current username
  5. Sends a register message with all of this metadata

The server assigns the agent an ID and stores its subnet list.

Kernel-Level Interception

Linux

The server uses iptables to intercept traffic destined for the agent subnets:

Protocol Mechanism
TCP OUTPUT REDIRECT to the TCP proxy port (1080)
UDP MARK + TPROXY to the UDP proxy port (1081)
ICMP MARK on echo-request packets, caught by raw socket
DNS OUTPUT REDIRECT port 53 to the DNS proxy port (5300)

The TCP proxy reads the original destination using SO_ORIGINAL_DST. The UDP proxy uses IP_RECVORIGDSTADDR to recover the original destination from TPROXY-intercepted packets.

When a default egress agent is set, a catch-all rule routes all non-subnet traffic through that agent as well.

Windows

The server embeds WinDivert.dll and WinDivert64.sys and extracts them at startup. WinDivert intercepts packets at the network layer before they leave the machine, allowing the server to redirect them to the appropriate agent.

BSD / macOS

The server configures pf rules and loopback routing to redirect traffic for agent subnets.

Proxy Listeners

The server runs four listeners that handle intercepted traffic:

TCP Transparent Proxy (default :1080)

  • Accepts connections redirected by iptables
  • Reads original destination via SO_ORIGINAL_DST
  • Looks up which agent owns that destination subnet
  • Sends a connect message to the agent
  • Bridges data bidirectionally

UDP TPROXY (default :1081)

  • Binds with IP_TRANSPARENT and IP_RECVORIGDSTADDR
  • Reads original destination from ancillary socket data
  • Forwards UDP datagrams to the owning agent
  • Spoofs UDP responses back using a raw socket bound to the original destination

DNS Proxy (default :5300)

  • Intercepts all DNS queries redirected from port 53
  • Fans the query out to all connected agents in parallel
  • Returns the first successful answer
  • Falls back to saved system DNS servers and public resolvers (8.8.8.8, 1.1.1.1)

ICMP Interceptor

  • Opens a raw ip4:icmp socket
  • Intercepts echo-request packets marked by iptables
  • Forwards them to the owning agent via icmp_proxy messages
  • Returns echo replies to the original sender

Multi-Agent Routing

The server maintains a routing table mapping each CIDR subnet to the agent that owns it. When a connection arrives at a proxy listener, the server does a longest-prefix match against this table to find the correct agent.

Messages include original_agent_id and target_agent_id fields, enabling the server to relay messages between agents for multi-hop pivoting scenarios.

Agent Message Loop

After registration the agent enters a loop processing server commands:

Message Action
connect Dial the target host and bridge data back
data Write payload to an open connection
icmp_proxy Send an ICMP echo request and return the result
dns_request Resolve a hostname and return the answer
ping-sweep-request Probe a subnet for live hosts via TCP
port-scan-request Scan TCP/UDP ports on a target
start-agent-listener Open a local port forward
stop-agent-listener Close a port forward
agent_fwd_open Initiate a reverse forward
reconnect Close and reconnect to the server
disconnect Exit

The agent also sends a heartbeat message every 10 seconds. If the transport drops, the agent reconnects with exponential backoff.

Bind Mode

In standard mode the agent connects outbound to the server. In bind mode the roles are reversed:

  1. Agent listens on a local TCP address
  2. Operator runs connect <ip:port> from the server REPL
  3. Server dials the agent and exchanges encrypted frames over the TCP connection
  4. Agent registers normally after the server receives a valid decrypted register message

This is useful when the agent host cannot reach the server directly but the server can reach the agent.

Use only on systems you own or have explicit written permission to test. Unauthorized use is prohibited.

Clone this wiki locally