Repository navigation
SOCKS proxy implementation
Upcoming. This page describes the
socks-proxybranch. It is not part of a release, and details may change.
Part of SOCKS proxy. The protocol of the proxy is a Maven module of its own, proxy, which knows
nothing of QUIC, CDI or sporks; the application module adds the daemon glue, the chain packets and the services
in service/chain. The chains themselves, their entries and the traffic between nodes are on
SOCKS proxy chains.
The module is the Maven module hedgehog-proxy, in the directory proxy/, and the JPMS module
org.unigrid.hedgehog.proxy, version 0.0.8-SNAPSHOT. It depends on no other Hedgehog module, only on Netty
4.1.90.Final (with its SOCKS codec), BouncyCastle 1.70, SLF4J and Lombok. The application module depends on it
(requires org.unigrid.hedgehog.proxy).
| Package | Main classes |
|---|---|
proxy/socks |
SocksInitializer, SocksGreetingHandler, SocksCommandHandler, HandshakeTimeout, QuietClose, Tunnel, TunnelStream, UdpAssociation, UdpBinder, UdpHeaderCodec
|
proxy/group |
ProxySettings, Epoch, ShardGroups, Rendezvous, ChainIds, GroupView, ExitChoice
|
proxy/chain |
Entry, EntryCodec, FnvJ64, WriterKey, WriterId, Feed, FeedWriter, Chain, ChainLimits, Link, Admission, HeadsDiff, Gap
|
proxy/session |
ClientSession, Session, Handshake, EphemeralKey, Transcript, SessionKeys, SessionCipher, SealedCodec, SocksAddress, DestinationPolicy
|
proxy/exit |
ExitRelay, NettyExitConnector, NettyUdpConnector, DatagramRelay, DatagramDestinations, Target
|
application, server/socks
|
SocksServer, ConnectionLimit
|
application, service/chain
|
ChainHost, HostedChain, ChainClient, ClientSessions, ChainTunnel, ChainExit, ChainOutbox, WriterPauses, StreamBudgets, ExitContexts
|
The rest of the glue in application: SocksOptions holds the two options, CommunicationSpork and
CommunicationFlag the spork, ChainProducer turns a stored spork into the ProxySettings the services read,
and TunnelProducer builds the ChainTunnel that SocksServer hands its connections to. PeerProtocol,
StreamRouter, ChainStreamInitializer, ChainPipeline and ChainCodecs carry the chain traffic between nodes.
The fields follow the spork header, big-endian, 32 bytes in all.
| Offset | Size | Field |
|---|---|---|
| 0 | 2 | groupSize |
| 2 | 4 | reorganizePeriodSeconds |
| 6 | 2 | chainsPerGroup |
| 8 | 2 | clientsPerChain |
| 10 | 4 | chainCapacityBytes |
| 14 | 2 | retentionSeconds |
| 16 | 4 | maxEntryBytes |
| 20 | 2 | sessionIdleSeconds |
| 22 | 2 | communicationFlags |
| 24 | 8 | Reserved, zero |
SporkData.validate enforces the bounds, which are those of the wire widths. The REST route
PUT /gridspork/communication answers 400 with its message, PublishSporkChannelHandler drops a spork that fails it
after the signature check and logs a warning, and ChainProducer treats a stored one that fails it as no spork, so
the proxy stays off. The REST route, the relay of a received spork and the periodic publish send it only to peers
for which PeerProtocol.speaksProxy holds.
SocksServer is an @Eager @ApplicationScoped bean that does nothing while --socks-port is 0. Otherwise it binds a
Netty server on two threads, and ConnectionLimit closes every connection beyond 512 at once. SocksInitializer
builds the pipeline of each connection: Socks5ServerEncoder, Socks5InitialRequestDecoder,
SocksGreetingHandler, Socks5CommandRequestDecoder, HandshakeTimeout, SocksCommandHandler and QuietClose.
Each stage removes itself or its decoder once its part is done.
-
Greeting. One that offers
NO AUTHgets0x00; any other gets0xFFand a close. -
Deadline.
HandshakeTimeoutcloses a connection that has sent no command 10 seconds after it opened. -
Read failures.
QuietClosecloses a connection whose read failed and logs only the class of the failure. -
Command.
SocksCommandHandlerstops reads at the command. ForCONNECTit holds bytes the client sends ahead of the reply, up to 128 KiB, and closes the connection beyond that.BINDgets0x07. -
Addresses.
SocksAddressaccepts a domain of 1 to 255 printable ASCII characters without a colon, bracket or percent sign, whose last label is not a number in decimal or hex (127.1,2130706433,0x7f.1), since resolvers read such names as addresses. Trailing dots do not hide the number. Anything else gets0x08.
The client gets one reply. An exit's bound address that the reply cannot carry is replaced by
0.0.0.0:0, and the stream opens all the same.
The front end talks to the network through Tunnel, which ChainTunnel implements over client sessions. It reads
the spork for every stream and association, refuses with 0x02 while the spork is absent or lacks the flag, and
gives each 120 seconds to be answered before it fails with 0x06. The two minutes cover 10 seconds per member asked
to join, 5 seconds for each of three exits and 8 addresses of 10 seconds at the exit.
Writes are cut into Data messages of at most maxEntryBytes less 181 bytes of entry overhead, 24 of seal and 1 of
type: 16178 bytes with the default spork. The front end reads its client only while the session's writer has
nothing waiting in its ChainOutbox and no member has paused it; ClientSessions hands each outbox's state to its
streams once a second. A client that reads slowly makes the stream hold what arrives, up to 1 MiB, and then drops
the stream, as the chain has no flow control that could slow the exit down. There is no half-close: Netty closes a
connection whose input shuts down, and with it the session.
UDP ASSOCIATE starts a session that carries Connect(UDP_ASSOCIATE, 0.0.0.0:0). When the exit answers,
UdpAssociation binds a datagram socket at the local address the client reached, replies with that address and
reads the control connection again, whose end ends the association. The socket serves only the control
connection's IP address, with the port the request named or, for port 0, the port of the first valid datagram.
UdpHeaderCodec requires RSV and FRAG to be zero, since fragments are not reassembled.
A datagram is dropped, never queued, when it does not fit one entry, while the chain has no room, when it comes from another source, when 64 datagrams from the exit already wait for the event loop, or when it would exceed 65507 bytes with its header on the way to the client. Drops are counted, and the association logs one warning once it has dropped more than 1000.
Every node derives the groups, chains and exits from the spork, the active gridnodes and the clock, without coordinating with any other.
-
Epoch. The epoch is ⌊t / 1000P⌋ for t, the time in milliseconds since 1970, and P,
reorganizePeriodSeconds; it is encoded as u64. -
Groups.
ShardGroups.ofranks the gridnode ids by rendezvous hashing, as in the placement of Network storage: highest SHA-256(s followed by the UTF-8 id) first, equal scores by id, with s = SHA-256(shard-groupfollowed by the epoch). It deals them round robin into max(1, ⌊n /groupSize⌋) groups, so sizes differ by at most one; with no gridnodes there is no group. -
Chains. Chain i of group g is SHA-256(
chainfollowed by the epoch as u64, g as u32 and i as u32). The members of a group host its chains.
Hosting. ChainHost refreshes once a second and rebuilds its hosting only when the settings, the epoch, the
active gridnodes or its own id change. A rebuilt hosting keeps a chain whose id and limits stay the same.
Joining. ChainClient picks a group at random among those that have not refused it, never the node's own.
With a fresh WriterKey it sends JOIN_CHAIN without a chain id to a random member it has a chain stream to; a
group with none is passed over at once. A JOINED reply must name a chain of the group, and the node follows that
chain and asks a second member for it as a spare path. A refusal, or no reply within 10 seconds
(PendingRequests.TIMEOUT), passes the group over. How a member places the writer is on
SOCKS proxy chains. All sessions of the node on one attachment share its
writer and its outbox.
The exit. ExitChoice ranks all members of the group, those the node has no stream to included, by
SHA-256(f followed by the id) for the session fingerprint f, 16 random bytes per attempt, and takes the highest not
yet tried. A member whose id does not fit an OPEN entry is passed over.
Retiring and draining. Chains a rebuilt hosting no longer wants retire. They take no new clients but still take
entries, and they are kept while their epoch is at most 4 behind the current one, 1024 at most, the oldest epoch
going first. One that no client and no session of the node uses goes once it has been retired for
retentionSeconds. Any goes when its grace, min(10 minutes, P), is over, after every ChainDrain was told, so that
ChainExit closes its relays DONE while the chain still carries the Close; a chain whose drains failed goes
after two graces. A client closes the sessions on an attachment that is no longer current DONE after the same
grace.
A session lives on one chain under one fingerprint. Both ends write to their own feed: the client to the writer of its attachment, the exit to its own writer on that chain.
ClientSession posts an OPEN. ChainExit answers an OPEN that names its own id, on a current or retired chain
of its group, while it holds fewer than 256 relays, fewer than 128 on that chain and fewer than 64 for that writer,
and none for that fingerprint on that chain.
| Entry | Payload |
|---|---|
OPEN |
The client's X25519 key (32 bytes), then the exit's gridnode id in UTF-8 |
ACCEPT |
The exit's X25519 key (32 bytes), then the confirmation (32 bytes) |
Both ends run HKDF-SHA256 over the X25519 secret, with the fingerprint as
salt and hedgehog-proxy/session followed by the 176-byte transcript as info. The 96 bytes it yields are the
client-to-exit key, the exit-to-client key and the confirmation the ACCEPT carries.
| Transcript bytes | Field |
|---|---|
| 0 to 31 | Chain id |
| 32 to 47 | Fingerprint |
| 48 to 79 | Client key |
| 80 to 111 | Exit key |
| 112 to 143 | Client writer |
| 144 to 175 | Exit writer |
The agreement refuses a key of the wrong size and one of low order, whose secret is all zeros. The client takes
the writer of the ACCEPT as the exit writer and passes over an ACCEPT whose confirmation does not match,
compared in constant time, so the chosen exit's own ACCEPT can still complete the handshake. It forgets its
private key once the handshake is over or given up, and sends its Connect at once. An attempt left unanswered for
5 seconds is posted again with a new fingerprint towards the next exit, up to 3 attempts; then the session closes
UNREACHABLE.
SessionCipher is AES-256-GCM. A sealed payload is the counter (u64), the ciphertext and the 16-byte tag. A feed
delivers a session's entries in order and without loss, so the receiver takes exactly the next counter: a lower one
(REPLAYED), a higher one (SKIPPED) or a tag that fails (TAMPERED) closes the session BROKEN and tells the
peer. d is the direction of the message: 1 from client to exit, 2 from exit to client.
| Nonce bytes | Content |
|---|---|
| 0 | d |
| 1 to 3 | Zero |
| 4 to 11 | Counter, big-endian |
| Associated data bytes | Content |
|---|---|
| 0 to 31 | Chain id |
| 32 to 47 | Fingerprint |
| 48 | d |
SealedCodec puts one message in each SEALED entry, starting with its type (u8). An address is a SOCKS5 type
byte (1 for IPv4 and 4 bytes, 3 for a domain with a u8 length and ASCII, 4 for IPv6 and 16 bytes) followed by a
u16 port.
| Type | After the type | Meaning |
|---|---|---|
1 Connect
|
Command (u8), address |
CONNECT (1) or UDP_ASSOCIATE (3) |
2 Connected
|
Status (u8, 0 to 8), address | The exit's SOCKS5 reply, with its bound address or 0.0.0.0:0
|
3 Data
|
Bytes | TCP bytes, in order |
4 Datagram
|
Address, bytes | The destination from the client, the source towards it |
5 Close
|
Reason (u8) |
DONE 0, IDLE 1, REFUSED 2, UNREACHABLE 3, BROKEN 4 |
A session carries one Connect, one Connected, Data or Datagram messages and a Close; anything else closes
it BROKEN at the exit. A Close ends a session even when it cannot be posted. A Close that arrives before the
reply fails the stream: REFUSED as 0x02, UNREACHABLE as 0x03, IDLE as 0x06, the others as 0x01.
Each end closes after sessionIdleSeconds without traffic either way, and on a Close, on a gap in the peer's
feed that names the fingerprint and on the drain of its chain.
ClientSession |
Left when |
|---|---|
NEW |
The OPEN is posted; or no exit fits, and it closes |
OPENING |
An ACCEPT confirms the handshake; after 5 seconds a new attempt stays here, and after 3 attempts or with no exit left it closes |
OPEN |
A Close or the idle limit |
CLOSED |
Never |
ExitRelay |
Left when |
|---|---|
WAITING |
Connect(CONNECT) leads to CONNECTING, Connect(UDP_ASSOCIATE) to OPENING
|
CONNECTING |
The destination is connected (RELAYING), or it was refused or failed (CLOSED) |
OPENING |
The datagram socket is bound (ASSOCIATED), or it failed (CLOSED) |
RELAYING, ASSOCIATED
|
Either side closes (CLOSED) |
CLOSED |
Never |
ChainExit also closes IDLE a relay whose client has sent nothing within 30 seconds of the ACCEPT.
ChainExit hands every entry of a fingerprint it relays to that session and its ExitRelay. Its sessions on one
chain share one writer, made fresh for that chain, and one ChainOutbox. A chain the host rebuilt or dropped, or a
feed whose outbox ended, closes its relays BROKEN, and the next OPEN starts a new feed.
Streams. On Connect(CONNECT) the relay checks the CONNECT flag of the spork it accepted the session under
and admits the destination. A name is looked up on a pool of 32 threads with a queue of 256, and a full pool
answers 0x01; an IP literal skips the pool. NettyExitConnector tries up to 8 admitted addresses, one after the
other, for 10 seconds each. A failure maps to 0x06 for a timeout, 0x04 for no route, 0x05 for a refusal,
0x03 for any other socket error and 0x01 otherwise. Success answers Connected(SUCCESS) with the exit's own end
as the bound address.
The relay reads the destination only while its outbox can write, which it learns once a second. It closes BROKEN
when more than 64 full Data messages wait for a slow destination, and answers the destination's close with
Close(DONE). When the relay ends, the connection to the destination gets 5 seconds to take what is queued before
it is closed.
Associations. On Connect(UDP_ASSOCIATE) the relay checks the UDP flag, opens one datagram socket on the
wildcard address and answers Connected(SUCCESS, 0.0.0.0:0). DatagramDestinations admits every destination before
it sends to it, and sends to the first checked address, never to the name. An association keeps 64 destinations, 4
lookups at once and 8 datagrams waiting for each lookup, and forgets a destination nothing was sent to for 60
seconds. A datagram to port 0 is dropped. A reply is taken only from an address and port the association sent to,
decided before its content is copied, and only when it fits one entry.
Refusals. A refused or failed Connect is answered Connected(status) and closed REFUSED for 0x02, 0x07
and 0x08, else UNREACHABLE; the client's tunnel fails the stream with that status and closes the session.
DestinationPolicy.publicOnly() is the only gate between a stranger's request and a connection. Production gets
it from ExitContexts.production(), and LoopbackPolicyIsTestOnlyTest asserts that only tests reach
allowLoopback(). For TCP and UDP alike it refuses these blocks, RFC 1918
among them:
| Family | Refused |
|---|---|
| IPv4 |
0.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 127.0.0.0/8, 169.254.0.0/16, 172.16.0.0/12, 192.0.0.0/24, 192.0.2.0/24, 192.168.0.0/16, 198.18.0.0/15, 198.51.100.0/24, 203.0.113.0/24, 224.0.0.0/4, 240.0.0.0/4
|
| IPv6 |
::/128, ::1/128, fc00::/7, fe80::/10, fec0::/10, ff00::/8, 2001:db8::/32, 64:ff9b:1::/48
|
An IPv6 address that carries an IPv4 address is also held to the IPv4 rules: IPv4-mapped (::ffff:0:0/96),
IPv4-translated (::ffff:0:0:0/96), IPv4-compatible (::/96), NAT64 (64:ff9b::/96), 6to4 (2002::/16), Teredo
(2001::/32, both the server and the flipped client address) and ISATAP (interface identifier 0000:5efe or
0200:5efe under any prefix).
localhost and every name below it, in any case and with trailing dots, are refused without a lookup. A name is
looked up once and refused when any of its addresses is refused, so it cannot pair a public answer with a local one.
The exit connects only to the addresses it checked and never looks the name up again. A refusal answers 0x02, and
a name without addresses 0x04.
The module proxy has a unit test class per mechanism, in the same packages as the code, among them the SOCKS
pipeline on embedded channels, where HeldWrites holds writes back as a slow socket would, and the exit over
loopback (ExitRelayLoopbackTest, ExitRelayUdpLoopbackTest). In application, ChainFleet is an in-process fleet
of four gridnodes in one group that share one clock, topology and spork and exchange packets over embedded
channels a round at a time. ChainProxyEndToEndTest and ChainProxyUdpEndToEndTest run a SOCKS5 client through the
front end, a client node, the fleet and an exit to a TCP or UDP echo on loopback, covering refusals, retries, idle
closes, fragments and strangers.
-
proxy/src/main/java/org/unigrid/hedgehog/proxy/socks/: the SOCKS5 front end,Tunneland the UDP association -
proxy/src/main/java/org/unigrid/hedgehog/proxy/group/: settings, epochs, shard groups and the exit choice -
proxy/src/main/java/org/unigrid/hedgehog/proxy/session/: the handshake, keys, sealing, messages andDestinationPolicy -
proxy/src/main/java/org/unigrid/hedgehog/proxy/exit/:ExitRelay, the connectors and the datagram destinations -
application/src/main/java/org/unigrid/hedgehog/server/socks/SocksServer.java,ConnectionLimit.java: the listener -
application/src/main/java/org/unigrid/hedgehog/command/option/SocksOptions.java: the options -
application/src/main/java/org/unigrid/hedgehog/model/spork/CommunicationSpork.java: the spork and its bounds -
application/src/main/java/org/unigrid/hedgehog/model/network/codec/chunk/CommunicationSporkEncoder.java: the spork's bytes -
application/src/main/java/org/unigrid/hedgehog/model/producer/ChainProducer.java,TunnelProducer.java: the settings and the tunnel -
application/src/main/java/org/unigrid/hedgehog/service/chain/ChainTunnel.java,ClientSessions.java,ChainClient.java: the client side -
application/src/main/java/org/unigrid/hedgehog/service/chain/ChainExit.java,ExitContexts.java: the exit side -
application/src/test/java/org/unigrid/hedgehog/service/chain/ChainFleet.java,ChainProxyEndToEndTest.java,ChainProxyUdpEndToEndTest.java: the fleet and the end-to-end tests