Skip to content

Commit

Permalink
Transit: change abilities encoding
Browse files Browse the repository at this point in the history
  • Loading branch information
@piegamesde
piegamesde committed Jan 13, 2022
1 parent 4acdcd3 commit 1a9bcc8
Show file tree
Hide file tree
Showing 2 changed files with 122 additions and 57 deletions.
46 changes: 7 additions & 39 deletions file-transfer-protocol.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -76,33 +76,17 @@ The `transit` message mentioned above is encoded following this schema:
```json
{
"transit": {
"abilities-v1": [
{
"type": "<string, one of {direct-tcp-v1, relay-v1, tor-tcp-v1}>"
}
],
"hints-v1": [
{
"type": "'direct-tcp-v1' or 'tor-tcp-v1'",
"hostname": "<string>",
"port": "<tcp port>",
"priority": "<number, usually [0..1], optional>"
},
{
"type": "relay-v1",
"hints": [
{
"hostname": "<string>",
"port": "<tcp port>",
"priority": "<number, usually [0..1], optional>"
}
]
}
]
"abilities-v1": [ … ],
"hints-v1": [ … ]
}
}
```

The `abilities-v1` entries follow the canonical encoding version A described in
the transit protocol.
The `hints-v1` entries follow the canonical encoding described in the transit
protocol.

The file-transfer application, when actually sending file/directory data,
may close the Wormhole as soon as it has enough information to begin opening
the Transit connection. The final ack of the received data is sent through
Expand All @@ -111,22 +95,6 @@ and `sha256: HEXHEX` containing the hash of the received data.

## Future Extensions

Transit will be extended to provide other connection techniques:

* WebSocket: usable by web browsers, not too hard to use by normal computers,
requires direct (or relayed) TCP connection
* WebRTC: usable by web browsers, hard-but-technically-possible to use by
normal computers, provides NAT hole-punching for "free"
* (web browsers cannot make direct TCP connections, so interop between
browsers and CLI clients will either require adding WebSocket to CLI, or a
relay that is capable of speaking/bridging both)
* I2P: like Tor, but not capable of proxying to normal TCP hints.
* ICE-mediated STUN/STUNT: NAT hole-punching, assisted somewhat by a server
that can tell you your external IP address and port. Maybe implemented as a
uTP stream (which is UDP based, and thus easier to get through NAT).

The file-transfer protocol will be extended too:

* "command mode": establish the connection, *then* figure out what we want to
use it for, allowing multiple files to be exchanged, in either direction.
This is to support a GUI that lets you open the wormhole, then drop files
Expand Down
133 changes: 115 additions & 18 deletions transit.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,26 +47,23 @@ This may be relaxed in the future, much as Wormhole was.
## Abilities

Each Transit object has a set of "abilities". These are outbound connection
mechanisms that the client is capable of using. The basic CLI tool (running
on a normal computer) has two abilities: `direct-tcp-v1` and `relay-v1`.
mechanisms that the client is capable of using. The following abilities are
specified in this document:

* `direct-tcp-v1` indicates that it can make direct outbound TCP connections to a
requested host and port number.
* `relay-v1` indicates it can connect to the Transit Relay and speak the
matching protocol.
* `tor-tcp-v1`

Future implementations may have additional abilities, such as connecting
directly to Tor onion services, I2P services, WebSockets, WebRTC, or other
connection technologies. Implementations on some platforms (such as web
browsers) may lack `direct-tcp-v1` or `relay-v1`.
* An optional `url_hints` (boolean, default false) attribute may be associated
with the ability. It declares support for an improved hint encoding that allows
using WebSockets (and possibly other protocols) for transport towards the relay
server.
* `tor-tcp-v1` allows both sides finding eath other over Tor

Together with each ability, the Transit object can create a
list of "hints", which tell the respective handshake how to find the other side.
Hints usually contain a `hostname`, a `port` and an optional `priority`.
Each hint will fall under one of the abilities that the peer indicated it
could use. Hints have types like `direct-tcp-v1`, `tor-tcp-v1`, and
`relay-v1`.
Each ability declares its own set of hints; hints have a `type` that is equal
to the name of the ability they hint for.

For example, if our peer can use `direct-tcp-v1`, then our Transit object
will deduce our local IP addresses (unless forbidden, i.e. we're using Tor),
Expand Down Expand Up @@ -135,11 +132,14 @@ your wormhole) can cause their peer to make a TCP connection (and send the
handshake string) to any IP address and port of their choosing. The handshake
protocol is intended to make this no more than a minor nuisance.

## Relay Handshake (`direct-relay-v1`)
## Relay Handshake (`relay-v1`)

The **Transit Relay** is a host which offers TURN-like services for
magic-wormhole instances. It uses a TCP-based protocol with a handshake to
determine which connection wants to be connected to which.
magic-wormhole instances. Clients connect to the relay and do a handshake
to determine which connection wants to be connected to which. The connection
is independent of the transport protocol (currently supported are TCP and
WebSockets), and the relay will also connect two clients using different protocols
together.

When connecting to a relay, the Transit client first writes RELAY-HANDSHAKE
to the socket, which is `please relay $channel for $side\n`, where `$channel`
Expand Down Expand Up @@ -171,13 +171,63 @@ The Transit client can attempt connections to multiple relays, and uses the
first one that passes negotiation. Each side combines a locally-configured
hostname/port (usually "baked in" to the application, and hosted by the
application author) with additional hostname/port pairs that come from the
peer. This way either side can suggest the relays to use. The `wormhole`
application accepts a `--transit-helper tcp:myrelay.example.org:12345`
command-line option to supply an additional relay. The connection hints
peer. This way either side can suggest the relays to use. The connection hints
provided by the Transit instance include the locally-configured relay, along
with the dynamically-determined direct hints. Both should be delivered to the
peer.

## Canonical hint encodings

The transit protocol relies on an existing secured (but possibly low-bandwidth)
communication channel to exchange the abilities and hints. Thus, it has no
influence over how they are encoded. However, we make a suggestion using JSON
messages for other protocols to use.

### `direct-tcp-v1`, `tor-tcp-v1`

```json
{
"type": "direct-tcp-v1" or "tor-tcp-v1",
"hostname": <string>,
"port": <u16>,
"priority": <float; optional>
}
```

`hostname` must be compliant with the `host` part of an URL, i.e. it may be an
IP address or a domain. Furthermore, IPv6 link-local addresses are not supported.

### `relay-v1`

```json
{
"type": "relay-v1",
"hints": [
{
"hostname": "<string>",
"port": "<tcp port>",
"priority": "<number, usually [0..1], optional>"
},
…
],
"urls": [],
}
```

Each entry in `hints` is equivalent to a `direct-tcp-v1` hint. `urls` must be
present if the `url_hints` was advertised. If present, it makes the `hints` field
redundant. If both sides support `url_hints`, `hints` may be omitted. Otherwise,
they must contain compatible fallback values.

`urls` must contain a list of strings which represent valid URLs. Currently known schemes
are `tcp`, `ws` and `wss`; unknown ones must be ignored. The `tcp` scheme is defined
as follows: `tcp://host:port` – the port is *not* optional.

All `urls` in a hint must point to the same relay server, and that server must relay between
any two of its end points, regardless of the used protocol. Clients are free to pick the most
convenient URL to connect to per hint. Use multiple hints for multiple distinct relay servers,
or if the relay server does not support relaying between the different endpoints.

## Encryption

If desired\*, transit provides an encrypted **record-pipe**, which means the two
Expand Down Expand Up @@ -219,6 +269,39 @@ received.
doing their own crypto (e.g. because they tunnel SSH or TLS over transit), then
they can use the "raw" TCP stream built by Wormhole without the encrypted record pipes.

## Canonical abilities encodings

We suggest using one of these two encodings for abilities.

### Version A

Abilities are encoded as a list, each item having a `type` tag. An entry of type
`relay-v1` may additionally contain a `url_hints` boolean attribute. Ability may
appear more than once in the list.

```json
[
{
"type": "<string, one of {direct-tcp-v1, relay-v1, tor-tcp-v1}>"
}
]
```

### Version B

Abilities are encoded as object, with one entry per ability (again an object):

```json
{
"direct-tcp-v1": {},
"relay-v1": { "url_hints": <bool> },
…
}
```

The presence of the key with a non-null indicates support for that ability. The
`relay-v1` key may contain a `url_hints` boolean.

### Nonces

Each message contains a 24 bytes long nonce. The first nonce is all zeroes and is
Expand Down Expand Up @@ -336,3 +419,17 @@ both sides' IP addresses have a longer prefix in common is preferable.
kernel will take care of translating the packets to IPv4 if required. That way,
no special support is required to handle both stacks. Conversely, only binding
to `127.0.0.1` means that the application will only support IPv4.

## Future Extensions

* WebSocket: usable by web browsers, not too hard to use by normal computers,
requires direct (or relayed) TCP connection
* WebRTC: usable by web browsers, hard-but-technically-possible to use by
normal computers, provides NAT hole-punching for "free"
* (web browsers cannot make direct TCP connections, so interop between
browsers and CLI clients will either require adding WebSocket to CLI, or a
relay that is capable of speaking/bridging both)
* I2P: like Tor, but not capable of proxying to normal TCP hints.
* ICE-mediated STUN/STUNT: NAT hole-punching, assisted somewhat by a server
that can tell you your external IP address and port. Maybe implemented as a
uTP stream (which is UDP based, and thus easier to get through NAT).

0 comments on commit 1a9bcc8

Please sign in to comment.