Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(iroh-net): simplify relay handshake #2164

Merged
merged 14 commits into from
Apr 15, 2024
Merged

Conversation

dignifiedquire
Copy link
Contributor

@dignifiedquire dignifiedquire commented Apr 9, 2024

The relay handshake is quite expensive currently

  • TLS 1.3 + HTTP 1 UPGRADE
  • Server sends FrameType::ServerKey
  • Client sends FrameType::ClientInfo
  • Server sends FrameType::ServerInfo

This simplifies the protocol to

  • TLS 1.3 + HTTP 1 UPGRADE
  • Client sends FrameType::ClientInfo

information changes

  • using a signature, instead of a shared secret to ensure the client identity.
  • server key is not sent (use certificate pinning if fixed relays are important)
  • remove unused configuration in the info frames
  • change magic from derp to relay
  • increase protocol version to 3
  • drop serverinfo in favor of fixed config on the client
  • enable tcp nodelay
  • switch to AES for encryption in favor over chacha
  • the server simply aborts when the client version doesn't match (matching what we do in other protocols)

Benchmarks:
https://gist.github.com/dignifiedquire/30131dbe8b87fb799a971068899656ef

BREAKING:
This breaks the relay handshake, and so requires updated relay nodes to work with.

@dignifiedquire dignifiedquire changed the title [Experiment] perf: simplify relay handshake perf: simplify relay handshake Apr 9, 2024
@dignifiedquire dignifiedquire marked this pull request as ready for review April 9, 2024 11:49
Copy link
Contributor

@flub flub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC the idea is to deploy this under a new URL?

I think this looks fine, there's still a TODO though

iroh-net/src/relay/codec.rs Outdated Show resolved Hide resolved
@ramfox
Copy link
Contributor

ramfox commented Apr 9, 2024

Haven't done a read-through yet, but just looking at the protocol changes: we haven't been using rate limiters for the client yet, but we should talk about what we might do to replace it in the future if we no longer have a ServerInfo. Or is this something we can get away with never doing?

@dignifiedquire
Copy link
Contributor Author

we haven't been using rate limiters for the client yet, but we should talk about what we might do to replace it in the future if we no longer have a ServerInfo. Or is this something we can get away with never doing?

I think we should communicate those expectations out of band. Eg have a default limiter in the client that we expect clients to use. If more details are needed we can add config options to specificy this per der relay on the client side.

/// 8 bytes: 0x44 45 52 50 f0 9f 94 91
const MAGIC: &str = "DERP🔑";
/// The Relay magic number, sent in the FrameType::ClientInfo frame upon initial connection.
const MAGIC: &str = "RELAY🔑";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oooo spicy

@dignifiedquire dignifiedquire added this to the v0.14.0 milestone Apr 10, 2024
Copy link
Contributor

@ramfox ramfox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! I just have some doc change suggestions.

very exciting

iroh-net/src/relay/codec.rs Show resolved Hide resolved
@@ -812,6 +805,8 @@ impl Actor {
.map_err(|_| ClientError::ConnectTimeout)?
.map_err(ClientError::DialIO)?;

tcp_stream.set_nodelay(true)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥

iroh-net/src/relay/codec.rs Show resolved Hide resolved
@rklaehn
Copy link
Contributor

rklaehn commented Apr 11, 2024

I just looked at the benchmarks and did not see any breathtaking improvements. Things seemed pretty much unchanged or even a bit worse. Is that expected?

Anyway, eliminating an entire roundtrip seems like a big win in any case, even if it does not show up in the benches.

@dignifiedquire
Copy link
Contributor Author

in US case we drop around 200ms by the end for connection, do not sure what you mean

@rklaehn
Copy link
Contributor

rklaehn commented Apr 11, 2024

in US case we drop around 200ms by the end for connection, do not sure what you mean

Probably should not comment without first ☕ ... I looked at the latency numbers.

@dignifiedquire dignifiedquire changed the title perf: simplify relay handshake perf(iroh-net): simplify relay handshake Apr 15, 2024
@dignifiedquire dignifiedquire added this pull request to the merge queue Apr 15, 2024
Merged via the queue into main with commit 70db5fb Apr 15, 2024
22 checks passed
@link2xt
Copy link
Contributor

link2xt commented Apr 17, 2024

In changelog it is marked as performance, should probably be somewhere at the top of changelog saying it is a breaking change. We were running 0.13 relay, but 0.14 clients failed to connect until we upgraded the relay to 0.14 as well.

@dignifiedquire
Copy link
Contributor Author

BREAKING:
This breaks the relay handshake, and so requires updated relay nodes to work with.

from the description above 😅

@dignifiedquire dignifiedquire deleted the feat-relay-handshake branch April 17, 2024 20:14
@link2xt
Copy link
Contributor

link2xt commented Apr 17, 2024

BREAKING:
This breaks the relay handshake, and so requires updated relay nodes to work with.

from the description above 😅

Yes, I have seen it, but this probably should should be on top of the https://github.com/n0-computer/iroh/blob/main/CHANGELOG.md entry

ppodolsky added a commit to izihawa/iroh that referenced this pull request Apr 19, 2024
ppodolsky pushed a commit to izihawa/iroh that referenced this pull request Apr 20, 2024
The relay handshake is quite expensive currently

- TLS 1.3 + HTTP 1 UPGRADE
- Server sends FrameType::ServerKey
- Client sends FrameType::ClientInfo
- Server sends FrameType::ServerInfo

This simplifies the protocol to 

- TLS 1.3 + HTTP 1 UPGRADE
- Client sends FrameType::ClientInfo

information changes
- using a signature, instead of a shared secret to ensure the client
identity.
- server key is not sent (use certificate pinning if fixed relays are
important)
- remove unused configuration in the info frames
- change magic from `derp` to `relay`
- increase protocol version to `3`
- drop `serverinfo` in favor of fixed config on the client
- enable tcp nodelay
- switch to AES for encryption in favor over chacha
- the server simply aborts when the client version doesn't match
(matching what we do in other protocols)


Benchmarks:
https://gist.github.com/dignifiedquire/30131dbe8b87fb799a971068899656ef


BREAKING: 
This breaks the relay handshake, and so requires updated relay nodes to
work with.

---------

Co-authored-by: Floris Bruynooghe <flub@n0.computer>
Co-authored-by: Kasey <kasey@n0.computer>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

5 participants