EIP-8: devp2p Forward Compatibility Requirements for Homestead

[fjl]

This is the first Networking EIP, in which I am trying to sneak some changes
into all implementations before the Homestead release. These changes should
make network protocol upgrades a lot easier in the future.

Merry 🎄 everyone.

[karalabe]

Aren't the hello packets being sent out immediately as the connection is established? Also not sure about it, but AFAIK there are no ordering guarantees between who sends such a packet first or second, so I don't really see a way to send the lower of the two, if both sides have to wait for the remote one to figure out the correct version.

Edit: Perhaps you meant not that they should reply with the lowest, but rather that they should use the lowest version throughout the communication?

[subtly]

Agree this requires clarification and that neither side should wait on the other side to send their version and either side can choose to disconnect or continue communication if the version is different. The proposed specifications are ambiguous because they state that either side can do what they want. This can be simplified to:

• Once a secure connection is established a Hello packet or Disconnect packet MUST be sent
• The hello packet MUST contain the highest version supported by the client

[fjl]

I will clarify this in the text, it was definitely not meant to imply that implementations should wait for the remote hello packet.

[karalabe]

In my opinion the proposal is solid apart from the RLP encoding question I've raised above. That should be looked at by someone more capable in cryptography.

I would however like to emphasize that this would be a very high priority feature to get into the Homestead fork, as the protocol in it's current state is not upgradable without a hard fork. There are issues with the current protocol already, and as time progresses I'm certain we'll find plenty more, or simply much needed extensions that cannot be implemented due to the forking requirements.

The most important aspect IMHO about the solution proposed by Felix is that it's extremely simple. It could be added to any implementation in a matter of minutes, which is imho worth doing now and paving the way forward to easier upgrades afterwards.

The upgrade mechanism itself is quite crude as any version bump requires adding fields to the end, no fields can be deleted or semantically changed (to keep backwards compatibility), but as we're not aiming to do many updates, just to facilitate occasional essential ones, I think this is fine. And as we're transitioning through metropolis and serenity, we can take the lessons learned from previous updates and refine/clean in each of these hard forks.

Best case scenario we can seamlessly upgrade the devp2p protocol as need arises, worst case scenario we don't use the upgrade mechanism, but it won't hinder anything either.

[subtly]

Can be simplified to:

• RLPx UDP packets with unknown type should be ignored

Why the specification to ignore additional list elements?

The version number in Ping/Pong packets shouldn't be ignored. It is to be cached and/or checked against existing metadata and maybe further verified as a compatible version following encrypted handshake.

[fjl]

I will clarify this too. "Ignore" may suggest that the version number should be discarded, which is not the intended meaning. It is simply meant to say that nodes with mismatching version should still be included in the discovery table.

The bit about additional list elements allows for backwards-compatible upgrades in the same manner as for the devp2p hello packet: If any packet is extended to include more information in a later version of the protocol, older nodes will simply ignore these elements.

[subtly]

Can this be a nested tuple? Something like:

auth-body     = rlp.list(auth-vsn, rlp.list(sig, initiator-pubk, initiator-nonce OR token))

Thus:

• The first item in auth-body will always be VERSION
• The second item in auth-body will always be a list for HANDSHAKE (3 items)

Notably, any extensions would need to be very strict to ensure they aren't introducing plaintext elements.

[wanderer]

Currently the id's are 64 bytes. Can we change them 32 bytes (a compressed public key)?

[subtly]

No plaintext. The problem with this is that the length will always be greater than X, and likely equal to X, so it can be inferred that the connection is rlpx. Leaking information (aka plaintext) is not within the design rationale of RLPx or any encrypted protocol.

[fjl]

See #49 (comment)

[subtly]

@karalabe If homestead is a hard-fork I think we should just fix the protocol. A design rationale behind RLPx is that communication MUST NOT rely on the authentication handshake for metadata, and that metadata is to be exchanged via discovery. The solution is so simple: Try to connect to the node via discovery first! This way both sides will authenticate what version they know they can speak before attempting the handshake – its also easy to write a discovery (UDP) stack which supports multiple versions. Case in point, a peer cannot connect to a node prior to discovery – as discovery is how the public key is obtained. No discovery == no TCP handshake.

[subtly]

@karalabe @fjl @wanderer

For the handshake, what do we need to consider other than the scenarios below?

Node AAA, Version 1
Node FFF, Version 2

Scenario 1: Node AAA(v1) attempts connecting to FFF(v2)
Scenario 2: Node FFF(v2) attempts connecting to AAA(v1)

Note: This upgrade is more complex than it seems. The necessity of this upgrade is in order to fix the way the KDF used and to enable framing. Ideally we also remove AES padding. What this means is that if new clients are to support old clients they will need to be able to enable the new framing system on a per-session.

[wanderer]

@subtly I'm not planning on supporting backwards compatibility.

[fjl]

And that's fine. You can always drop connections with lower version (the EIP doc explicitly permits this).

[fjl]

Test vectors have been added.

[fjl]

@subtly (aggregating our discussion a bit)

Your concerns about the RLPx handshake changes seem to be:

• Adding plaintext of any form is bad. I agree that plaintext that leaks secrets is bad, but the size of the handshake is not a secret.
• The size prefix makes identifying RLPx TCP connections easy. My take on it is that the fixed size handshake is just as recognizable and that randomizing the length offers adequate protection in most cases.
• The handshake should not carry metadata, it should be communicated through discovery instead. The issue with this is that discovery isn't always available (e.g. when connecting through Tor). We are also planning to offer discovery over RLPx/TCP in the near future, in which case the RLPx handshake is the entry point.
• It requires a backwards-compatible implementation for the RLPx v5 KDF changes. If you don't care about backwards-compatibility, you can refuse communication with older nodes. The change merely shifts responsibility so the newer node is in control about this decision.

[Gustav-Simonsson]

After discussing with @fjl and @nagydani I think we need to write up a spec of exact requirements of RLPx in terms of threat models - what attacks are we interested in defending against right now, and which can we leave for future protocol upgrades. It's hard to have a discussion of details such as leaking plaintext and/or ciphertext length without having consensus on threat models & general reqs.

[fjl]

For anyone following this discussion: A lot has happened. We've had multiple long calls about EIP-8, a counterproposal and a rebuttal of the counterproposal have been written.

I've updated the text of EIP-8 to clarify some things. Notably, Postel's Law is now mentioned because it
might not be so obvious that it is at play here. A new Rationale section on the discovery change has been added, and there is now a section exploring the implications of the plaintext size prefix for the handshake packets.

[wanderer]

However, such an attacker can also be expected to read and or participate in RLPx Discovery traffic, in which case effective blocking is possible irrespective of the size prefix.

So I think in that case the discovery traffic wouldn't be UDP. It could be by written notes sent by pigeons, ect. But the point is just because the normal method for discovery is blocked the transport can still operate.

[fjl]

The document assumes nodes talking using the currently specified protocols, not a version where arbitrary transport is negotiated using an arbitrary method of discovery. Not sure whether this needs to be made explicit. We can improve on everything in the future, but for the RLPx protocol that is in use right now, correlating UDP traffic is definitely possible (and there are other indicators, including commonly used ports, the fixed-size handshake traffic pattern, the first byte always being 0x04, etc.).
I guess what I'm trying to say is that the prefix won't make things worse than they are now.

[wanderer]

The document assumes nodes talking using the currently specified protocols, not a version where arbitrary transport is negotiated using an arbitrary method of discovery

I think I have a slightly different view here. I view the three protocols (discovery, handshake, transport) as being mutually independent and composable.

[fjl]

This is my abstract view, too, but when discussing the possibility of filtering RLPx v4 TCP connections
we need to look at what people actually use. There are two choices for users now: using UDP discovery or supplying the pubkey+endpoint. There won't be a third option just tomorrow, unless we implement it (or use a library for it ;)) and it gets adopted.

[fjl]

Anyway, I've fixed up the sentence to state explicitly that TCP connections could be blocked by looking at UDP traffic, which was the intended meaning.

[fjl]

Another short overview document (as requested by @chriseth): https://gist.github.com/6c6d24dc1c7ab5d77c5c

[fjl]

Note: test vectors need to be regenerated for the size prefix authentication change. I'll do that and then squash all the commits together.

[fjl]

Test vectors are now up to date.

[fjl]

Squashed.

[karalabe]

Could we get this merged into the EIP repo if it was accepted for homestead? (Also specify it as "Accepted"?)

[wanderer]

@karalabe yes definitely! @fjl could change the status to "accepted" on this? It would b ready to merge then

[fjl]

Done.