abstraction for transport level streams

[whyrusleeping]

Some transports we will add in the future (example: QUIC) will support opening new streams at the transport level instead of requiring we use a stream multiplexer. This won't work with our current level of abstractions, we will need some thought around how to make this work.

[daviddias]

The way I do it in js-libp2p is that I break it down in 3 subsets

• transport - That follow interface-transport, meaning it has a dialing + listening interface https://github.com/diasdavid/interface-transport
• connection - The duplex pipe that we get when 'we dial a stream' https://github.com/diasdavid/interface-connection
• stream-muxer - a connection that 'was upgraded' (or is yield by default like so) that offers the .newStream api, that yields a new connection object to the same peer. https://github.com/diasdavid/interface-stream-muxer

I keep 'warmed' conns in a different pool than the pool of connections that were 'stream muxed', and when libp2p.dial is called, it checks if there is a stream muxer to the peer, if not, checks if there is a warmed conn (a conn that was established but never dialed a protocol on it, useful for Discovery protocols to attempt to dial, and not consume the stream), and if not, then do the transport dance.

[marten-seemann]

Hi guys,

I just started integrating quic-go into IPFS. I already had a brief discussion today with @whyrusleeping, and we agreed that it might be helpful to have a transport that is able to signal its abilities to the swarm (i.e. if it supports transport level stream multiplexing).

Fundamentally, the main difference is that the Conn returned by Dialer.Dial and Listener.Accept in the Transport interface (see here) has to look completely different than a net.Conn (e.g. it doesn't have Read and Write methods, but AcceptStream and OpenStream) for stream-multiplexed transports.

I'm not really sure what the cleanest solution is here, since I'm pretty new to the libp2p code. Would it make sense to define two separate Transport interfaces?

[whyrusleeping]

@marten-seemann Hrm... the quic conn will be closest to the go-libp2p-net interface here: https://github.com/libp2p/go-libp2p-net/blob/master/interface.go#L50
It looks like it will plug in at roughly the same layer as go-peerstream: https://github.com/libp2p/go-peerstream/blob/master/conn.go

The question we need to ask first though is whether or not we want to run secio over quic. I think the answer is no, but i'm not sure.

[whyrusleeping]

Looking into this more, I think adding capabilities to transport.Conn and then checking those in go-peerstream.addConn will probably be the best way forward. In that method we can check if the conn natively supports stream multiplexing and skip most of the setupConn code that is used for setting up the multiplexer. Doing things this way we shouldnt even need to change the transport.Listener interface.

I think on making transport.Conn a much slimmer interface, we have two options:

• Make the Read/Write calls on the QUIC implementation of transport.Conn return errors (noop)
• Slim the interface down to maybe just Close() and Transport() and then introduce new child types that we can type assert back up (this could be a way to implement the 'capabilities' mentioned earlier)

Having written those down, i'm thinking that the second one might end up being the cleanest solution.

[marten-seemann]

@whyrusleeping The handshake question is something I've thought about a lot already (and also discussed with Juan), and we decided to use QUIC's TLS for doing the handshake.
That being said, there are currently two different things called "QUIC": the QUIC deployed by Google (and implemented in quic-go) and the IETF WG draft. As Google regularly releases new Chrome versions, those two QUICs are expected to converge eventually, but it might take some time. The current version of QUIC doesn't use TLS yet, but its own crypto handshake, which unfortunately doesn't support client authentication.

It would be nice if we define the interface such that we have the option to do the secio handshake over a stream-multiplexed connection. I'd use that in QUIC at first, and switch it off as soon as QUIC implements TLS (this will save us a couple of RTTs during the handshake).

[whyrusleeping]

@marten-seemann hrm... that won't work. secio isnt just a handshake, its actually the encryption layer that everything else needs to run on top of.

Lets just go with whatever works for now (a.k.a. get it working).

[daviddias]

We currently have some disparity between Go and JS interfaces at the transport/conn level, so bear in mind that I try to follow the language defined through our interfaces and spec

I'm with @whyrusleeping that we don't need to change the Conn interface to signal its properties, special since Conn should be the bare minimum interface to send/receive data over a symmetric stream.

I'm also up for the 'let's get it to work', even if it means to do secio over QUIC TLS. Currently, we have the WebRTC Crypto handshake (which is pretty much the same as QUIC) and on top we have secio.

@marten-seemann I've had the conversation with @jbenet on how to signal the capabilities of each transport too and we kept trying to come with a clean language in the limited time we had in the past to dedicate to it. We've made some progress, but nothing yet to be considered finished.

In JS, libp2p-swarm handles all the dialing and listening on different transports + upgrading the connections to a crypto channel and stream muxer. Right now its language enables us to define several combinations, but we haven't included any reasoning over different transport capabilities, it just puts all of them in the same box. Nevertheless, it should be inside libp2p-swarm (or the name we end up picking for it as right now it might be misleading https://github.com/libp2p/js-libp2p-swarm/issues/40), that as we dial and listen for connections, we realize what we already 'bought into' that connection and what we need to add on top to fulfill all the requirements.

This could be achieve today by just adding a properties object to each transport, but in the future we would like to have a language for it ipfs/notes#209

[ghost]

I agree with all of the above, and another future transport capability in addition to stream muxing and crypto, will be reliability. Depending on the swarm's needs regarding an individual connection, capabilities will be transparently wrapped around the connection.

In the midterm it would be really awesome if DTLS from quic-go could be reused as a standalone capability, e.g. for wrapping a PacketConn. That'd enable us to finally have the swarm expose datagram communications, not just streams. (The same about standalone capability goes for QUIC's reliability and congestion control parts.)

[daviddias]

Can QUIC in libp2p be tested today? Now would be a good time how a transport can expose the properties it offers and therefore reduce the amount of other steps that are normally done.

[Stebalien]

Integrating QUIC is currently in progress (bad timing given that @whyrusleeping isn't here right now and I'm not sure how he'll feel if I start accepting sweeping interface changes...).

[marten-seemann]

@diasdavid: You can, but it will be a lot work. You'd have to check out all the PRs referenced in libp2p/go-libp2p-transport#20. I had to change a bunch of interfaces to make native stream-multiplexing work in libp2p. I hope we can get these changes merged as soon as Jeromy returns.

Is there anything you want to test in particular?

[daviddias]

@marten-seemann I'm interested on exposing all the properties that QUIC has so that we can start having dials that skip SECIO in favour of the QUIC crypto handshake (although we still want to do the challenge for PeerId)

You mention that is hard to plug this transport in, any hope of getting transports be modules that are plugged in into the instance in the same way that happens in js-libp2p ? See https://github.com/libp2p/js-libp2p/tree/master/examples/transports for more details.

[Stebalien]

skip SECIO

Currently, we do SECIO over each QUIC stream because QUIC only allows us to authenticate connections in one direction (i.e., the client can verify the server's identity). We can run a simple handshake protocol over an "authentication" stream before declaring the connection secure to authenticate the connection in the other direction however:

1. This would mean auditing a new security protocol. @whyrusleeping and I came up with a protocol we think will work but...
2. When QUIC gets TLS 1.3, we likely won't need it anyways (TLS 1.3 supports authenticating both servers and clients).

---

Protocol if you're interested:

1. Client sends nonceC to the server.
2. Server signs (with their Peer key) (nonceS, serverPeerID, serverQUICPublicKey, nonceC) to client.
3. Client verifies that it is talking to a server using serverQUICPublicKey and that nonceC is correct and then signs (clientPeerID, nonceS, serverPeerID, serverQUICPublicKey, nonceC) and sends it back to the server.
4. The server verifies everything in this bundle.

• nonceC and step 3 allows the client to verify that the server currently controls the peerID (without forcing the server to use their peer key as their QUIC key).
• nonceS and step 4 allows the server to verify that their talking to the correct client and that the client knows which server it's talking to.

Note: To save bandwidth, we could send a hash of known information instead of sending it back and forth. That is, the signed message in step 2 could be (nonceS, hash(serverPeerID, serverQUICPublicKey, nonceC)) and the signed message in step 3 could be (clientPeerID, hash(nonceS, hash(serverPeerID, serverQUICPublicKey, nonceC))).

[Kubuxu]

Does the client get anywhere the PubliKey/Cert of the server in the API? I can't find it anywhere: https://godoc.org/github.com/lucas-clemente/quic-go#Session
Also we run QUIC in with InsecureSkipVerify which might be additional problem. I am also worried about key renegotiation, I don't know if it is part of QUIC spec but if it is it might cause a problem.

In my opinion, deploying QUIC with crafted crypto on top to verify the identity is very risky. I am not sure if it is a risk we want to take.

---

secio on one QUIC stream is not enough as it might be MitMed, we would have to do it in every stream or as how @Stebalien shown, by binding server authenticity to the public key used by QUIC.

[marten-seemann]

At the moment, we're not using any security guarantees of the QUIC crypto at all. The only reason the QUIC server sends a certificate is because there's no way to run QUIC unencrypted (the protocol explicitly forbids this). For reasoning about security properties, you should treat the QUIC connection as unencrypted.

This is due to the fact that the currently implemented version of QUIC doesn't support client certificates (Google didn't feel the need to design for and implement that). This will change as soon as I implement TLS 1.3 for QUIC, which I'm planning to do later this year, but this is far from a trivial change, and might take a couple of months.

Connection establishment over QUIC

Considering that we currently don't use the QUIC crypto, here is how QUIC connections are secured (all the code for that is in libp2p/go-libp2p-conn#9):

1. We're running the secio handshake on one stream. From this we gain two things: Firstly, we verify that the peer is who he says he is. Second, we get a secio.Session running on top of this stream.

At this point, the peers can send encrypted data on the handshake stream. All other streams are not secure, and could be MITMed. To prevent this, we

2. use the handshake stream to establish a pre-shared key. This PSK is then used to encrypt all other streams using the go-libp2p-protector (see https://github.com/libp2p/go-libp2p-conn/pull/9/files#diff-a2ddca1600f34abb91244424f8431c2cR138).

The security properties of this handshake should be equivalent to secio + protector (please let me know if you disagree). I agree that from a conceptual standpoint, this feels a little bit hacky, and I'm really looking forward to get rid of this in favor of TLS 1.3 soonish. However, I'd be happy if this doesn't block the merge.

[Kubuxu]

The go-libp2p-pnet Protector does not authenticate data, it was designed for use on a layer below than Authenticated Encryption so it didn't need it. Which if we treat the QUIC encryption as MitMed then it means that data is not authenticated and can be for example arbitrary bit flipped at the start of the stream when it is predictable.

• MitM the QUIC connection
• proxy the first stream for secio
• every new stream uses non-authenticated encryption, and starts of stream are stable, or message patterns can be detected which means that it is possible to bitflip encrypted stream to change it and have it interpret correctly

---

RFC7250 (proposed standard) added raw public key key handling to TLS but it needs full public key (possible to get from DHT) and QUIC would have to use the full private key as its server cert.

[marten-seemann]

@Kubuxu: You're right, I didn't think of that. RFC7250 sounds interesting, we might be able to use that with QUIC. That means we probably should wait with the QUIC rollout until TLS 1.3 works properly.
However, I still hope that my PRs for the interface changes can be merged soon (just without implementing the tpt.MultiStreamConninterface).

[Stebalien]

Fixed.