Enhancement: SCTP for reliable UDP

[linas]

As you know, I've been trying to use OpenDHT to host a distributed shared database. An ongoing problem until now has been intermittent data loss when a single node publishes to the network, and then exits (hit and run). It appears that sometimes not all of the data from the singleton makes it out.

I just now realized this is due to the use of UDP as the underlying protocol for OpenDHT. UDP is not reliable: the operating system, the network cards, and network routers are free to discard UDP packets when there is traffic congestion. it doesn't take much for this to happen.

The best way to get reliable datagrams is SCTP - https://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol - its widely available, it works, and it avoids data loss at the network level.

I'll take a few minutes and see how hard this might be...

[linas]

There seem to be two ways of using this: directly (linux native) or via a portable lib.

• libsctp-dev -- direct, provides http://man7.org/linux/man-pages/man7/sctp.7.html and more generally here: https://tools.ietf.org/html/rfc6458
• libusrsctp-dev -- "portable" https://github.com/sctplab/usrsctp

The problem with the first one is that it's not available on Windows; also, some NAT firewalls may intefere(?). The second one provides a driver that sits on top of UDP; it might not be as fast...

[linas]

I'm thinking of the following:

• If HAVE_SCTP defined, then compile the native bindings.
• Use port 4333 as the default for SCTP connections
• Attempt to connect to a node via SCTP first; if not available, then fallback to UDP.
• Disable one or the other with config.dht_config.node_config.enable_sctp and enable_udp
• Create class OPENDHT_PUBLIC SctpSocket : public DatagramSocket
• Hack NetworkEngine::requestStep as needed.

As usual, I suspect this is one day's worth of work that will take two full time weeks to actually make it work :-( How hard can this be?

[ghenry]

How did you get on with this?

[linas]

How did you get on with this?

Nothing was implemented. I stopped using OpenDHT for unrelated reasons. I was interested in OpenDHT for a very specific project, and eventually realized that the architecture I was imagining was incorrect/inappropriate for that particular project. Without that motivation, I lost interest in OpenDHT.

(For the project I was interested in, I eventually realized that what I really needed to have was constant, reliable "seeders" that are available 24x7 on the internet, and that the most I could use OpenDHT for was as a distributed directory service. In other words, I realized that I needed a very traditional distributed architecture. That I could not use OpenDHT to "magically" solve the persistence and replication and query issues I was facing. A bit sad, I suppose, that the magic answer to all problems doesn't exist.)

[ghenry]

Ah, makes sense. Thanks for the detailed explanation Linas.

[aberaud]

We might use QUIC, the spiritual successor to SCTP, in the future.

This would be OpenDHT 3!

No schedule for now. Keeping the ticket open for related discussions.

[Lanius-collaris]

This would be OpenDHT 3!

Is it related to dhtnet?

[aberaud]

@Lanius-collaris Somehow, yes.
Dhtnet uses OpenDHT, an ICE implementation (pjnath) and a TLS library (gnutls) to provide peer-to-peer, authenticated and secure connectivity establishment.

[Allium-altaicum]

How about integrating asio-utp (uTP is stream oriented, so need a framing method) ?