Should use correct DHT routing table data structure, per spec

[feross]

We currently use a hacky solution adapted from http://github.com/bmcmahen/bittorrent-dht

Consider learning from:

• https://github.com/stbuehler/node-dht
• https://github.com/nikhilm/kademlia

Specs here:

• BitTorrent spec: http://www.bittorrent.org/beps/bep_0005.html
• Kademlia paper: http://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf

[feross]

https://github.com/jech/dht is a legit implementation used by Transmission, but it's 3000 lines of C and doesn't look easy to port.

[feross]

Also useful:

• KadOH
• Kademlia specs

[dzlab]

here is a ready to use node module https://github.com/indutny/dht.js

[ivantodorovich]

Another one https://github.com/deoxxa/bittorrent-dht-byo

Also interesting: https://github.com/tristanls/k-bucket and https://github.com/tristanls/discover

[ivantodorovich]

Can't we just use one of them? Maybe extend kademlia?

[feross]

Yeah, will consider dht.js. Seems legit.

[ivantodorovich]

👍

[feross]

I just updated bittorrent-dht to 1.0.0. It now follows the spec, contains a proper routing table, answers find_node requests from other nodes, and more efficiently crawls the DHT.

Previously, to find peers we would do a very aggressive crawl where we stuck every new node into a big queue and queried them all, not paying any attention to how close they were to the target info hash. So we had lots of pointless traffic to far away nodes that weren't helping us at all. Now, we only recurse on the K closest nodes, so we find the correct nodes with a lot less UDP traffic.

I'm really happy with the new API. Check it out: https://github.com/feross/bittorrent-dht

[jure]

Nice! Is the idea for bittorrent-dht to one day work over WebRTC as well, i.e. have an exchangeable networking module, or will that be a different project?

[feross]

@jure I would definitely prefer to utilize the work that's already gone into bittorrent-dht when writing the WebRTC version. bittorrent-dht is quite fast, reliable, and well-tested. And also used by the peerflix/torrent-stream projects. So it's going to keep getting better over time.

Perhaps the parts of the module that use dgram can be abstracted out so the majority of the code could be re-used?

[jure]

I see, bittorrent-dht is made specifically to work in Node and with the existing bittorrent network. Just a bit of background: what I'm looking for is not a protocol specific (i.e. bittorrent) DHT implementation, but a general DHT implementation (Chord/Kademlia) in the browser, backed by WebRTC (not UDP). There are a few attempts out there but for some reason all of them are somewhere in the alpha-stage zone.

I know this is very off topic, but if you were to start writing a WebRTC DHT framework, where would you begin?

[piranna]

I know this is very off topic, but if you were to start writing a WebRTC DHT framework, where would you begin?

Write it transport agnostic, in Node.js, but only with APIs available
on the browser (in Node.js would be acceptable to use modules that
implement this APIs).

As a personal side-note, I'm thinking about to fork Chord.js and
remove the dependency of PeerJS, this would be a good starting point.

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux