Eclipse-attack hardening for kad-dht: our approach, and why we dropped randomized lookup selection (#1383 / #1384)

[yashksaini-coder]

What this is about

We've been working on making py-libp2p's Kademlia DHT harder to eclipse. This post explains our thinking in plain terms, so anyone reviewing the related issues/PRs can see the reasoning without digging through code.

The problem, simply

A peer ID is just a hash of a public key — anyone can make as many as they want, for free. An attacker can generate lots of IDs that sit "close" (in Kademlia's XOR distance) to a key they want to censor, and run them all from a handful of machines. If enough of those fake peers fill the slots nearest that key in everyone's routing table, honest nodes can only reach the attacker's peers for that key. That's an eclipse. Because every message is properly signed, signature checks don't help — the attack is about who is in the routing table, not about bad data.

Our approach: two layers

Layer 1 — Don't let one network fill the slots (issue #1383).
Real IP addresses cost real money; peer IDs don't. So we cap how many peers from the same subnet can sit in one bucket (default: 2 per /24 for IPv4, /48 for IPv6). Now an attacker needs many different networks, not just many keypairs from one machine. This is implemented and opt-out-safe (localhost, private, and relayed peers are exempt, so CI and local testnets are unaffected). PR: #1399.

Layer 2 — Don't let one bad peer steer a lookup (issue #1384).
A lookup walks hop by hop toward a key. If an attacker controls a peer early in that walk, they can shape the rest of it.

Why we dropped the first idea for Layer 2

Our first idea for #1384 was simple: instead of always asking the closest peers first, pick them randomly from the closest few. The thought was that this removes the attacker's guaranteed early slot.

When we actually built it, it broke the search. The lookup stops as soon as a round finds nothing new. If the random pick happens to choose the farther peers in a round and they return nothing, the whole lookup can stop before it ever asks the genuinely-closest peers — so it can return a worse answer than today. And the obvious fix (always ask the closest first) just brings back the exact guaranteed slot we were trying to remove. This is also why go-libp2p and rust-libp2p don't randomize this step.

So randomizing selection is a dead end: it either hurts the search or gives no benefit.

What we're doing instead

The better-known answer (used by rust-libp2p) is disjoint lookup paths: run several lookups at once that don't share peers. Each one searches normally (so no search quality is lost), and diversity comes from having several independent paths instead of perturbing one. That's the direction for #1384 going forward.

Summary

• security(kad-dht): add IP/subnet diversity enforcement to KBucket #1383 (subnet diversity): done → PR feat(kad_dht): enforce IP subnet diversity in k-buckets (#1383) #1399.
• enhancement(kad-dht): add disjoint lookup paths to find_closest_peers_network #1384 (lookup steering): randomized selection dropped (it regressed the search); moving to disjoint paths.
• Together they cover the two ways an eclipse happens: filling the closest slots, and steering the walk.

Feedback very welcome — especially on the subnet prefix defaults (/24 vs go-libp2p's /16) and whether disjoint paths should default to 2 or match rust-libp2p's parallelism.