feat: use ipfs dht

[2color]

This PR changes the protocol ID used by the app to use the IPFS Kademlia DHT.

Open Questions

• Why was a separate DHT id created for this app in the first place?
• What are the implications of using the IPFS DHT for this app if the presumption is that most users are browser users anyways

What roles does the DHT play in this app

Resolving bootstrap node PeerIDs to multiaddrs: My original thinking was that the DHT would be used to make it easier to discover the multiaddrs of deployed Rust and Go peers which have an ephemeral (because of the temp cert which rotates) WebRTC-direct and WebTransport multiaddrs. That way, we can avoid hardcoding ephemeral multiaddrs in the frontend and instead just hardcode the PeerIDs which would be looked up in the DHT. Either way, browsers are still incapable of being reliable DHT clients due to transport constraints.

By using the IPFS KAD DHT maybe we could lean on the delegated routing endpoint in the browser to resolve bootstrap node PeerIDs to multiaddrs. That would require that the Go and Rust peers publish their multiaddrs to the DHT (which should be pretty simple)

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
universal-connectivity	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 24, 2024 3:18pm

[MarcoPolo]

Not sure why a new ID was chosen. Maybe the reasoning is that if folks use this code as a jumping off point we want them to use their own DHT by default instead of using the IPFS dht? If they want to use the IPFS dht, that's fine but maybe it should be a conscious decision.

[maschad]

@MarcoPolo At the time when this originally implemented to be demoed at IPFS thing it was integrated with RC versions of the respective implementations, as opposed to the more commonly deployed instances connected to the IPFS DHT - for instance there were no deployed go-peers that supported webRTC, and so to avoid wasted dials (which we also didn't have a mechanism for dealing with in js-libp2p as efficiently at the time) we just created our own mesh through our own DHT, which gave us faster discovery when we were demoing.

[MarcoPolo]

Thanks for the context! That makes sense. I think it's probably okay to use the IPFS dht now.

[SgtPooki]

I think with using an isolated prefix and some bootsrappers aware of that prefix, its much easier to discover peers you actually care about. Theres a ton of peers in the ipfs dht that could make discovering peers you care about more challenging

[guillaumemichel]

Are the default bootstrappers configured to provide peers with protocol ids other than /ipfs/kad/1.0.0?

[2color]

I think with using an isolated prefix and some bootsrappers aware of that prefix, its much easier to discover peers you actually care about. Theres a ton of peers in the ipfs dht that could make discovering peers you care about more challenging

I'm trying to avoid requiring dedicated bootstrappers for this app because they'd need TLS certificates. The purpose of this PR is to ease the peer routing for known Peers, i.e. the hosted Rust and Go peers.

As you hint, inevitably, the Go/Rust peers will establish connections to peers that aren't relevant for this app, but there's also an advantage, by establishing those connections, their multiaddrs (with ephemeral keys) will be "published" (peer routing is rather implicit) making them easier to resolve from PeerID in the browser.

As for the browser client, I'm a little concerned that this will end up opening a bunch of random connections to the bootstrap nodes, but I'm sure we can avoid that by relying on the delegated routing endpoint like we do in Helia-libp2p (https://github.com/ipfs/helia/blob/main/packages/helia/src/utils/libp2p-defaults.browser.ts#L65).

Are the default bootstrappers configured to provide peers with protocol ids other than /ipfs/kad/1.0.0?

I don't know. I presume not 🤷‍♂️

[2color]

I deployed the latest main and I can already resolve the multiaddrs of the go-peer from the delegated routing endpoint:

https://delegated-ipfs.dev/routing/v1/peers/12D3KooWFhXabKDwALpzqMbto94sB7rvmZ6M28hs9Y9xSopDKwQr

The rust-peer on the other hand isn't resolvable from the delegated routing endpoint. But it's also not resolvable from Kubo. I think that might be because it's not connecting to the libp2p bootstrap nodes.

[2color]

Are the default bootstrappers configured to provide peers with protocol ids other than /ipfs/kad/1.0.0?

@guillaumemichel Most are just Kubo nodes with the exception of the one Rust one

➜  ~ ipfs id QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb
{
	"ID": "QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
	"PublicKey": "CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCpj58Xdfr2RSycD+MW2j0RgjDqCh1p6RCXxIF58hmaRqSlMX6K6fdE05EExMI+rwPZ3AkWy0jftPzcARCAmR3kPcaWyp3fA8ZFl01ebQKXy37jjMELSkjHZWSX30faCvh++mQMsxaR9Gslivq/Kl9g1HM4krYqwuMG/RY/KLsSKPaQLAZ3iD9Q0Ph3BetQBu+RvF2Z7VCEoW7JUgPFYBbPQItvKWfQ2+xvRRWCoJUHgCRUHvpzyt0dlljVTfzfJZYqkypvwEeCQnoCAzsn9qabYSrmOg8q9fCQ7d0tMPoluaMmJJkZWgSsFNT7IPuUDWgjWLZ+elgXk38WahuAA/ZjAgMBAAE=",
	"Addresses": [
		"/dns4/am6.bootstrap.libp2p.io/tcp/443/wss/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
		"/dns6/am6.bootstrap.libp2p.io/tcp/443/wss/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
		"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
		"/ip4/147.75.87.27/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
		"/ip4/147.75.87.27/udp/4001/quic-v1/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
		"/ip4/147.75.87.27/udp/4001/quic-v1/webtransport/certhash/uEiAQI9fwsOm5CuYA5iyGRiNxA8_J_IXgRK-c1vuMjzmv8g/certhash/uEiD3Q2mfd5EYt6Y2M9rsge_nna4hVyCgUVlfSz3IjAK8ew/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
		"/ip6/2604:1380:4602:5c00::3/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
		"/ip6/2604:1380:4602:5c00::3/udp/4001/quic-v1/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
		"/ip6/2604:1380:4602:5c00::3/udp/4001/quic-v1/webtransport/certhash/uEiAQI9fwsOm5CuYA5iyGRiNxA8_J_IXgRK-c1vuMjzmv8g/certhash/uEiD3Q2mfd5EYt6Y2M9rsge_nna4hVyCgUVlfSz3IjAK8ew/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
	],
	"AgentVersion": "kubo/0.28.0/e7f0f34",
	"Protocols": [
		"/ipfs/bitswap",
		"/ipfs/bitswap/1.0.0",
		"/ipfs/bitswap/1.1.0",
		"/ipfs/bitswap/1.2.0",
		"/ipfs/id/1.0.0",
		"/ipfs/id/push/1.0.0",
		"/ipfs/kad/1.0.0",
		"/ipfs/lan/kad/1.0.0",
		"/ipfs/ping/1.0.0",
		"/libp2p/circuit/relay/0.2.0/hop",
		"/libp2p/circuit/relay/0.2.0/stop",
		"/libp2p/dcutr",
		"/x/"
	]
}

[MarcoPolo]

The rust-peer on the other hand isn't resolvable from the delegated routing endpoint. But it's also not resolvable from Kubo. I think that might be because it's not connecting to the libp2p bootstrap nodes.

I was able to find my rust peer using ipfs routing findpeer <peerid>. Shouldn't that be enough?

[2color]

I was able to find my rust peer using ipfs routing findpeer . Shouldn't that be enough?

It's working now since the Rust peer connects to the Bootstrap nodes ( #114)