WIP: update go-libp2p to v0.24.0

[marten-seemann]

This won't be fun. Lots of dependencies to update / release. This is because we need to get rid of the go-libp2p-core dependency (even the indirect one).

Here's the list, in no particular order:

• release v0.6.0 libp2p/go-libp2p-pubsub-router#99
• chore: update go-lib2p, avoid depending on go-libp2p-core go-ipfs-provider#43
• chore: update go-libp2p go-ipfs-keystore#11
• chore: update go-bitswap go-blockservice#94
• chore: update dependencies to avoid indirect go-libp2p-core dependency go-bitswap#595
• chore: update go-libp2p to v0.23.4 interface-go-ipfs-core#92
• chore: update some libp2p dependencies libp2p/go-libp2p-kad-dht#798
• chore: update go-libp2p to v0.23.4, update go.mod version to 1.18 go-namesys#33

[BigLep]

@Jorropo : can you please work with @marten-seemann to get this landed? I want to make sure he has a dedicated point of contact so this doesn't stall. And if it makes sense to take something over, go for it. I leave it up to you guys to decide.

[marten-seemann]

@Jorropo : can you please work with @marten-seemann to get this landed? I want to make sure he has a dedicated point of contact so this doesn't stall. And if it makes sense to take something over, go for it. I leave it up to you guys to decide.

1. All of the PRs (except one, which I'll handle) concern ipfs repos. I'd appreciate if an IPFS steward could handle merging these, cut the respective releases and bubble those releases up to this PR.
2. Other than that, I've seen the linter fail due to an fx deprecation warning. I assume this is because go-libp2p now also uses fx, and is probably bumping the fx version used for kubo (due to minimum version selection). This would need to be fixed inside the kubo code base, which again would best be handled by the kubo team.
3. Getting this PR merged (obviously, we actually need to release v0.24.0 first, but that will happen early this week) is only part of the story. Some migrations are required around QUIC versions and WebTransport, I've opened migrations needed after updating go-libp2p v0.24.0 update #9410 last week explaining the steps needed.

Does that sound reasonable, @BigLep and @Jorropo?

[BigLep]

Thanks for creating the overarching tracking issue @marten-seemann . I had missed it. The plan makes sense. I have it now on in our current 0.18 "iteration".

[marten-seemann]

@Jorropo What are the plans for bubbling up the dependency updates?

[Jorropo]

@marten-seemann Ill take Care of it for 0.18 once 0.24 is released.
It will include turning webtransport on by default and the migration to add webtransport listens to people that listen on QUIC already.

[marten-seemann]

It will include turning webtransport on by default and the migration to add webtransport listens to people that listen on QUIC already.

I'd suggest making this 2 separate PRs. One (this PR) that only updates go-libp2p to v0.24.0 (dealing only with breaking API / dependency changes), and then a follow-up to do the WebTransport-related changes.

[Jorropo]

@marten-seemann sounds like a good plan

[marten-seemann]

@Jorropo Updated the PR to the recently released v0.24.0.

[Jorropo]

When trying to use fx.Populate I get:

ERROR	core	core/builder.go:157	constructing the node: missing dependencies for function "reflect".makeFuncStub (/home/hugo/Documents/Scripts/go/src/reflect/asm_amd64.s:28): missing type: core.IpfsNode
Error: constructing the node (see log for full detail): missing dependencies for function "reflect".makeFuncStub (/home/hugo/Documents/Scripts/go/src/reflect/asm_amd64.s:28): missing type: core.IpfsNode

for some reason, this doesn't need to be investigated now, we can defer this to later, just mutting staticcheck's complain for now.

[marten-seemann]

• chore: update some libp2p dependencies libp2p/go-libp2p-kad-dht#798

go-libp2p-kad-dht v0.19.0 was just released.

[marten-seemann]

I don't think this is ready for review yet. All the packages I linked in the original post still need to be released, and the new versions need to be bubbled up.

[lidel]

@marten-seemann since this is still wip, is it possible to include libp2p/go-libp2p-kad-dht#793?

[marten-seemann]

We released v0.19.0 yesterday.

libp2p/go-libp2p-kad-dht#793 would require a new release, but this is orthogonal to this PR, isn't it?

[lidel]

Yes, this could be a separate release and could be a separate PR.

I am flagging this because "required" list for Kubo 0.18 (#9417) includes #9389 which (iiuc) needs a new libp2p release with libp2p/go-libp2p-kad-dht#794 and libp2p/go-libp2p-kad-dht#793.

So either we need a patch release, or to descope the above from Kubo 0.18. (cc @BigLep)

[marten-seemann]

I am flagging this because "required" list for Kubo 0.18 (#9417) includes #9389 which (iiuc) needs a new libp2p release with libp2p/go-libp2p-kad-dht#794 and libp2p/go-libp2p-kad-dht#793.

go-libp2p-kad-dht is not a dependency of go-libp2p, it's the other way around. So no new go-libp2p release required.

[lidel]

Ah, sweet! In that case is it ok for me to just merge libp2p/go-libp2p-kad-dht#794 and libp2p/go-libp2p-kad-dht#793 and make a new v0.20.0 release of go-libp2p-kad-dht?

[Jorropo]

@lidel I've bumped go-libp2p-kad-dht to v0.19.0 in ipfs/go-namesys#33 as long as you don't break v0.19.0's api in v0.20.0 (which neither of the PR you linked do), I'm good.

[lidel]

Pinged some folks for review on go-libp2p-kad-dht PRs, but it should not block this PR.
We can bump in a follow-up one.

[lidel]

@Jorropo before you merge this PR, please incorporate go-libp2p-kad-dht update from #9491 🙏

[Jorropo]

I finally fixed it by doing this: https://github.com/ipfs/kubo/blob/ee11ef6d2ca0df494e4d709f8354109452185a47/internal/i-hate-what-I-am-doing-here.go it's horrible looking but we are good enough since this isn't actually compiled.