Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: add @helia/http to monorepo #372

Merged
merged 20 commits into from
Jan 17, 2024
Merged

feat!: add @helia/http to monorepo #372

merged 20 commits into from
Jan 17, 2024

Conversation

achingbrain
Copy link
Member

@achingbrain achingbrain commented Jan 9, 2024

Adds code from https://github.com/meandavejustice/helia-http to the monorepo, preserving commit history.

Fixes: #289 and #344

BREAKING CHANGE: the libp2p property has been removed from the Helia interface in @helia/interface - it is still present on the return type of createHelia from the helia module

@achingbrain achingbrain requested a review from a team as a code owner January 9, 2024 09:40
@2color
Copy link
Member

2color commented Jan 9, 2024

I think you still want libp2p here, only configured without transports or listen addresses - instead configured with only http delegated routing clients as in https://github.com/libp2p/js-libp2p-example-delegated-routing/blob/main/src/libp2p.js - this will allow for it's use with @helia/ipns.

Besides libp2p being required for @helia/ipns, what's the reason you'd still want libp2p? If you're only fetching from gateways, you shouldn't even need delegated routing.

@achingbrain
Copy link
Member Author

Besides libp2p being required for @helia/ipns, what's the reason you'd still want libp2p?

Basically anything that uses peer/content routing, which is most things outside of block retrieval. The future is for PUTs on gateways so by configuring HTTP peer/content routers we can use the same code paths as everything else and not need to maintain extra libraries.

@2color
Copy link
Member

2color commented Jan 9, 2024

Basically anything that uses peer/content routing, which is most things outside of block retrieval.

The whole point of @helia/http is to be focused solely on verified block retrieval from gateways, which wouldn't use peer/content routing.

The future is for PUTs on gateways so by configuring HTTP peer/content routers we can use the same code paths as everything else and not need to maintain extra libraries.

What's the source for this? (are you referring to data onboarding via gateways?). This sounds like a premature optimisation for a use-case that hasn't even been planned.

@achingbrain
Copy link
Member Author

The whole point of @helia/http is to be focused solely on verified block retrieval from gateways

If this is the case then maybe it doesn't belong in the @helia namespace since this implies a certain amount of compatibility with the other modules in the ecosystem.

I think it's completely reasonable for a user to assume that @helia/http will work with @helia/ipns for example, which can be enabled by configuring a bare-bones libp2p with @helia/delegated-routing-v1-http-api-client to support IPIP-379.

I'm interested in the resistance to this, could you speak a bit more to that?

@2color
Copy link
Member

2color commented Jan 9, 2024

If this is the case then maybe it doesn't belong in the @helia namespace since this implies a certain amount of compatibility with the other modules in the ecosystem.

That's an important point to consider. This idea for the package started initially with discussions around verified retrieval in browsers for dapp developers.

I think it's completely reasonable for a user to assume that @helia/http will work with @helia/ipns for example, which can be enabled by configuring a bare-bones libp2p with @helia/delegated-routing-v1-http-api-client to support ipfs/specs#379.

I agree. The boundaries between libp2p and Helia are a bit blurry. I presumed that you'd be able to use delegated routing too, without libp2p, but that may arise from a misunderstanding of the boundaries between the two.

I'm interested in the resistance to this, could you speak a bit more to that?

  • The need for this came up in the context of dapp developers who are currently doing async fetch of unixfs data/json from trusted gateways.
  • The idea was that to encourage verification, we need to make it as easy as possible and try to abstract as much as we can.
  • My thinking was that dapp developers shouldn't be concerned with libp2p, either by not relying on it as a dep, or by abstracting as much as possible
  • Another point —admittedly a hunch I don't have data to back up– is around package size, and security implications with a larger dependency tree that may put off dapp develoeprs,.

packages/core/README.md Outdated Show resolved Hide resolved
packages/http/src/index.ts Outdated Show resolved Hide resolved
packages/http/src/index.ts Outdated Show resolved Hide resolved
@achingbrain
Copy link
Member Author

The boundaries between libp2p and Helia are a bit blurry.

@2color I've opened #378 - I think it might draw a bit more of a dividing line between them?

The need for this came up in the context of dapp developers who are currently doing async fetch of unixfs data/json from trusted gateways.

Ah, ok cool - this sounds like the sort of use case we'd be encouraging people to use https://github.com/ipfs/web3-fetch/ for?

Obviously this PR is needed to implement that functionality but it's got the nice familiar fetch-style API, etc.

@2color
Copy link
Member

2color commented Jan 12, 2024

@2color I've opened #378 - I think it might draw a bit more of a dividing line between them?

Thank you!

Ah, ok cool - this sounds like the sort of use case we'd be encouraging people to use https://github.com/ipfs/web3-fetch/ for?

Exactly!

Obviously this PR is needed to implement that functionality but it's got the nice familiar fetch-style API, etc.

Yep! 💯

packages/http/src/index.ts Outdated Show resolved Hide resolved
Removes all libp2p/bitswap deps from `@helia/interface` but retains
them in `helia`.

Adds a general `Routing` interface to `@helia/interface` that can
be used to resolve IPNS names amoung other things.

This means `@helia/http` can be libp2p-free.
packages/core/README.md Show resolved Hide resolved
packages/interface/src/index.ts Show resolved Hide resolved
packages/http/test/factory.spec.ts Outdated Show resolved Hide resolved
packages/helia/src/index.ts Outdated Show resolved Hide resolved
packages/core/src/routing.ts Show resolved Hide resolved
@achingbrain achingbrain changed the title chore: add http module to monorepo feat!: add @helia/http to monorepo Jan 16, 2024
@achingbrain achingbrain linked an issue Jan 16, 2024 that may be closed by this pull request
Copy link
Member

@2color 2color left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat having the routers abstraction, thanks for getting this in shape!

I left some a few cosmetic comments, but looks generally good.

@achingbrain
Copy link
Member Author

I'm going to merge this so we get RCs published for further experimentation.

@achingbrain achingbrain merged commit 76220cd into main Jan 17, 2024
34 checks passed
@achingbrain achingbrain deleted the chore/add-helia-http branch January 17, 2024 07:06
@achingbrain achingbrain mentioned this pull request Jan 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: 🔎 In Review
Archived in project
Development

Successfully merging this pull request may close these issues.

Expose libp2p-style content/peer routing API? feat: create @helia/http library without bitswap/libp2p deps
4 participants