feat!: integrate with node HTTP servers, WebSockets and others #57

achingbrain · 2025-03-28T16:45:31Z

Provides adapters for working with W3C Request/Response based servers (e.g. Hono) and with Node.js http.Server ones (basically everything else)
Adds support for opening WebSockets over libp2p and integration with existing WebSocket servers
Ensures server code is opt-in to reduce client-only bundle sizes
Adds browser-friendly server capabilities to allow serving webapps from browsers
Allows using libp2p as the routing layer with Node.js native fetch and/or http client via custom agents
Updates all examples to use new API

BREAKING CHANGE: the API has change significantly, this module has been renamed @libp2p/http since it now does much more than fetch

TODO:

Ship fix: handle dialing /p2p/Qmfoo-style addresses js-libp2p#3064
Integrate peer id auth with the .fetch and .connect methods
Polish API
Update README
Update examples

Change checklist

I have performed a self-review of my own code
I have made corresponding changes to the documentation if necessary (this includes comments as well)
I have added tests that prove my fix is effective or that my feature works

…ibp2p Supports running libp2p streams over HTTPm, as well as HTTP requests over libp2p. Also supports upgrading HTTP-over-libp2p requests to WebSockets, as well as libp2p-over-HTTP requests so you can do full-duplex libp2p things and aren't limited to simple request/response protocols. As a bonus, it also supports accepting HTTP & WebSocket requests in browsers without having to polyfill and Node.js internals. BREAKING CHANGE: the API has change significantly, this module has been renamed `@libp2p/http` since it now does much more than fetch

achingbrain · 2025-03-28T16:47:28Z

Opened as a draft as it still needs some polish and the tests won't pass until libp2p/js-libp2p#3064 is shipped.

MarcoPolo · 2025-04-07T16:56:49Z

Supports running libp2p streams over HTTP

I don't understand this. Maybe there's some confusion: libp2p/js-libp2p#3030 (comment)

MarcoPolo · 2025-04-09T21:47:07Z

I think the confusion is resolved. Happy to review this PR once it's ready, please tag me :)

MarcoPolo

I've skimmed the code and the examples.

Holy smokes, this is a herculean effort! You've done an incredible amount of great work.

If there's anything you want me to dive deep on, I'm happy to give you a review. If you want me to review the whole thing it'll take some time.

In the mean time, I'd like to update the Go implementation here to the latest version. I'll open a PR targetting this branch

README.md

MarcoPolo · 2025-05-20T03:12:20Z

examples/serving-websites-from-web-browsers/README.md

+
+## Step 3 - run the HTTP proxy
+
+Unfortunately we still can't listen on a socket in a web browser, so if we want


I don't understand this section. Can't you connect the two browsers via webrtc?
Why do you need a proxy server?

The demo is to use a web browser to load a page from a server running in another web browser, but without the first browser loading any intermediate page or running code.

You can't use the browser address bar to make a WebRTC connection, open a data channel and send an HTTP request so the proxy server is necessary.

MarcoPolo · 2025-05-20T04:25:16Z

packages/http-websocket/README.md

+
+-->
+
+This is an implementation of the [WebSocket API](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket)


Can you explain here and for me, what the use case is for a WebSocket api on top
of libp2p streams?

We discussed this on the go-libp2p call but for anyone reading this in the future the use cases are:

A developer integrating with an existing app that accepts a WebSocket as an argument

Allowing developers to treat libp2p as pure networking and keeping libp2p types/abstractions out of any business logic they may have

Because we can! It's a small jump to transform a HTTP request into a WebSocket and it shows how flexible the underlying transport primitives are

MarcoPolo · 2025-05-20T04:57:48Z

packages/http-server/src/node/server.ts

+    return this._listening
+  }
+
+  set listening (listening: boolean) {


no-op on purpose?

Yes, this is a setter so it makes the listening property read-only.

- Provides adapters for working with W3C Request/Response based servers (e.g. Hono) and with Node.js `http.Server` ones (basically everything else) - Adds support for opening WebSockets over libp2p and integration with existing WebSocket servers - Ensures server code is opt-in to reduce client-only bundle sizes - Adds browser-friendly server capabilities to allow serving webapps from browsers - Allows using libp2p as the routing layer with Node.js native fetch and/or http client via custom agents - Updates all examples to use new API BREAKING CHANGE: the API has change significantly, this module has been renamed `@libp2p/http` since it now does much more than fetch

## @libp2p/http-peer-id-auth-v1.0.0 (2025-05-28) ### ⚠ BREAKING CHANGES * the API has change significantly, this module has been renamed `@libp2p/http` since it now does much more than fetch ### Features * integrate with node HTTP servers, WebSockets and others ([#57](#57)) ([6aa9551](6aa9551))

## @libp2p/http-utils-v1.0.0 (2025-05-28) ### ⚠ BREAKING CHANGES * the API has change significantly, this module has been renamed `@libp2p/http` since it now does much more than fetch ### Features * integrate with node HTTP servers, WebSockets and others ([#57](#57)) ([6aa9551](6aa9551))

## [@libp2p/http-fetch-v3.0.0](https://github.com/libp2p/js-libp2p-http/compare/@libp2p/http-fetch-2.2.2...@libp2p/http-fetch-3.0.0) (2025-05-28) ### ⚠ BREAKING CHANGES * the API has change significantly, this module has been renamed `@libp2p/http` since it now does much more than fetch ### Features * integrate with node HTTP servers, WebSockets and others ([#57](#57)) ([6aa9551](6aa9551))

## @libp2p/http-websocket-v1.0.0 (2025-05-28) ### ⚠ BREAKING CHANGES * the API has change significantly, this module has been renamed `@libp2p/http` since it now does much more than fetch ### Features * integrate with node HTTP servers, WebSockets and others ([#57](#57)) ([6aa9551](6aa9551))

## @libp2p/http-v1.0.0 (2025-05-28) ### ⚠ BREAKING CHANGES * the API has change significantly, this module has been renamed `@libp2p/http` since it now does much more than fetch ### Features * integrate with node HTTP servers, WebSockets and others ([#57](#57)) ([6aa9551](6aa9551))

## @libp2p/http-ping-v1.0.0 (2025-05-28) ### ⚠ BREAKING CHANGES * the API has change significantly, this module has been renamed `@libp2p/http` since it now does much more than fetch ### Features * integrate with node HTTP servers, WebSockets and others ([#57](#57)) ([6aa9551](6aa9551))

## @libp2p/http-server-v1.0.0 (2025-05-28) ### ⚠ BREAKING CHANGES * the API has change significantly, this module has been renamed `@libp2p/http` since it now does much more than fetch ### Features * integrate with node HTTP servers, WebSockets and others ([#57](#57)) ([6aa9551](6aa9551))

achingbrain force-pushed the feat/support-http-over-libp2p branch from 2d876b9 to 82192d7 Compare March 28, 2025 16:46

achingbrain changed the title ~~feat!: support libp2p over http/websockets and http/websockets over libp2p~~ feat!: support http over libp2p as well as libp2p over http Mar 28, 2025

chore: simplify registrar

174979c

achingbrain added 2 commits April 8, 2025 13:53

chore: support cookies

e210d7c

chore: cookies and middleware

718286f

achingbrain added 13 commits April 23, 2025 20:05

chore: auth working

d5a365c

chore: fix firefox webworkers

5ab1c70

chore: rename for monorepo

8322eb9

chore: convert to monorepo

2aabfdb

chore: update deps, tests passing

cf51696

chore: updated examples

81b6e38

chore: docs

8259fe0

chore: update project config

e968865

chore: update readmes

57195a8

chore: linting

9fc12db

chore: more linting

62a605c

chore: add header

b6cc2ec

chore: spelling

9bc6266

achingbrain changed the title ~~feat!: support http over libp2p as well as libp2p over http~~ feat!: refactor to support existing HTTP servers and other use cases Apr 29, 2025

achingbrain changed the title ~~feat!: refactor to support existing HTTP servers and other use cases~~ feat!: integrate with node HTTP servers, WebSockets and others Apr 29, 2025

achingbrain marked this pull request as ready for review April 29, 2025 19:02

achingbrain requested review from 2color, MarcoPolo and SgtPooki April 29, 2025 19:02

MarcoPolo reviewed May 20, 2025

View reviewed changes

achingbrain force-pushed the main branch from 88cdb16 to c980444 Compare May 28, 2025 09:10

chore: add example of agent/dispatcher with fetch/request

8dfddf1

achingbrain force-pushed the feat/support-http-over-libp2p branch from e54326c to 8dfddf1 Compare May 28, 2025 09:11

achingbrain added 4 commits May 28, 2025 11:04

chore: linting, deps and project config

00fe781

chore: changelog

3eb148f

chore: update interop

f7c41d8

chore: update config

0f54f17

achingbrain merged commit 98bea41 into main May 28, 2025
18 checks passed

achingbrain deleted the feat/support-http-over-libp2p branch May 28, 2025 10:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat!: integrate with node HTTP servers, WebSockets and others #57

feat!: integrate with node HTTP servers, WebSockets and others #57

Uh oh!

achingbrain commented Mar 28, 2025 •

edited

Loading

Uh oh!

achingbrain commented Mar 28, 2025 •

edited

Loading

Uh oh!

MarcoPolo commented Apr 7, 2025

Uh oh!

MarcoPolo commented Apr 9, 2025

Uh oh!

MarcoPolo left a comment

Uh oh!

Uh oh!

MarcoPolo May 20, 2025

Uh oh!

achingbrain May 28, 2025

Uh oh!

MarcoPolo May 20, 2025

Uh oh!

achingbrain May 28, 2025

Uh oh!

MarcoPolo May 20, 2025

Uh oh!

achingbrain May 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		## Step 3 - run the HTTP proxy

		Unfortunately we still can't listen on a socket in a web browser, so if we want


		-->

		This is an implementation of the [WebSocket API](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket)

feat!: integrate with node HTTP servers, WebSockets and others #57

feat!: integrate with node HTTP servers, WebSockets and others #57

Uh oh!

Conversation

achingbrain commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO:

Change checklist

Uh oh!

achingbrain commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MarcoPolo commented Apr 7, 2025

Uh oh!

MarcoPolo commented Apr 9, 2025

Uh oh!

MarcoPolo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MarcoPolo May 20, 2025

Choose a reason for hiding this comment

Uh oh!

achingbrain May 28, 2025

Choose a reason for hiding this comment

Uh oh!

MarcoPolo May 20, 2025

Choose a reason for hiding this comment

Uh oh!

achingbrain May 28, 2025

Choose a reason for hiding this comment

Uh oh!

MarcoPolo May 20, 2025

Choose a reason for hiding this comment

Uh oh!

achingbrain May 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

achingbrain commented Mar 28, 2025 •

edited

Loading

achingbrain commented Mar 28, 2025 •

edited

Loading