IPIP-417: Delegated Peer Routing HTTP API #417

hacdias · 2023-05-29T09:13:55Z

This IPIP introduces the /routing/v1/peers endpoint to the Routing HTTP V1 API for peer routing support.

lidel

@hacdias I had to rebase this one to be based on the latest versions of the other two, apologies for the noise.

src/routing/http-routing-v1.md

guillaumemichel · 2023-06-07T13:15:50Z

IMO it would be great to have delegated peer routing use signed peer records. It seems essential because without signatures you have to trust a middleman.

A simple "Signature" field could be added to the proposed schema. The signature can be the array of bytes resulting of the signature using the private key associated with the peer ID (and corresponding algorithm) of (all) the other (non-empty) fields of the schema.

hacdias · 2023-06-14T11:07:08Z

@guillaumemichel I like the idea, but that'd require deeper changes to the way we implement things. Currently, the idea was to mostly "relay" the results of dht findpeer in Kubo for example. That doesn't give us signed records. Besides, this specification also indicates that the addresses and protocols may be incorrect.

All in all, we have to find the balance. For IPNS (#379), it is easy to send signed records, because that's literally what is passed around. Here it seems it'd be more complicated.

src/routing/http-routing-v1.md

hacdias · 2023-07-26T08:59:34Z

src/routing/http-routing-v1.md

+#### Response Body
+
+A [`peer` schema record](#peer).


Open Question: should the response body for /routing/v1/peers be a single entity? What we want to obtain is the addresses of the peer. But addresses can end up being discovered on the fly. Supporting streaming is reasonably easy and could be part of the spec.

Should be multiple, and work the same as /providers – we may have more than one result for the same peerid coming from different backend services (e.g. if /routing/v1 queries both DHT and IPNI)

cid.contact offers both: /providers returns list of all providers and /providers/<peer-id> returns a specific one.

I updated the Peers API to indicate it can return more than one. It will conform with the streaming API we already have too. Whether we add /peers and /providers too is TBD. Reasonably easy to add. Wdyt @masih @lidel?

src/routing/http-routing-v1.md

lidel · 2023-08-25T15:51:16Z

FYSA we've got implementation in

client and server in boxo/routing/http library (boxo/pull/422)
- which will ship in Kubo 0.23
  - server opt-in via Gateway.ExposeRoutingAPI=true config: ipfs/kubo/pull/9877
  - client mode can be enabled via config (example in test/cli/delegated_routing_v1_http_proxy_test.go#L28-L44)

lidel

Applied final editorials, and included legacy schemas with guidance to parse them the same way as peer one (to leverage presence of ID and Addrs fields). This is what boxo does for Schema: bitswap entries.

This is ready for final reviews and ratification, will discuss during IPFS Implementers call today

BigLep · 2023-09-20T20:06:20Z

@lidel : are we good to merge now?

Co-authored-by: Marcin Rataj <lidel@lidel.org>

Co-authored-by: Masih H. Derkani <m@derkani.org>

we will have legacy schemas in the wild for a while, added a note that clients may parse them the same way as peer one (both have ID and Addrs)

lidel

This has been waiting for ratification feedback for a few weeks now, and no concerns were raised.
Thank you @hacdias for landing this.

For anyone reading this in the future:

Self-hosting /routing/v1 endpoint for delegated routing needs will ship in Kubo 0.23

hacdias changed the base branch from main to issues/344 May 29, 2023 09:15

hacdias changed the title ~~ipip: Streaming Routing V1 HTTP API~~ IPIP-0417 May 29, 2023

hacdias changed the title ~~IPIP-0417~~ IPIP-0417: Introduce Peer Schema in Routing API May 29, 2023

hacdias self-assigned this May 29, 2023

hacdias force-pushed the issues/344 branch from eef0074 to a028443 Compare May 30, 2023 14:07

hacdias changed the title ~~IPIP-0417: Introduce Peer Schema in Routing API~~ IPIP-417: Delegated Peer Routing HTTP API May 30, 2023

hacdias force-pushed the issues/344 branch from a028443 to 01a3f09 Compare May 30, 2023 14:26

hacdias force-pushed the ipip/schema-peer branch from 490ccdb to 944500f Compare May 30, 2023 14:32

hacdias force-pushed the issues/344 branch from 01a3f09 to 36571b0 Compare May 30, 2023 14:42

hacdias force-pushed the ipip/schema-peer branch from 944500f to b3828f6 Compare May 30, 2023 14:54

hacdias requested a review from lidel May 30, 2023 14:57

hacdias force-pushed the ipip/schema-peer branch from b3828f6 to c1eaebf Compare May 30, 2023 15:00

lidel force-pushed the issues/344 branch from 61186f0 to e55e319 Compare May 31, 2023 19:20

lidel force-pushed the ipip/schema-peer branch from c1eaebf to 4cf3e5f Compare May 31, 2023 19:37

lidel reviewed May 31, 2023

View reviewed changes

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

lidel force-pushed the ipip/schema-peer branch 2 times, most recently from b12388d to 9790174 Compare May 31, 2023 19:50

lidel mentioned this pull request May 31, 2023

IPIP-378: Delegated Routing HTTP POST API #378

Open

lidel reviewed Jun 7, 2023

View reviewed changes

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

lidel mentioned this pull request Feb 13, 2023

Support DHT-based Peer routing over HTTP ipfs/someguy#14

Closed

3 tasks

Base automatically changed from issues/344 to main June 15, 2023 20:27

lidel self-assigned this Jul 6, 2023

lidel mentioned this pull request Jul 25, 2023

feat(gateway): expose /routing/v1 server (opt-in) ipfs/kubo#9877

Merged

13 tasks

hacdias force-pushed the ipip/schema-peer branch 3 times, most recently from 25d51dc to e66a70a Compare July 26, 2023 08:54

hacdias commented Jul 26, 2023

View reviewed changes

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

hacdias commented Jul 26, 2023

View reviewed changes

hacdias marked this pull request as ready for review July 26, 2023 09:00

hacdias requested a review from a team as a code owner July 26, 2023 09:00

hacdias force-pushed the ipip/schema-peer branch from e66a70a to 4ee487a Compare July 31, 2023 09:17

hacdias requested review from lidel and masih August 1, 2023 11:13

masih approved these changes Aug 1, 2023

View reviewed changes

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

hacdias mentioned this pull request Aug 2, 2023

feat(routing/http)!: delegated peer routing server and client, IPIP 417 ipfs/boxo#422

Merged

5 tasks

BigLep mentioned this pull request Aug 3, 2023

Release 0.22 ipfs/kubo#9911

Closed

aschmahmann mentioned this pull request Aug 15, 2023

Trying to Dial Kubo Node with Web Transport libp2p/js-libp2p#1951

Closed

lidel mentioned this pull request Aug 25, 2023

Switch unknown providers at /routing/v1/providers to peer schema ipni/indexstar#147

Closed

lidel approved these changes Aug 31, 2023

View reviewed changes

BigLep mentioned this pull request Sep 6, 2023

Support IPIP-417 (Delegated Peer lookups) libp2p/js-delegated-routing-v1-http-api-content-routing#25

Closed

hacdias and others added 10 commits September 25, 2023 10:59

docs: start IPIP 0417

1467e48

Co-authored-by: Marcin Rataj <lidel@lidel.org>

add paragraph about additional fields

e1b335f

remove known transfer protocols section

b77728b

cleanup provider records story

b1ed79e

add notes to the compatibility section

828378d

editorial touches

2dbdb54

finish sections in IPIP

4adf8b7

editorial fixes

86ecfa0

Co-authored-by: Masih H. Derkani <m@derkani.org>

ipip-0417: final editorials, list legacy schemas

f3fe9d5

we will have legacy schemas in the wild for a while, added a note that clients may parse them the same way as peer one (both have ID and Addrs)

chore: bump date

95999ef

hacdias force-pushed the ipip/schema-peer branch from 34dbaba to 95999ef Compare September 25, 2023 08:59

lidel approved these changes Sep 25, 2023

View reviewed changes

lidel merged commit 970b81d into main Sep 25, 2023
4 checks passed

lidel deleted the ipip/schema-peer branch September 25, 2023 17:18

This was referenced Oct 3, 2023

Reframe peer records #279

Closed

routing api: clarify string in the protocol field #403

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IPIP-417: Delegated Peer Routing HTTP API #417

IPIP-417: Delegated Peer Routing HTTP API #417

hacdias commented May 29, 2023 •

edited

lidel left a comment •

edited

guillaumemichel commented Jun 7, 2023

hacdias commented Jun 14, 2023

hacdias Jul 26, 2023

lidel Jul 28, 2023 •

edited

masih Jul 31, 2023

hacdias Aug 1, 2023 •

edited

lidel commented Aug 25, 2023 •

edited

lidel left a comment •

edited

BigLep commented Sep 20, 2023

lidel left a comment

IPIP-417: Delegated Peer Routing HTTP API #417

IPIP-417: Delegated Peer Routing HTTP API #417

Conversation

hacdias commented May 29, 2023 • edited

lidel left a comment • edited

Choose a reason for hiding this comment

guillaumemichel commented Jun 7, 2023

hacdias commented Jun 14, 2023

hacdias Jul 26, 2023

Choose a reason for hiding this comment

lidel Jul 28, 2023 • edited

Choose a reason for hiding this comment

masih Jul 31, 2023

Choose a reason for hiding this comment

hacdias Aug 1, 2023 • edited

Choose a reason for hiding this comment

lidel commented Aug 25, 2023 • edited

lidel left a comment • edited

Choose a reason for hiding this comment

BigLep commented Sep 20, 2023

lidel left a comment

Choose a reason for hiding this comment

hacdias commented May 29, 2023 •

edited

lidel left a comment •

edited

lidel Jul 28, 2023 •

edited

hacdias Aug 1, 2023 •

edited

lidel commented Aug 25, 2023 •

edited

lidel left a comment •

edited