Skip to content

Latest commit

 

History

History
483 lines (360 loc) · 18.2 KB

http-routing-v1.md

File metadata and controls

483 lines (360 loc) · 18.2 KB
Raw
title description date maturity editors xref order tags
Delegated Routing V1 HTTP API
Delegated routing is a mechanism for IPFS implementations to use for offloading content routing, peer routing and naming to another process/server. This specification describes an HTTP API for delegated routing of content, peers, and IPNS.
2023-08-31
reliable
name github affiliation
Gus Eggert
guseggert
name url
Protocol Labs
name github affiliation
Masih H. Derkani
masih
name url
Protocol Labs
name url github affiliation
Henrique Dias
hacdias
name url
Protocol Labs
name github url affiliation
Marcin Rataj
lidel
name url
Protocol Labs
ipns-record
0
routing

Delegated routing is a mechanism for IPFS implementations to use for offloading content routing, peer routing, and naming to another process/server. This specification describes a vendor-agnostic HTTP API for delegated content routing.

API Specification

The Routing HTTP API uses the application/json content type by default. For :ref[IPNS Names], the verifiable application/vnd.ipfs.ipns-record content type is used.

As such, human-readable encodings of types are preferred. This specification may be updated in the future with a compact application/cbor encoding, in which case compact encodings of the various types would be used.

Common Data Types

Until required for business logic, servers should treat these types as opaque strings, and should preserve unknown JSON fields.

Versioning

This API uses a standard version prefix in the path, such as /v1/.... If a backwards-incompatible change must be made, then the version number should be increased.

Content Routing API

GET /routing/v1/providers/{cid}

GET Path Parameters

  • cid is the CID to fetch provider records for.

GET Response Status Codes

  • 200 (OK): the response body contains 0 or more records.
  • 404 (Not Found): must be returned if no matching records are found.
  • 422 (Unprocessable Entity): request does not conform to schema or semantic constraints.

GET Response Body

{
  "Providers": [
    {
      "Schema": "<schema>",
      "ID": "bafz...",
      "Addrs": ["/ip4/..."],
      ...
    },
    ...
  ]
}

The application/json responses SHOULD be limited to 100 providers.

The client SHOULD be able to make a request with Accept: application/x-ndjson and get a stream with more results.

Each object in the Providers list is a record conforming to a schema, usually the Peer Schema.

PUT /routing/v1/providers

PUT Response codes

  • 200 (OK): the server processed the full list of provider records (possibly unsuccessfully, depending on the semantics of the particular records)
  • 400 (Bad Request): the server deems the request to be invalid and cannot process it
  • 422 (Unprocessable Entity): request does not conform to schema or semantic constraints
  • 501 (Not Implemented): the server does not support providing records

PUT Request Body

{
  "Providers": [
    {
      "Schema": "announcement",
      ...
    }
  ]
}

Each object in the Providers list is a write provider record entry.

Server SHOULD accept representing writes is Announcement Schema.

:::warn

TODO: is below a sensible limit?

There SHOULD be no more than 100 Providers per request.

:::

PUT Response Body

{
    "ProvideResults": [
        { ... }
    ]
}
  • ProvideResults is a list of results in the same order as the Providers in the request, and the schema of each object is determined by the Protocol of the corresponding write object
    • This may contain output information such as TTLs, errors, etc.
    • It is undefined whether the server will allow partial results
  • The work for processing each provider record should be idempotent so that it can be retried without excessive cost in the case of full or partial failure of the request

Peer Routing API

GET /routing/v1/peers/{peer-id}

Path Parameters

  • peer-id is the Peer ID to fetch peer records for, represented as a CIDv1 encoded with libp2p-key codec.

Response Status Codes

  • 200 (OK): the response body contains the peer record.
  • 404 (Not Found): must be returned if no matching records are found.
  • 422 (Unprocessable Entity): request does not conform to schema or semantic constraints.

Response Body

{
  "Peers": [
    {
      "Schema": "peer",
      "ID": "bafz...",
      "Addrs": ["/ip4/..."],
      "Protocols": ["<protocol-a>", "<protocol-b>", ...],
      ...
    },
    ...
  ]
}

The application/json responses SHOULD be limited to 100 peers.

The client SHOULD be able to make a request with Accept: application/x-ndjson and get a stream with more results.

Each object in the Peers list is a record conforming to the Peer Schema.

PUT /routing/v1/peers

PUT Response codes

  • 200 (OK): the server processed the full list of provider records (possibly unsuccessfully, depending on the semantics of the particular records)
  • 400 (Bad Request): the server deems the request to be invalid and cannot process it
  • 422 (Unprocessable Entity): request does not conform to schema or semantic constraints
  • 501 (Not Implemented): the server does not support providing records

PUT Request Body

{
  "Providers": [
    {
      "Schema": "announcement",
      ...
    }
  ]
}

Each object in the Providers list is a write provider record entry.

Server SHOULD accept writes represented with Announcement Schema objects with CID list.

IPNS API

GET /routing/v1/ipns/{name}

Path Parameters

  • name is the :ref[IPNS Name] to resolve, encoded as CIDv1.

Response Status Codes

  • 200 (OK): the response body contains the :ref[IPNS Record] for the given :ref[IPNS Name].
  • 404 (Not Found): must be returned if no matching records are found.
  • 406 (Not Acceptable): requested content type is missing or not supported. Error message returned in body should inform the user to retry with Accept: application/vnd.ipfs.ipns-record.

Response Headers

  • Etag: a globally unique opaque string used for HTTP caching. MUST be derived from the protobuf record returned in the body.
  • Cache-Control: max-age={TTL}: cache TTL returned with :ref[IPNS Record] that has IpnsEntry.data[TTL] > 0. When present, SHOULD match the TTL value from the record. When record was not found (HTTP 404) or has no TTL (value is 0), implementation SHOULD default to max-age=60.

Response Body

The response body contains a :ref[IPNS Record] serialized using the verifiable application/vnd.ipfs.ipns-record protobuf format.

PUT /routing/v1/ipns/{name}

Path Parameters

  • name is the :ref[IPNS Name] to publish, encoded as CIDv1.

Request Body

The content body must be a application/vnd.ipfs.ipns-record serialized :ref[IPNS Record], with a valid signature matching the name path parameter.

Response Status Codes

  • 200 (OK): the provided :ref[IPNS Record] was published.
  • 400 (Bad Request): the provided :ref[IPNS Record] or :ref[IPNS Name] are not valid.
  • 406 (Not Acceptable): submitted content type is not supported. Error message returned in body should inform the user to retry with Content-Type: application/vnd.ipfs.ipns-record.

Pagination

This API does not support pagination, but optional pagination can be added in a backwards-compatible spec update.

Streaming

JSON-based endpoints support streaming requests made with Accept: application/x-ndjson HTTP Header.

Steaming responses are formatted as Newline Delimited JSON (ndjson), with one result per line:

{"Schema": "<schema>", ...}
{"Schema": "<schema>", ...}
{"Schema": "<schema>", ...}
...

:::note

Streaming is opt-in and backwards-compatible with clients and servers that do not support streaming:

  • Requests without the Accept: application/x-ndjson header MUST default to regular, non-streaming, JSON responses.
  • Legacy server MAY respond with non-streaming application/json response even if the client requested streaming. It is up to the client to inspect the Content-Type header before parsing the response.
  • The server MUST NOT respond with streaming response if the client did not explicitly request so.

:::

Error Codes

  • 400 (Bad Request): must be returned if an unknown path is requested.
  • 429 (Too Many Requests): may be returned along with optional Retry-After header to indicate to the caller that it is issuing requests too quickly.
  • 501 (Not Implemented): must be returned if a method/path is not supported.

CORS and Web Browsers

Browser interoperability requires implementations to support CORS.

JavaScript client running on a third-party Origin must be able to send HTTP request to the endpoints defined in this specification, and read the received values. This means HTTP server implementing this API must (1) support CORS preflight requests sent as HTTP OPTIONS, and (2) always respond with headers that remove CORS limits, allowing every site to query the API for results:

Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, PUT, OPTIONS

Known Schemas

This section contains a non-exhaustive list of known schemas that MAY be supported by clients and servers.

Peer Schema

The peer schema represents an arbitrary peer.

{
  "Schema": "peer",
  "ID": "bafz...",
  "Addrs": ["/ip4/..."],
  "Protocols": ["transport-bitswap", ...]
  ...
}
  • ID: the Peer ID as Multihash in Base58btc or CIDv1 with libp2p-key codec.
  • Addrs: an optional list of known multiaddrs for this peer.
    • If missing or empty, it means the router server is missing that information, and the client should use ID to lookup updated peer information.
  • Protocols: an optional list of protocols known to be supported by this peer.
    • If missing or empty, it means the router server is missing that information, and the client should use ID and Addrs to lookup connect to the peer and use the libp2p identify protocol to learn about supported ones.

:::note

To allow for protocol-specific fields and future-proofing, the parser MUST allow for unknown fields, and the clients MUST ignore unknown ones.

Below is an example on how one could include protocol-a and protocol-b protocols that includes an additional fields protocol-a and protocol-b.

If the client knows the protocol, they are free to use the extra binary (base64) or JSON information contained in the additional field. If that is not the case, the field MUST be ignored.

{
  "Schema": "peer",
  "ID": "bafz...",
  "Addrs": ["/ip4/..."],
  "Protocols": ["transport-bitswap", "protocol-a", "protocol-b", ...],
  "protocol-a": "[base64-blob]",
  "protocol-b": { "foo": "bar" }
}

:::

Announcement Schema

The announcement schema can be used in PUT operations to announce content providers or peer routing information.

  {
    "Schema": "announcement",
    "Payload": {
      "CID": ["cid1", "cid2", ...],
      "Scope": "block",
      "Timestamp": "YYYY-MM-DDT23:59:59Z",
      "TTL": 0,
      "ID": "12D3K...",
      "Addrs": ["/ip4/...", ...],
      "Protocols": ["foo", ...],
      "Metadata": "mbase64-blob",
    },
    "Signature": "mbase64-signature"
  }
  • Schema: tells the server to interpret the JSON object as announce provider
  • Payload: is a DAG-JSON-compatible object with a subset of the below fields
    • CID is a list of the CIDs being provided.
      • Skipped when used for PUT /routing/v1/peers
      • Scope is an optional hint that provides semantic meaning about announced identifies:
        • block announces only the individual blocks (implicit default if Scope is missing).
        • entity announces enumerable entities behind the CIDs (e.g.: all blocks for UnixFS file or a minimum set of blocks to enumerate HAMT-sharded UnixFS directory).
        • recursive announces entire DAGs behind the CIDs (e.g.: entire DAG-CBOR DAG, or everything in UnixFS directory, including all files in all subdirectories).
    • Timetamp is the current time, formatted as an ASCII string that follows notation from rfc3339.
    • TTL is caching and expiration hint informing the server how long to keep the record available, specified in milliseconds.
      • If this value is unknown, the caller may skip this field, or use a value of 0. The server's default will be used.
    • ID is Peer ID of the node that provides the content and also indicates the libp2p-key that SHOULD be used for verifying Signature field.
    • Addrs is an a list of string-encoded multiaddrs without /p2p/peerID suffix.
    • Protocols is a list of protocols supported by ID and/or Addrs, if known upfront.
    • Metadata is a string with multibase-encoded binary metadata that should be passed as-is
  • Signature is a string with multibase-encoded binary signature that provides integrity and authenticity of the Payload field.
    • Signature is created by following below steps:
      1. Convert Payload to deterministic, ordered DAG-JSON map notation
      2. Prefix the DAG-JSON bytes with ASCII string PUT /routing/v1 announcement:
      3. Sign the bytes with the private key of the Peer ID specified in the Payload.ID.
        • Signing details for specific key types should follow libp2p/peerid specs, unless stated otherwise.
    • Client SHOULD sign every announcement.
    • Servers SHOULD verify signature before accepting a record, unless running in a trusted environment.
      • ED25519 and other small public keys MUST be inlined inside of the ID field with the identity multihash type.
      • Key types that exceed 42 bytes (e.g. RSA) SHOULD NOT be inlined, the ID field should only include the multihash of the key. The key itself SHOULD be obtained out-of-band (e.g. by fetching the block via IPFS) and cached. If support for big keys is needed in the future, this spec can be updated to allow the client to provide the key and key type out-of-band by adding optional PublicKey fields, and if the Peer ID is a CID, then the server can verify the public key's authenticity against the CID, and then proceed with the rest of the verification scheme.
    • A 400 Bad Request response code SHOULD be returned if the Signature check fails.

:::warn

TODO: what should be the limits? Max number of CIDs per announcement ?

:::

Use in PUT responses

Server MAY return additional TTL information if the TTL is not provided in the request, or if server policy is to provide TTL different than the requested one.

{
  "Schema": "announcement",
  "Payload": {
    "TTL": 17280000
  }
}
  • TTL in response is the time at which the server expects itself to drop the record
    • If less than the TTL in the request, then the client SHOULD repeat announcement earlier, before the announcement TTL expires and is forgotten by the routing system
    • If greater than the TTL in the request, then the server client SHOULD save resources and not repeat announcement until the announcement TTL expires and is forgotten by the routing system
    • If 0, the server makes no claims about the lifetime of the record

Legacy Schemas

Legacy schemas include ID and optional Addrs list just like the peer schema does.

These schemas are deprecated and SHOULD be replaced with peer over time, but MAY be returned by some legacy endpoints. In such case, a client MAY parse them the same way as the peer schema.

Bitswap Schema

A legacy schema used by some routers to indicate a peer supports retrieval over the /ipfs/bitswap[/*] libp2p protocol.

{
  "Protocol": "transport-bitswap",
  "Schema": "bitswap",
  "ID": "bafz...",
  "Addrs": ["/ip4/..."]
}

Graphsync Schema

A legacy schema used by some routers to indicate a peer supports retrieval over the graphsync libp2p protocol.

{
  "Protocol": "transport-graphsync-filecoinv1",
  "Schema": "graphsync-filecoinv1",
  "ID": "bafz...",
  "Addrs": ["/ip4/..."],
  "PieceCID": "<cid>",
  "VerifiedDeal": true,
  "FastRetrieval": true
}

Filecoin Graphsync Write Provider Records

There is currently no specified schema.