Skip to content

IPNS Link V2.1

Jump to bottom
Winterhuman edited this page Jun 9, 2022 · 27 revisions

IPNS-Link V2.1

Exposers

  • Exposers now consist of a single, public IPFS node.
  • Bandwidth limiting is achieved through the ResourceManager.
  • IPNS-Link Gateways are allowed to connect to Origins only if they are present in that Origin's trusted Gateway list.
  • OriginIDs now contain inline content and replace Manifest files & IPNS entirely.

Single-node Exposers

The IPFS node will have full DHT access (DHTClient), this will allow it to freely distribute Provider records to the public DHT, allow for more reliable CircuitV2 usage as AutoRelay can now autodiscover relays, and it reduces our reliance on bootstrap nodes by restarting the node much less frequently than before. The ResourceManager will handle bandwidth limiting from now on.

Trusted Gateway List

Each Origin will have a trusted: option in their config, this option allows specifying not only what PeerIDs to allow, but also, where to find the PeerID from. This allows dynamically updating the PeerID even when specifying Private Gateways. For example:

trusted: {
	https://domain.tld/peerid
	ipfs://"PeerID"
	ipfs://"CIDofPeerIDfile"
	ipns://"IPNSaddressOfEmptyIPNSrecord"
}

Note: Each of the above URLs can point to the same IPNS-Link Gateway for redundancy.

OriginIDs as Inline CIDs

Instead of using Manifest files and IPNS records, OriginIDs will contain all the necessary information needed for IPNS-Link Gateways to connect to the right Exposer verifiably, here's how these new OriginIDs will work:

  1. The user executes ipns-link add service.
  2. The user modifies service.json to state origin: "127.0.0.1:80", host: "127.0.0.1:80", and trusted: "ipnslink.com".
  3. The user executes ipns-link expose service.
  4. The binary concatenation of ('PeerID public key' + 'host') is hashed with Blake2s-128 and given multihash ID 0xb25010, we'll refer to this as the Multihash Proof.
  5. The binary concatenation of ('Multihash Proof' + 'padded host'), where the value of host: is padded to 13 bytes with null data, is then constructed and inlined into a CID of ID bafkqai (base32, cidv1, raw, identity:256). This is an OriginID which we'll refer to as bafkqaiproof.
  6. The Exposer opens multistream protocol /origin/bafkqaiproof which leads to the origin: address.
  7. An end-user goes to bafkqaiproof.ipfs.ipnslink.com.
  8. ipnslink.com will search IPFS for proof and find multiple Provider records, some of which may contain PeerIDs not belonging to the Exposer.
  9. For each unique PeerID, the function Blake2s-128('PeerID public key' + 'host') is ran to determine which PeerID belongs to the Exposer. If the multihash proof calculated is identical to the multihash proof in the OriginID, then the PeerID belongs to the Exposer.
  10. The Gateway creates a Libp2p stream mount from base64(bafkqaiproof).socket to /origin/bafkqaiproof at 12D3KooWexposer, it then tells it's reverse proxy that bafkqaiproof.ipfs.ipnslink.com should lead to base64(bafkqaiproof).socket and to modify the Host: header to state Host: 127.0.0.1:80 (the value of host:).
  11. HTTP requests of Host: 127.0.0.1:80 will exit /origin/bafkqaiproof and towards 127.0.0.1:80.
  12. Origin 127.0.0.1:80 can now be served at bafkqaiproof.ipfs.ipnslink.com.
OriginID: bafkqaiproof

"bafkqai": "base32, cidv1, raw, identity, 256 bits"
"proof": {
	"MultihashProofID": "0xb25010" (3 bytes)
	"MultihashProofDigest": "Blake2s-128('PeerID public key' + 'host')" (16 bytes)
	"Host": "127.0.0.1:80" (13 bytes, with null bytes for padding)
}

Warning: ipns-link add and ipns-link expose must ensure that the value of host: does not exceed 13 bytes (origin: is unrestricted), the port number can be omitted if it is implied (e.g. Scheme: http implies :80; Scheme: https implies :443).

Note: OriginIDs won't work for DNSLink. Instead, the user can run ipns-link expose service --dnslink which will output an inline CID, which contains a <meta> tag like Manifests used to, for use as the DNSLink TXT value.

Future changes: When signed Provider records are supported by go-libp2p and go-ipfs nodes by default, host will be protected by the signature and we can switch to Blake2s-128('PeerID public key') for the multihash proof.

New Commands

To demonstrate what the new commands look like, here's an example scenario (tab spaces denote output):

ipns-link version
	go-exposer v0.1.0
ipns-link init
	Initiliased repo at $HOME/.config/ipns-link
ipns-link daemon
	Starting IPFS node...
	"IPFS daemon output here"...
	...
	IPNS-Link daemon is ready.
ipns-link log
	"Display content of '$HOME/.config/ipns-link/log' here"
ipns-link add service
	Created 'service.json'.
ipns-link mod service --origin "127.0.0.1:80" --host "127.0.0.1:80"
	Changed origin of 'service' to '127.0.0.1:80'.
	Changed host of 'service' to '127.0.0.1:80'.
ipns-link config service
	Default editor not configured, please state editor in global.json or use $EDITOR.
$EDITOR=nano ipns-link config
	--- Nano $HOME/.config/ipns-link/global.json ---

		editor: # Change to "nano"
		ipfs_repo: "$HOME/.config/ipns-link/.ipfs"
		default_origin: "localhost"
		default_trusted: {
			https://ipnslink.com/peerid
			ipfs://PeerID
			ipfs://CIDofPeerIDfile
			ipns://IPNSaddressOfEmptyIPNSrecord
		}

	--- End of file ---
ipns-link config service
	--- Nano $HOME/.config/ipns-link/origin/service.json ---

		origin: "127.0.0.1:80"
		host: "127.0.0.1:80"
		trusted: {
			https://ipnslink.com/peerid
			ipfs://PeerID
			ipfs://CIDofPeerIDfile
			ipns://IPNSaddressOfEmptyIPNSrecord
		}
		cache: {}

	---	End of file ---
ipns-link expose service
	Exposing 'service'...
	'/origin/bafkqai...' is now open. OriginID is bafkqai...
ipns-link expose service --dnslink
	'/origin/bafkqai...' is already open.
	DNSLink CID is 'bafkqae...'
ipns-link list
	NAME		ORIGIN		HOST		STATUS		OriginID
	service		127.0.0.1:80	127.0.0.1:80	OPEN		bafkqai...
ipns-link hide service
	Hiding 'service'...
	'/origin/bafkqai...' is now closed.
ipns-link del service
	Are you sure you want to delete 'service'?
	"Yes"
	Deleted 'service'. Old config located at 'ipns-link/origin/service.json.old'
ipns-link quit
	Shutting down IPNS-Link daemon...

TLS Passthrough

An end-to-end TLS connection between an Origin and a browser, such that a Gateway cannot inspect the traffic flowing between them, is as simple as configuring host: "". This instructs the Gateway to not modify the browser's HTTP packets, nor to terminate the TLS connection at itself, and instead send the browser's HTTP packets as they are to base64(bafkqaiproof).socket.

From the Origin's side, the incoming Host: bafkqaiproof.ipfs.gateway.tld HTTP packets can be handled by a reverse proxy at origin: setup by the Exposer's operator, the reverse proxy can then use ACME challenges to fetch a TLS certificate to facilitate TLS passthrough.

Additionally, it is recommended that the Exposer operator configures an MITM detection system so that Gateways terminating the TLS to anywhere but the Origin will be flagged as malicious, this feature may be introduced into Exposers themselves in the future to allow distrusting the Gateway on detection.

Warning: If two Origins share a reverse proxy, a Gateway trusted by Origin 1 can send HTTP packets of Host: bafkqaiOrigin2.ipfs.TrustedGatewayOfOrigin1.tld through /origin/origin1 to access Origin 2 even if that Gateway isn't trusted by Origin 2. To prevent this from happening, make sure all Origins using the same reverse proxy share the same trusted Gateway list between each other, any Origin with a different list must be given it's own reverse proxy.

Gateways

  • IPNS-Link Gateways must provide a file at https://gateway.tld/peerid containing the PeerID of the internal IPFS node, this PeerID must be an inline ed25519 PeerID so that the public key doesn't need to be stated separately.
  • IPNS-Link Gateways will not cache any content by default.
  • oid.json files are no longer created.
  • All attempts for *.ipns.gateway.tld will be rejected (unless caching is enabled).
  • Only CIDs of ID bafkqai will be considered for *.ipfs.gateway.tld (unless caching is enabled).

Caching

The reason why caching will be disabled by default, and likely be disabled on our official IPNS-Link Gateways, is because of concerns of legal action or threats from DNS registrars. Cached content might include illegal or malicious content which Gateways would publicly serve, this can lead DNS registrars or hosting providers to swiftly prevent access to the Gateway if they received a notice from an individual or company reporting this content. For this reason, caching will be disabled by default by IPNS-Link Gateway software.

Since there is no Manifest file to define what content is cached by the Exposer, IPNS-Link Gateways will instead ask all connected Exposers for any content they are providing, this content will then be stored in the internal IPFS node. This content will not be pinned and will instead be cleared by the garbage collection algorithm used by the internal IPFS node.

Note: Because caching is safe for Private Gateways, and because IPNS-Link Gateway operators may choose to accept the risks, caching will not be removed entirely.

Gateway Logic

gateway.tld/peerid

  • Serve the content of the file.

Caching disabled

*.ipns.gateway.tld or gateway.tld/ipns/*

  • If the IPNS address is not a valid multiformat (e.g. A longer multihash than specified by the ID), return error code 400.
  • If the IPNS address is a valid multiformat, 307 redirect to a random IPFS Gateway as is (e.g. gateway.tld/ipns/key redirects to ipfs.io/ipns/key, not key.ipns.ipfs.io).

gateway.tld/ipfs/*

  • If the CID is not a valid multiformat, return error code 400.
  • If the CID is a valid multiformat, but does not start with ID bafkqai, 307 redirect to a random IPFS Gateway as is.
  • If the CID being requested does start with ID bafkqai, do a 301 redirect to bafkqaiproof.ipfs.gateway.tld.

*.ipfs.gateway.tld

  • If the CID is not a valid multiformat, return error code 400.
  • If the CID is a valid multiformat, but does not start with ID bafkqai, 307 redirect to a random IPFS Gateway as is.
  • If the CID does start with ID bafkqai, continue to next step.
  • If the host: value at the end of the CID is not a valid Host: header value, return error code 502.
  • If the host: value at the end of the CID is a valid Host: header value, or host: has no value, continue to next step.
  • If the multihash proof in the CID does not match the multihash proofs generated with all PeerIDs found after 30 seconds, return error code 504.
  • If the multihash proof in the CID does match a multihash proof generated with one of the PeerIDs found, continue to next step.
  • If the PeerID that generated the multihash proof can not be connected to, likely because the Exposer is not publicly accessible, return error code 503.
  • If the PeerID that generated the multihash proof can be connected to, continue to next step.
  • If the host: value at the end of the CID was nothing, do not terminate the TLS connection at yourself and instead pass it through to base64(bafkqaiproof).socket, then continue to next step.
  • If the host: value at the end of the CID was not nothing, do terminate the TLS connection at yourself and modify all HTTP packets sent to base64(bafkqaiproof).socket to state Host: 'host: value'. Then, continue to final step.
  • If the /origin/bafkqaiproof multistream protocol does not respond to any requests after 10 seconds, return error code 502.
  • If the /origin/bafkqaiproof multistream protocol does respond to requests, meaning a connection was successfully made to the Origin, serve the responses at bafkqaiproof.ipfs.gateway.tld.

Caching enabled

  • If *.{ipfs,ipns}.gateway.tld or gateway.tld/{ipfs,ipns}/* are requested, and the requests are valid multiformats, but these request do not pass the ID check, continue to next step.
  • If the CID or IPNS address requested does exist in the internal IPFS node's cache, return the content.
  • If the CID or IPNS address requested does not exist in the internal IPFS node's cache, 307 redirect to a random IPFS Gateway as is.

Summary

Single-node Exposers, trusted Gateway authentication, no IPNS or Manifests, no Gateway caching (by default).

Advantages

  • Not using IPNS is faster for both distributing and fetching OriginIDs.
  • Only the self key and the Origin configs need to be backed up to fully restore an Exposer.
  • Exposers can expose reverse proxies since origin: and host: are two separate values.
  • Single-node Exposers are simpler to manage and are less resource intensive than two IPFS nodes, the ResourceManager also offers much finer resource control.
  • Exposers can now better use CircuitV2 relays, are less reliant on bootstrap nodes, and are more flexible for future features that require DHT access.
  • Gateways don't need to write oid.json files to a fileserver, the internal IPFS node's DHT contains all the necessary information.

Disadvantages

  • The Origin's host: value is limited to 13 bytes (origin: value is unrestricted since it's not in the OriginID).
  • Changing the Origin's host: value changes it's OriginID (recommend users to use domains to reduce changing host: frequently).
  • A new solution is needed to prevent DNSLinks from non-owners (https://github.com/ipns-link/specs/discussions/5).
Clone this wiki locally