IPIP-504: provider query parameter as hint for HTTP Gateways

[vasco-santos]

Based on initial exploration https://github.com/vasco-santos/provider-hinted-uri/blob/main/EXPLORATION.md

[SgtPooki]

first read through.. makes sense and seems like it could really benefit the ecosystem. One problem I can imagine is that people start encoding dynamic IP address providers in urls and they quickly become useless, so we should probably call that out.

[SgtPooki]

I feel like the below details should be linking to another spec i'm sure we have written somewhere

[vasco-santos]

Mind pointing out where it lives? I do not know about this and could not find nothing when working on this

[SgtPooki]

why repeating parameter instead of comma-delimited?

[lidel]

Probably to make spec simpler to implement. Some reasons to repeat instead of comma-separating:

• Future-proof in case future values may include ,?
• URL-escaping: in many standard libraries. In Golang, items=apple,banana,orange gets turned into request for https://example.com/?items=apple%2Cbanana%2Corange. This means your server code needs to url-decode before splitting at ,. Going with repeated parameters avoids this complexity.
• Prior art of Magnet links, which do not support comma-separated values, and magnet:?tr=udp://tracker1:80&tr=udp://tracker2:80 is how one specifies multiple trackers

[SgtPooki]

Or even rely on discovery strategies in parallel.

I feel like if there are provider hints, we SHOULD attempt to process them before IPNI/DHT to prevent any negative cascading network effort

[vasco-santos]

Why a multiaddr with additional info could be a problem for DHT/IPNI? They should not try to match by multiaddr or anything like that.
Anyway, there would be no reason to remove this info before passing over to those systems if that is a concern and they do not support this

[SgtPooki]

Are CIDs valid in multiaddr at all (besides Peer ID)? might be worth linking to multiaddr spec: https://github.com/libp2p/specs/blob/master/addressing/README.md#multiaddr-in-libp2p

[vasco-santos]

I am a bit confused. CID is not encoded in the multiaddr

[lidel]

Things to add here:

• Empower user with ability to exfiltrate / migrate data from providers that do not announce data on Amino DHT and/or during IPNI (cid.contact) outages
• Improved initial seeding: opportunistically append /p2p/peerid hint to links generated by IPFS Desktop (if publicly diallable)

[vasco-santos]

@lidel I wrote some use cases that may be interesting to add here: https://github.com/vasco-santos/provider-hinted-uri/blob/main/EXPLORATION.md#use-cases

What do you think?

[lidel]

Yes, feel free to copy 1-3 make sense to be included here (unsure about 4, but we can discuss once its copied here).

[aschmahmann]

While I understand that there's a latency improvement that can be had here by hard-coding a provider into the URL in practice when I've seen this come up in the past it's been due to people not wanting to use the "mainnet" routing systems while sort of pretending that the content is available via mainnet (e.g. a pinning service not wanting to advertise their data to the Amino DHT or IPNI, but instead have their users use URIs like ipfs://bafyfoo?provider=<the-pinning-service>. In this light this proposal seems more likely to harm than help the IPFS ecosystem.

Some examples:

• URIs become ephemeral. While encoding ipfs://bafyfoo used to not be ephemeral ipfs://bafyfoo?provider=<pinning-service-that-had-the-cid-when-the-link-was-made> is now ephemeral. Yes, you could fallback to ignoring the provider but:
  • If the ecosystem has come to rely on them for routing then the link is just broken
  • If the ecosystem has come to rely on them for a performance boost then users encoding ipfs://bafyfoo?provider=<pinning-service-that-had-the-data-when-the-link-was-made> into their applications, smart contracts, etc. will now need to figure out how to update the provider part of the URI vs previously when they could just be ephemeral
• This incentivizes pinning service lock-in where moving data off a given provider means latency goes up for all related links unless all the places the link has been shared are updated

Some alternatives to this approach that seem like they resolve much of the problems for users:

• Invest in improving the routing system(s) for mainnet which have received really very little investment over the past several years and which could really use it
• Application developers can hard-code additional routing systems into their applications. For example, if the developers for a given dApp are already hosting all their data and/or paying a pinning service to host the data saying to check that endpoint first seems fine and is inline with how many of them already operate by hard-coding a gateway endpoint provided by the pinning service they pay for storage

It'd be useful to understand why the benefits of this outweigh the associated ecosystem risks

[2color]

I've seen many situations in which you want to share content addressed data with all the benefits of verification and p2p without necessarily caring about long-term persistence. If we don't formalise this in a spec, we end up pushing users to solving this in user/application space (see example), leading to fragmentation, and no ecosystem benefits from a conventional approach.

Application developers can hard-code additional routing systems into their applications. For example, if the developers for a given dApp are already hosting all their data and/or paying a pinning service to host the data saying to check that endpoint first seems fine and is inline with how many of them already operate by hard-coding a gateway endpoint provided by the pinning service they pay for storage

This would typically be done using the Delegated Routing API for which we have a lot of tooling and support in implementations. Practically, this which would involve setting a application specific endpoint somewhere, which — just like a specific provider maddr — can go down, or become stale eventually. But for as long as that delegated routing endpoint is up, it helps the app can map CIDs to provider maddrs.

It seems to me that insisting that this is the recommended to do it, over encoding it along with the CID is overly pedantic, given that the two approaches are not all that dissimilar, with the one exception that your mixing "permanent" information, i.e. the CID, with impermanent, i.e. a specific provider maddr.

URIs become ephemeral. While encoding ipfs://bafyfoo used to not be ephemeral ipfs://bafyfoo?provider= is now ephemeral.

This is the crux of the debate here; and ultimately a question of where this boundary between ephemeral and permanent should be delineated. I happen to think it maps elegantly to query parameters.

But more broadly, content routing is hard and adds a performance tax that undercuts adoption in scenarios where the benefits of verification and content-addressing are desired.

Optimising for successful retrieval by CID should be a higher level goal, and think that the broad strokes proposed here advance this goal. Moreover, it paves a path for incremental adoption of content addressing.

So my take is that that this we should just encode all of these caveats into this IPIP, e.g. strongly recommend against persistence of the provider hint in long term storage like on-chain.

[vasco-santos]

Thanks for writing this @2color . This is exactly what I feel like! I wrote some use cases that can benefit directly from this https://github.com/vasco-santos/provider-hinted-uri/blob/main/EXPLORATION.md#use-cases

Note that some of them are actually depending on a follow up of this (adding tags) to the multiaddr, which for now is a different conversation, as also tags are optional ways to expand on this even further

[aschmahmann]

Independent of my main objection to the idea of provider I'm not sure multiaddr is a great idea here. Maybe it's the best we have, but my suspicion is that importing the not-really-existent multiaddr spec (due to many years of neglect from the libp2p side of things here) into the gateway spec is a pretty unfortunate dependency.

In Go we saw so many badly written parsers of multiaddrs that there's an in progress regex-like library to try to make them easier to work with.

[bumblefudge]

whoa that parser library is neat! would it be possible to just retrospecify how that library does it and call THAT the spec, or at least a starting point for it? i'm not sure this IPIP needs to be blocked on that specification process, but it seems as good a time/occasion as any to finally nail down multiaddr (and make it easier for a provably interoperable parser library to be made for other languages!)

[lidel]

Not a fan of pulling in libp2p concepts into gateway spec here (peerid, multiaddr).
HTTP gateway spec should not come with hard lock-in into libp2p (RASL does not).

What if in the future everything will be HTTP providers – are we cosplaying libp2p with multiaddrs and fake PeerIDs still?

Maybe to do bare minimum to future proof this, state this field is an opaque string that should be parsed as Multiaddrs (if starting with /) OR URIs (everything else). Keeping the door open for a sane implementation of webseeds.

[vasco-santos]

I find surprising that multiaddr is assumed to be a concept of libp2p. I see space for multiaddr usage outside of libp2p as any other multiformat.

Anyway, after syncing with @bumblefudge last week, I totally agree that we should also accept string provider, while keeping multiaddr if starting by /.

[lidel]

If all hints fail, clients SHOULD fall back to default discovery strategies (e.g., DHT/IPNI), if available.

The idea that hints should be executed first, and then regular routing system should be used as a fallback only when all hints failed, is in my opinion, an antipattern:

• ⚠️ it only benefits users in short term
• ⚠️ creates latency and cost center over time

I am having hard time seeing "User benefit" here, without actually writing in the spec that ?provider SHOULD be the fallback evaluated AFTER regular routing system.

Tried to clarify my position with example below.

On why acting on ?provider= should never supersede regular routing, and why this has to be a fallback thing

If ?provider is used exclusively first, that looks good only short term:

• I made a viral content and pinned it to a storage provider (like https://storacha.network, https://pinata.cloud, https://filebase.com) and paid for a month of storage.
• I shared this content as https://gateway.example.com/ipfs/cid?provider=https://storage.provider or ipfs:///cid?provider=https://storage.provider, it became popular, people put this in blogpost, chats, wikipedia put it in citation references, web crawlers consumed it and LLMs now have this URL with this hardcoded hint
• Everyone is happy, because routing system is skipped, third-party gateways or clients instantly fetch data from "webseeed" at https://storage.provider, don't need to ask DHT/IPNI.

Then, I stopped paying storage provider and https://storage.provider/ipfs/cid started returning HTTP Error 404 Not Found because they will not host my content for free.

How does IPFS experience look like now and long term ?

(A) If ?provider= is evaluated FIRST, and the rest of routing is fallback

• 🔴 Every time user opens hardcoded address with ?provider=https://storage.provider there is a delay, because the request to webseed needs to fail before regular routing system engages
  • 🔴 This effectively put users hostage: someone will have to keep paying the specific storage provider to have decreased latency. This also means nobody will have incentive to work on improving routing systems.
  • 🔴 But what if the hardcoded storage provider goes out of business? The latency is there forever.

This is user-hostile and is why ?provider can't be implemented in a way that uses regular routing system as a fallback if all URL hints fail.

(B) If ?provider= is evaluated in PARALLEL to regular routing system

It is better for user-perceived latency, but still not the best for either party:

The user wastes time on a request to https://storage.provider/ipfs/cid however they asked routing in parallel and eventually found a provider. In this scenario

• 🟠 user did not pay the cost of extra latency, but sent parallel HTTP request that always fails.
• 🟠 storage provider will still get requests for CID they no longer pin.. forever.

(C) If ?provider= is evaluated as a fallback (AFTER regular routing system)

I my mind this was always the only sensible way of introducing these ad-hoc routing hints.
Use them as a fallback (or after some delay), when regular routing system does not provide better provider.

• 🟢 user does not send unnecessary requests if there are other providers
• 🟢 storage provider is not receiving requests for content they don't host anymore

[vasco-santos]

This is good to be raised. While I tend to position myself in a different perspective here, I understand that this may have multiple correct ways.

What feels more accurate to me is to allow clients to decide implementation details, or to allow configuration of this, rather than the spec mandate a certain implementation behaviour. The current proposal framing addresses this by stating:

• Clients MAY:
  • ...
  • Apply their own prioritization or fallback strategies. If all hints fail, clients SHOULD fall back to default discovery strategies (e.g., DHT/IPNI), if available. Or even rely on discovery strategies in parallel.

In other words, this frames allows a client to implement A, B, C, or even allow configuration to let user choose a path that is not the default

[lidel]

What feels more accurate to me is to allow clients to decide implementation details, or to allow configuration of this, rather than the spec mandate a certain implementation behaviour.

Reality is, leaving implementation details of "evaluation order" out of spec IS effectively leading to (A): even if someone initially implements (C), over time, motivated people will eliminate "annoying latency" by switching client code from (C) to (B), and then to "remove unnecessary lookups" switch from (B) or (A).

Next steps:

• the IPIP should clearly address risks that come with evaluation order choice, and inform reader about the externalized ecosystem cost listed above ( 🟠 🔴 ).
  • Do not pretend downsides do not exist, but document them and make a compelling case in "User benefit" the benefits outweigh the identified cost for IPFS ecosystem long term.
• the Gateway spec should not ignore the topic of ?provider= evaluation order, but address it in the spec head-on:
  • a "SHOULD" recommendation for (C) with rationale of 🟢
  • a "MAY" for (B) with 🟠 cost documented
  • a "MUST NOT" for (A) with 🔴 risks documented as reason why it is considered harmful
    • we can't block people from YOLOing, but with this, whatever implementation or part of ecosystem want to shut themselves and their users in the foot long erm, we can always point at spec they ignored

[SgtPooki]

Let's call out client security/privacy concerns here as well. i.e. If malicious or nosy tracking entity (nft marketplace?) starts encoding provider hints into all their URLS, every single client can be easily tracked. Another point for not prioritizing provider hints over routing system as called out by @lidel.

[lidel]

💭 alternative approach for this use case is for IPFS Desktop to do manual provide on Amino DHT of the CID that is being shared. This would not require hardcoding anything in shared URL.