Rename ?format=car URL params to match IPIP-402

[lidel]

This issue is based on discussions I had with @aschmahmann and with @hannahhoward (slack) about around depth=0|1|all.
cc ipfs/specs#348

Problem

The depth=0|1|all parameter was provisionally introduced as part of Rhea project and we quickly realized we need to improve if we want to upstream it to https://specs.ipfs.tech/http-gateways/trustless-gateway/

The name is unfortunate, because after all Graph API interations we've ended up with "depth" that lost original meaning ("how deep to fetch a DAG (or path)"), and ended up meaning "logical depth from the perspective of end user thinking in terms of a single block, a file or a directory enumeration".

In that framing, the name makes very little sense:

• a block can be resolved only via depth=0
• things that support byte-range seeking, like unixfs files, can have max depth=1
• hypothetical depth=2 makes sense only for directories and DAGs built out of IPLD things like DAG-JSON/CBOR documents (but we don't use this for Rhea nor need for gateway atm).

I don’t think we should have an integer range where there are only two possible values with well-defined behavior.

I agree, we ended up with three states:

• 0 - blocks for path + only the root block for terminal CID
  • Gateway uses this for IPFSBackend.GetBlock and for IPFSBackend.ResolvePath
• 1 - blocks for path + all blocks for a file, or a minimal set of blocks to enumerate directory or HAMT
  • Gateway uses this for all IPFSBackend.Get requests, as we don't want to over-fetch directories, and we don't know which path ends with a dir
• all - blocks for path + all blocks for entire DAG
  • Gateway uses this only for TAR (IPFSBackend.GetAll)and CAR (IPFSBackend.GetCAR) responses, but this is also the implicit default for ?format=car, so we don't really need this.

I think we both want to clean this up. and we agree "depth" is simply an invalid term here. I also agree with you, strings are more meaningful than numbers for this abstraction layer.

What we want

• this is not "depth" → this is more a predefined "scope" with specific meaning
• hard to reason what integers mean, or what the parameter is related to → include "car" in name, avoid magical IPLD/ADL terms, and other PL-specific words
• we don't accept integers >1 → make it opaque string
• we may want to use "depth" for selecting actual DAG depth in the future → rename

Solution

Based on my notes and discussion with Hannah, we agreed car-scope=block|file|all is a better framing:

• depth=0 → car-scope=block
• depth=1 → car-scope=file (non-files, like directories, and IPLD Maps only return blocks necessary for their enumeration)
  • we use file here to make more sense to non-PL user. "file" is a synonym for "the blocks needed to interact with the ADL in the transformed view", and the pathing on /ipfs/ defines the implicit ADLs to use.
• depth=all → car-scope=all (implicit default when car-scope is unset, send everything)

This is way easier to reason about, no need to read docs, hard to make a mistake,
and a better fit for a future IPIP that updates https://specs.ipfs.tech/http-gateways/trustless-gateway/ which we want to do this year.

Transition plan.

• bifrost-gateway starts sending requests with both depth and car-scope
  • done: feat: car-scope support #81
• Saturn L1 forwards car-scope to lassie Implement depth=0 for CAR requests filecoin-saturn/L1-node#323 (comment)
• Lassie implements car-scope → update lassie v0.8.1 filecoin-saturn/L1-node#324
• confirm Lassie implements car-scope=block → https://github.com/filecoin-saturn/L1-node/pull/324/files#r1160386359
• biforst-gateway stops sending depth (chore: remove unused CAR URL params #144)
• draft IPIP with car-scope → IPIP-402: Partial CAR Support on Trustless Gateways ipfs/specs#402 (comment)
• live with it for some time

Updated plan

• bifrost-gateway starts sending requests with depth, car-scope and dag-scope AND both bytes and entity-bytes as noted in IPIP-402 – refactor: use IPIP-402 param names when GRAPH_BACKEND=true #108
• new parameters released and deployed to Rhea prod
• Lassie implements new parameters Update HTTP server + protocol to support IPIP-402 parameter renames filecoin-project/lassie#238
• Saturn L1 implements new parameters
• dotStorage implements new parameters
• Live with it for some time
• remove old params from bifrost-gateway chore: remove unused CAR URL params #144

[lidel]

Eyeballed metrics on staging, unsurprisingly the file scope is requested the most often:

[lidel]

We had another round of renames in ipfs/specs#402 (comment):

• name changes based on rough consensus. (Juan, Rod, and Adin all suggested removing ambiguity and name dependency on CAR and UnixFS formats, making things more future-proof):
  • car-scope=file → dag-scope=entity
  • bytes → entity-bytes

Repurposing this issue to align bifrost-gateway implementation with specs.