feat: listUploads includes parts property with list of links to CARs that contain it

[gobengo]

Motivation:

• npx @web3-storage/w3 list --json should show CAR CIDs containing the data CID where possible #2343

Why touch the sql files?

• the db tests wouldn't run cleanly without these changes. It would try to drop tables but fail due to dependencies on the uuid extension, other cols, etc. I think these changes are safe/good and will help others too but I can remove them if needed

[gobengo]

example upload list item (note partOf):

{
  "_id": "2",
  "type": "Upload",
  "name": "rand-07270196086601688",
  "created": "2024-01-09T01:17:28.22+00:00",
  "updated": "2024-01-09T01:17:28.22+00:00",
  "cid": "QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR",
  "dagSize": 1000,
  "pins": [
    [
      null
    ]
  ],
  "partOf": [
    "ipfs://bagbaiera6xcx7hiicm7sc523axbjf2otuu5nptt6brdzt4a5ulgn6qcfdwea"
  ],
  "deals": []
}

[alanshaw]

Maybe just pluralise as parts? I like that this is similar to a partition content claim.

[gobengo]

Will do.

Just to check my understanding. Do we ever expect there to be more than one CAR that the Upload is sharded across? In my light testing it looked like only CARs with all blocks included. So I was thinking we could make a slightly different claim than just 'it's spread across these parts' but instead 'it's entirely within this CAR, i.e. it is a subset/partOf these CARs'.

Just now, I did make the change to call it parts as you suggested.

[alanshaw]

It can be multiple. If they send another CAR with the same root CID it should be added to the list.

[github-actions]

Website preview 🔗✨

• ⛅️ https://c0c78f1a.web3-storage-staging.pages.dev

build log

[gobengo]

@alanshaw Thanks for the review. I think I made changes for each of your requests, and now the tests here are green.

I'll work in a branch off of this to add support for the backup table, but would love to test this part in staging++ and see how it goes.

[gobengo]

This patch would add in support for the backup table #2352

[gobengo]

✅ tested on staging

⚡ npm -s start -w packages/w3 -- list --json | head -n1 | jq .
{
  "_id": "315318962269342672",
  "type": "Car",
  "name": "Upload at 2024-01-22T19:10:44.048Z",
  "created": "2024-01-22T19:10:44.048+00:00",
  "updated": "2024-01-22T19:10:44.048+00:00",
  "cid": "bafybeieevwnu57cbcp5u6jsy6wxpj2waq5gfq5gc4spss4skpzk34vvxyy",
  "dagSize": 8848144,
  "pins": [
    {
      "status": "Pinned",
      "updated": "2024-01-22T19:10:44.048+00:00",
      "peerId": "bafzbeibhqavlasjc7dvbiopygwncnrtvjd2xmryk5laib7zyjor6kf3avm",
      "peerName": "elastic-ipfs",
      "region": null
    }
  ],
  "parts": [
    "bagbaierakuersmo7wndedhwk43e5xwcpzwenuda3dhpcsvkfibewg5gxl7oa"
  ],
  "deals": []
}

that part CID corresponds to https://bafybeieevwnu57cbcp5u6jsy6wxpj2waq5gfq5gc4spss4skpzk34vvxyy.ipfs-staging.w3s.link/