Filecoin storage support: which provider, given Lighthouse retrieval is payment-gated?

[umeradl]

Background

DIN currently uses Filebase as its default IPFS pinning provider. All model artifacts (client updates, aggregated models, manifests, service files) flow through dincli/services/ipfs.py and are referenced on-chain by CID only — the contract layer is fully provider-agnostic. Adding a Filecoin-backed provider therefore requires only an adapter behind the existing custom provider path: no contract changes, no CID format changes.

We want Filecoin-backed storage because:

• Decentralisation: a trust-minimised ML coordination network shouldn't depend on a single centralised pinning vendor.
• Workload fit: model weight updates are write-once, content-addressed, and referenced across training rounds — permanency and auditability matter more than ultra-low read latency.
• Cost: at Filecoin rates, a typical round (100 validators × 10 MB updates ≈ 1 GB) costs a fraction of a cent — effectively a rounding error in any fee structure.

The internal design doc (Developer/discussion/add-filecoin-support.md) recommended Lighthouse as the first integration, and PR #16 implemented a Lighthouse provider adapter.

What we found: Lighthouse retrieval is payment-gated ⚠️

While testing PR #16 with a real Lighthouse account (not just mocks):

• ✅ Upload works exactly as documented and returns a valid CID.
• ❌ Retrieval does not. GET https://gateway.lighthouse.storage/ipfs/<cid> returns 402 Payment Required — including for the file's own uploader, with and without the uploader's own API key attached (identical response either way, so this is not an auth-header issue).
• ❌ The content is not discoverable on the public IPFS network at all. A generic public gateway (ipfs.io) fails for the same CID with "no providers found for the CID".

This breaks DIN's core assumption: an artifact's CID must be retrievable by other participants (aggregators fetching client updates, auditors fetching models) — not just uploadable. A provider where even the uploader can't read their own file back without a paid retrieval plan is not viable as-is.

The question

How should we add Filecoin-backed storage, and which provider should we choose?

Candidates we're aware of:

Option	Pros	Concerns
Storacha (Web3.Storage)	UCAN delegated-upload capabilities are a first-class primitive — exactly the scoped credential model we need for sponsored uploads in P3; multi-provider Filecoin deals	Need to verify retrieval is genuinely open (public gateway + IPFS DHT announcement), not gated like Lighthouse
Lighthouse + paid retrieval plan	Adapter already written in PR #16	Retrieval gated behind payment; unclear if a plan makes CIDs publicly resolvable or only via their gateway
Filebase Filecoin bucket	Zero migration friction — same API and credential model we use today	Filebase remains the centralised intermediary; least decentralisation gain
Direct Filecoin deals / FVM	Maximum trustlessness; smart-contract-native storage deals (escrow, SLA verification, auto-renewal)	Deal lifecycle management is significant operational complexity; retrieval latency
Something else?	—	—

Specific questions:

1. Is Storacha's retrieval path actually open? Before we swap adapters, has anyone verified that content uploaded to Storacha resolves via public IPFS gateways / is announced to the DHT?
2. Is there any Filecoin-backed provider where uploaded CIDs are reliably retrievable by third parties for free (or where retrieval cost is bounded and payable by the model owner, not each reader)?
3. Should we reconsider an FVM-native approach — storage deals managed by contract (escrow, provider SLA verification, automated renewal) — even if it's more work, given it aligns with the P3 sponsored-upload/storage-budget design?
4. Is a hybrid sensible? Filecoin for durable archival + a hot IPFS pin (even Filebase) for the active round's retrieval path?

Any hands-on experience with these providers' retrieval behaviour — especially failure modes like the one we hit with Lighthouse — would be extremely valuable.

cc @Abidoyesimze — you've built on FVM (storage-deal escrow with SLA verification and auto-renewal), which is directly relevant here; would love your take on the provider choice and whether an FVM-native path is worth the complexity.
cc @Santiagocetran (PR #16 author)