Skip to content

IPFS Layer

umeradl edited this page Jul 8, 2026 · 1 revision

IPFS Layer

The blockchain coordinates the network, but it never stores the heavy artifacts. Everything content-shaped in DIN lives on IPFS, and the contracts store and vote on CIDs (content identifiers) only:

  • model weights — genesis model, clients' local models, T1/T2 aggregated models
  • service files — the model owner's Python logic (model.py, client.py, auditor.py, aggregator.py, modelowner.py)
  • manifests — each model's manifest.json
  • test datasets — per-audit-batch evaluation data
  • contract ABIs — custom task-contract ABIs referenced from a manifest

Because a CID is a cryptographic hash of the content, on-chain CID votes double as integrity checks: two aggregators produced the same model if and only if they submitted the same CID.

One interface, three backends

All upload/retrieve traffic in dincli goes through a single abstraction (upload_to_ipfs / retrieve_from_ipfs in dincli/services/ipfs.py) rather than scattered HTTP calls. The backend is chosen by configuration — participants can switch providers without touching any workflow:

Provider When to use Setup
env (default) you run your own IPFS node or any IPFS-compatible HTTP API IPFS_API_URL_ADD / IPFS_API_URL_RETRIEVE in the project .env
filebase (recommended) you want a managed, pinned backend without running a node dincli system configure-ipfs --provider filebase --api-key <filebase_rpc_token>
custom full control — any storage you can wrap in Python --provider custom --service-path /abs/path/to/custom_ipfs.py

Check or change the active provider anytime:

dincli system configure-ipfs                     # show current config
dincli system configure-ipfs --provider env      # switch explicitly

Notes per provider

  • env accepts an API root (http://127.0.0.1:5001/api/v0), full add/cat endpoints, or a retrieve URL template containing {cid}. If you run a local node, make sure artifacts stay pinned — a garbage-collected model breaks the round for everyone who needs it.
  • filebase uploads through Filebase's IPFS RPC API and issues a pin request after every upload; the token is stored in the user-level dincli config (per-provider, ipfs_api_key_<provider>).
  • custom modules must export two functions — upload_to_ipfs(file_path, msg=None) -> str (returns a non-empty CID) and retrieve_from_ipfs(cid, file_path) -> int | None (writes the artifact to file_path). That's the whole contract; anything satisfying it can back the network's storage.

CID-based caching

Content addressing makes caching trivial: same CID, same bytes. dincli caches every fetched artifact in its cache directory keyed by CID, so services, models, and manifests are downloaded only when their CID actually changes — repeated GI phases don't re-fetch unchanged files.

Further reading

Clone this wiki locally