refactor(backend): restructure config to support both networks

[silent-cipher]

Built on top of #569

Major changes

1. Environment variables have been split into two categories:
   • global variables
   • per-network variables
2. Per-network variables are defined using network-specific prefixes (MAINNET_* and CALIBRATION_*).
3. Job scheduling now schedules provider jobs for both networks.
4. Workers now receive the network through job data, and the network is propagated throughout the call stack where necessary.
5. WalletSDKService now maintains a per-network cache.
6. SynapseViemClient / Synapse instances are accessed through the WalletSDKService cache. If an instance is unavailable, it is created on demand using the synapse factory commons. The cache behavior remains unchanged and is not updated after on-demand creation.
7. Network information is also propagated through logContext across checks.
8. Pull pieces continue to be hosted using pieceCid, making the process network-agnostic.

[Copilot]

Pull request overview

Refactors the backend configuration system to support running against multiple Filecoin networks (e.g., calibration + mainnet) from a single deployment, and threads network through chain- and provider-dependent services so jobs/metrics/DB operations are correctly network-scoped.

Changes:

• Replaces the legacy single-network config (app.config.ts + validationSchema) with a new multi-network config loader + Joi env validation + legacy env compatibility shim.
• Updates wallet/chain integration and check services (deal, retrieval, pull-check, piece-cleanup, data-retention, metrics collectors, dev-tools) to accept and use an explicit network parameter and per-network config.
• Extends provider API responses and filtering to support network scoping (controller/service/DTO + blocklists per network).

Reviewed changes

Copilot reviewed 58 out of 58 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
apps/backend/src/worker.module.ts	Switches Nest config wiring to new loadConfig/validateConfig.
apps/backend/src/wallet-sdk/wallet-sdk.service.ts	Refactors wallet SDK to maintain per-network Synapse + provider caches/state.
apps/backend/src/wallet-sdk/wallet-sdk.service.spec.ts	Updates wallet-sdk tests for multi-network state and config shape.
apps/backend/src/retrieval/retrieval.service.ts	Threads network through retrieval flow and deal selection/liveness checks.
apps/backend/src/retrieval/retrieval.service.spec.ts	Updates retrieval tests for network-scoped config/labels/entities.
apps/backend/src/retrieval-addons/strategies/ipfs-block.strategy.ts	Looks up provider info using the deal’s network.
apps/backend/src/pull-check/pull-piece-stream-tracker.service.ts	Updates config typing/imports for new config module.
apps/backend/src/pull-check/pull-piece-stream-tracker.service.spec.ts	Aligns pull-piece stream tracker tests with new config types.
apps/backend/src/pull-check/pull-check.service.ts	Makes pull-check execution explicitly network-scoped (config + synapse client).
apps/backend/src/pull-check/pull-check.service.spec.ts	Updates pull-check tests to pass network and validate new labels/context.
apps/backend/src/providers/providers.service.ts	Adds optional network filter + per-network blocklist application in queries.
apps/backend/src/providers/providers.service.spec.ts	Updates providers service tests for per-network blocklist query clauses.
apps/backend/src/providers/providers.controller.ts	Adds network query param with validation for providers listing endpoint.
apps/backend/src/providers/dto/provider-list-response.dto.ts	Adds network to provider DTO for API responses/docs.
apps/backend/src/piece-cleanup/piece-cleanup.service.ts	Scopes cleanup quotas, DB queries, provider lookups, and Synapse access by network.
apps/backend/src/piece-cleanup/piece-cleanup.service.spec.ts	Updates piece-cleanup tests for network-scoped methods and Synapse sourcing.
apps/backend/src/metrics-prometheus/wallet-balance.collector.ts	Collects wallet balances per active network and labels metrics with network.
apps/backend/src/metrics-prometheus/wallet-balance.collector.spec.ts	Updates wallet-balance collector tests for per-network metric labels.
apps/backend/src/jobs/data-set-creation.handler.ts	Threads network into dataset provisioning/repair calls.
apps/backend/src/http-client/http-client.service.ts	Updates config import to new module.
apps/backend/src/dev-tools/dto/trigger-retrieval.dto.ts	Adds optional network query param validation for retrieval trigger.
apps/backend/src/dev-tools/dto/trigger-deal.dto.ts	Adds optional network query param validation for deal trigger.
apps/backend/src/dev-tools/dev-tools.service.ts	Adds default network behavior and passes network through deal/retrieval operations.
apps/backend/src/dev-tools/dev-tools.controller.ts	Exposes optional network query params for dev-tools endpoints.
apps/backend/src/deal/deal.service.ts	Makes deal creation, dataset liveness/repair, and wallet address/config network-scoped.
apps/backend/src/deal/deal.service.spec.ts	Updates deal service tests for new signatures and per-network config.
apps/backend/src/deal-addons/strategies/ipni.strategy.ts	Updates config import to new module.
apps/backend/src/dataSource/dataSource.service.ts	Updates config import to new module.
apps/backend/src/dataSource/dataSource.service.spec.ts	Updates config import to new module.
apps/backend/src/dataset-liveness/dataset-liveness.service.ts	Makes liveness probes network-aware (synapse client + provider registry access).
apps/backend/src/dataset-liveness/dataset-liveness.service.spec.ts	Updates liveness tests to pass network argument.
apps/backend/src/database/database.module.ts	Updates config import to new module.
apps/backend/src/data-retention/data-retention.service.ts	Makes data-retention polling network-scoped and uses per-network blocklists/providers.
apps/backend/src/config/types.ts	Introduces new typed config model including per-network slices and active networks.
apps/backend/src/config/loader.ts	Implements env-based config loader, including per-network prefixed vars.
apps/backend/src/config/loader.spec.ts	Adds tests for config loader behavior across networks and env layouts.
apps/backend/src/config/legacy-env-compat.ts	Adds compatibility shim translating legacy single-network env vars to prefixed form.
apps/backend/src/config/legacy-env-compat.spec.ts	Adds tests for legacy env translation behavior.
apps/backend/src/config/index.ts	Exposes config entry points (loadConfig, validateConfig) and types.
apps/backend/src/config/env.schema.ts	Adds Joi env validation including dynamic per-network requirements + legacy shim hook.
apps/backend/src/config/env.schema.spec.ts	Adds extensive tests for env schema validation rules.
apps/backend/src/config/env.parsers.ts	Adds domain-level env parsing helpers (networks, run mode, lists).
apps/backend/src/config/env.helpers.ts	Adds low-level env getters/coercion helpers.
apps/backend/src/config/constants.ts	Defines per-network default values and supported env prefixes.
apps/backend/src/config/app.config.ts	Removes legacy single-network config loader and schema.
apps/backend/src/common/synapse-factory.ts	Updates Synapse factory to accept INetworkConfig (wallet-key vs session-key variants).
apps/backend/src/common/sp-blocklist.ts	Updates blocklist helper to accept per-network blocklist fields.
apps/backend/src/common/sp-blocklist.spec.ts	Updates blocklist tests for new config shape.
apps/backend/src/common/logging.ts	Extends job/provider log context types to include network.
apps/backend/src/clickhouse/clickhouse.service.ts	Updates config import to new module.
apps/backend/src/app.module.ts	Switches Nest config wiring to new loadConfig/validateConfig.
apps/backend/src/app.controller.ts	Updates config endpoint to return per-active-network rates/intervals.
apps/backend/.env.example	Documents and demonstrates multi-network env layout and per-network overrides.
.env.example	Updates secrets-only example to use per-network prefixed wallet/session keys.