Skip to content

refactor: generic database config parsing#462

Merged
0237h merged 1 commit intomainfrom
refactor/generic-dbs-config
Apr 2, 2026
Merged

refactor: generic database config parsing#462
0237h merged 1 commit intomainfrom
refactor/generic-dbs-config

Conversation

@0237h
Copy link
Copy Markdown
Collaborator

@0237h 0237h commented Apr 2, 2026

Summary

  • Replace hardcoded database field parsing in dbsConfig.ts with a generic loop that maps any YAML key to ${key}Databases
  • New database types can be added to dbs-config.yaml without modifying the Zod schema — only the ParsedDbsConfig interface needs a new property
  • Rename nftDatabasesnftsDatabases, dexDatabasesdexesDatabases, contractDatabasescontractsDatabases to align with YAML key names (nfts, dexes, contracts)
  • Add Polymarket entry to dbs-config.yaml.example
  • Add unit tests for loadDbsConfig

Refs: #461


🤖 Generated with Claude Code

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors database YAML config parsing to be generic (mapping arbitrary YAML keys to ${key}Databases), updates config property names to match YAML keys (nfts/dexes/contracts), and wires the new names through route code.

Changes:

  • Refactor loadDbsConfig to generically map network-level YAML keys into typed ${key}Databases maps and return an always-initialized config object.
  • Rename config accessors across routes (dexDatabasesdexesDatabases, nftDatabasesnftsDatabases, contractDatabasescontractsDatabases).
  • Add loadDbsConfig unit tests and extend the example YAML (including a Polymarket stanza).

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/config/dbsConfig.ts Implements generic YAML-to-{key}Databases parsing and updates parsed config typing.
src/config/dbsConfig.spec.ts Adds unit tests for loadDbsConfig.
src/config.ts Spreads parsed DB maps into runtime config and updates network derivation to renamed maps.
src/clickhouse/client.ts Updates cluster lookup to use renamed maps and includes new maps.
src/supported-routes.ts Updates DB category map to use renamed config properties.
src/supported-routes.spec.ts Updates tests to use renamed config properties.
src/banner.ts Updates logging to use renamed config properties.
src/routes/networks.ts Updates dex DB reference to renamed config property.
src/routes/health.ts Updates DB fallback selection to renamed config properties.
src/routes/transfers/svm.ts Updates dex DB reference to renamed config property.
src/routes/swaps/{evm,svm,tvm}.ts Updates dex DB reference to renamed config property.
src/routes/pools/{evm,svm,tvm}.ts Updates dex DB reference to renamed config property.
src/routes/ohlcv/{evm,svm,tvm}.ts Updates dex DB reference to renamed config property.
src/routes/dexes/{evm,svm,tvm}.ts Updates dex DB reference to renamed config property.
src/routes/nft/*.ts Updates NFT/contracts DB references to renamed config properties.
dbs-config.yaml.example Updates documentation and adds a Polymarket example entry.
Comments suppressed due to low confidence (1)

src/config.ts:364

  • config.networks is built solely from evmNetworks, svmNetworks, and tvmNetworks, which in turn are derived only from balances/transfers/nfts/dexes/contracts maps and strict type checks (=== 'evm'|'svm'|'tvm'). Any network present only in newly added maps (e.g. polymarketDatabases / scraperDatabases) or with a nonstandard type will never appear in config.networks, which can break endpoints/schemas that rely on config.networks as the canonical network list. If the intent is to support these as real networks, consider including the new maps (and/or a new network-type bucket) in the network list derivation.
    .transform((data) => ({
        ...data,
        evmNetworks: Object.keys({
            ...data.balancesDatabases,
            ...data.transfersDatabases,
            ...data.nftsDatabases,
            ...data.dexesDatabases,
            ...data.contractsDatabases,
        })
            .filter((networkId) => {
                return (
                    {
                        ...data.balancesDatabases,
                        ...data.transfersDatabases,
                        ...data.nftsDatabases,
                        ...data.dexesDatabases,
                        ...data.contractsDatabases,
                    }[networkId]?.type === 'evm'
                );
            })
            .sort(),
        svmNetworks: Object.keys({
            ...data.balancesDatabases,
            ...data.transfersDatabases,
            ...data.nftsDatabases,
            ...data.dexesDatabases,
            ...data.contractsDatabases,
        })
            .filter((networkId) => {
                return (
                    {
                        ...data.balancesDatabases,
                        ...data.transfersDatabases,
                        ...data.nftsDatabases,
                        ...data.dexesDatabases,
                        ...data.contractsDatabases,
                    }[networkId]?.type === 'svm'
                );
            })
            .sort(),
        tvmNetworks: Object.keys({
            ...data.transfersDatabases,
            ...data.dexesDatabases,
        })
            .filter((networkId) => {
                return (
                    {
                        ...data.transfersDatabases,
                        ...data.dexesDatabases,
                    }[networkId]?.type === 'tvm'
                );
            })
            .sort(),
    }))
    .transform((data) => ({
        ...data,
        networks: [...data.evmNetworks, ...data.svmNetworks, ...data.tvmNetworks],
    }))

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/config/dbsConfig.spec.ts Outdated
Comment thread dbs-config.yaml.example
Comment thread src/config/dbsConfig.ts
Replace hardcoded database field parsing in dbsConfig with a generic
loop that maps any YAML key to `${key}Databases`. This allows adding
new database types (e.g., Polymarket) without modifying the schema.

- NetworkConfigSchema uses `.catchall(z.string())` for database keys
- Rename `nftDatabases` → `nftsDatabases`, `dexDatabases` → `dexesDatabases`,
  `contractDatabases` → `contractsDatabases` to match YAML key names
- Add Polymarket entry to dbs-config.yaml.example
- Add dbsConfig unit tests

Refs: #461

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@0237h 0237h force-pushed the refactor/generic-dbs-config branch from c328345 to 2325c93 Compare April 2, 2026 19:14
@YaroShkvorets YaroShkvorets temporarily deployed to refactor/generic-dbs-config - token-api PR #462 April 2, 2026 19:14 — with Render Destroyed
@0237h 0237h merged commit db3c187 into main Apr 2, 2026
3 checks passed
@0237h 0237h deleted the refactor/generic-dbs-config branch April 2, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants