Skip to content

Conversation

@rickyrombo
Copy link
Contributor

@rickyrombo rickyrombo commented Oct 14, 2025

Main changes:

  • Refactors Solana Indexer to be composed of many indexers, each with their own package
  • Refactors sol_unprocessed_txs to sol_retry_queue since the new structure needs to support retries not just on transactions but on updates from the gRPC
  • Refactors the indexer responsible for DBC pools to grab DBCs by address rather than config
  • Adds batching of the subscriptions that paginates for when we scale too much
  • Adds support for DAMM V2 pool and position tracking (and respective tables)
  • Adds claimable fee info to v1_coin and /v1_coinsresponses using newcalculate_artist_coin_fees()` sql function
  • Adds columns damm_v2_pool and dbc_pool to the artist_coins table
  • Changes notification used in Solana Indexer to fire specific notifs for each composed indexer

Bonus:

  • Adds a withRetries helper that doesn't have a result
  • Adds a Uint256LE helper struct to deserialize the 256-bit little-endian integers used in the meteora accounts so we can use our existing deserializing strategy
  • Fixes a bug where ProcessSignature would ignore signatures it had cached, even on failures/retries
  • Makes coinTicker and coin requests share the same common SQL to prevent bugs where only one gets updated
  • Makes Solana Indexer check more frequently for unprocessed updates in dev to make development easier
  • Fixes transaction retry ticker job - was exiting early before and not running(!)
  • Improves make test-schema to always use the test db, and tear-down and re-up the test db when running to ensure that the schema state is correct (adds a health check to db container to ensure it's up before attempting to dump schema)

Known items:

  • There's now artist_coin_pools and sol_meteora_dbc_pools - the former has price and price_usd, the latter does not. The former is updated by the job, the latter by the indexer. Will be following up with another PR to move away from using the former by doing the calculations of price in SQL, and then a PR after that to remove it entirely to avoid confusion. In the meantime, the read-side API will be using the artist_coin_pools table.
  • The testing of Solana Indexer is a bit less coverage than it used to be, and relies far more heavily on real world fixtures and probing a real database, which I think makes things slightly slower but has more integration coverage. I think that's an ok tradeoff. I do want to circle back and test the Start method of each indexer in a quick followup if the patterns used here make sense.

@gitguardian
Copy link

gitguardian bot commented Oct 15, 2025

⚠️ GitGuardian has uncovered 6 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
21616586 Triggered Generic High Entropy Secret 38b886c solana/indexer/token/indexer_test.go View secret
21650187 Triggered Generic High Entropy Secret d16ba21 solana/indexer/damm_v2/indexer_test.go View secret
21650188 Triggered Generic High Entropy Secret d16ba21 solana/indexer/damm_v2/indexer_test.go View secret
1606950 Triggered Generic High Entropy Secret d16ba21 solana/indexer/damm_v2/indexer_test.go View secret
21616588 Triggered Generic High Entropy Secret 38b886c solana/indexer/token/indexer_test.go View secret
21650189 Triggered Generic High Entropy Secret d16ba21 solana/indexer/damm_v2/indexer_test.go View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@rickyrombo rickyrombo changed the title Mjp solana indexer packages Refactor Solana Indexer Oct 15, 2025
@rickyrombo rickyrombo changed the title Refactor Solana Indexer Refactor Solana Indexer, Support DAMM V2 Oct 15, 2025
@rickyrombo rickyrombo merged commit c5bf506 into main Oct 17, 2025
5 checks passed
@rickyrombo rickyrombo deleted the mjp-solana-indexer-packages branch October 17, 2025 18:05
schottra added a commit that referenced this pull request Oct 24, 2025
* main:
  Add logo_uri to user coin (#487)
  [PE-7200] Add wallet coins endpoint (#486)
  Solana Indexer DBC pool improvements (#485)
  Remove log line (#484)
  Don't update DBC pool address from job to prevent spam to slack (#483)
  Refactor Solana Indexer, Support DAMM V2 (#473)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants