feat(Rarity System): add automatic NFT Rarity rank and scoring based on traits distribution#1
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a rarity system for NFTs by computing rarity scores/ranks/percentiles/tiers from trait distributions, persisting the results on NFTEntity, and triggering recalculation when collections change.
Changes:
- Adds rarity fields to the GraphQL schema and generated
NFTEntitymodel, plus a DB migration for new columns and indexes. - Introduces rarity computation + batch DB update utilities, with “dirty collection” tracking and flushing from the main mappings loop.
- Hooks rarity dirty-marking into mint/burn/metadata/attribute handlers and adds Vitest coverage for determinism and edge cases.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/rarity.test.ts | Adds unit tests for scoring, deterministic ranking, tier boundaries, and traitless edge cases. |
| src/model/generated/nftEntity.model.ts | Extends the ORM model with nullable rarity fields and per-field indexes. |
| src/mappings/utils/rarity.ts | Implements rarity scoring/ranking, dirty-collection tracking, and bulk SQL updates. |
| src/mappings/uniques/setMetadata.ts | Marks collection rarity dirty on uniques metadata events. |
| src/mappings/uniques/setAttribute.ts | Marks collection rarity dirty on uniques attribute changes. |
| src/mappings/uniques/mint.ts | Marks collection rarity dirty on uniques mint. |
| src/mappings/uniques/burn.ts | Marks collection rarity dirty on uniques burn. |
| src/mappings/nfts/setMetadata.ts | Marks collection rarity dirty on nfts metadata events. |
| src/mappings/nfts/setAttribute.ts | Marks collection rarity dirty on nfts attribute changes. |
| src/mappings/nfts/mint.ts | Marks collection rarity dirty on nfts mint. |
| src/mappings/nfts/burn.ts | Marks collection rarity dirty on nfts burn. |
| src/mappings/index.ts | Flushes dirty rarity recomputation during batch processing. |
| schema.graphql | Adds rarity fields to NFTEntity with @index. |
| db/migrations/1760160000000-Data.js | Adds DB columns and composite indexes for collection-level rarity queries. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
📝 WalkthroughWalkthroughAdds NFT rarity metrics: DB columns and indexes, GraphQL fields, event hooks marking collections dirty, a rarity utility that computes and persists scores/ranks/percentiles/tiers (with backfill and batching), and tests for the rarity logic. Changes
Sequence DiagramsequenceDiagram
participant Event as Blockchain Event
participant Handler as Event Handler
participant Cache as Dirty Cache
participant Frame as Main Frame
participant Rarity as Rarity Service
participant DB as Database
Event->>Handler: emit (mint / burn / setAttribute)
Handler->>Cache: markCollectionRarityDirty(collectionId)
Cache-->>Handler: ack
Note over Frame,Rarity: end of block processing
Frame->>Rarity: flushDirtyCollectionRarity(store)
activate Rarity
Rarity->>DB: SELECT tokens + attributes for collection
DB-->>Rarity: token rows
Rarity->>Rarity: calculateCollectionRarity(tokens)
Rarity->>DB: Bulk UPDATE nft_entity (score, rank, percentile, tier)
DB-->>Rarity: update result
Rarity-->>Frame: flush complete
deactivate Rarity
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
No actionable comments were generated in the recent review. 🎉 Tip Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@src/environment.ts`:
- Around line 11-12: The RARITY_BACKFILL_PER_BATCH environment value can parse
to NaN and get passed into SQL LIMIT; update the initialization of
RARITY_BACKFILL_PER_BATCH to validate the parsed value (e.g., parse with
Number.parseInt, then check Number.isFinite/Number.isNaN or Number.isInteger and
ensure > 0) and fall back to a safe default (like 10) or clamp to a minimum (1)
when the value is invalid or <= 0; ensure consumers such as
flushMissingCollectionRarity receive a guaranteed positive integer.
In `@src/mappings/utils/rarity.ts`:
- Around line 238-250: The code clears dirtyCollectionIds before processing, so
if updateCollectionRarity(store, collectionId) throws, remaining dirty IDs are
lost; instead preserve pending IDs by not clearing the set up-front — iterate
over a snapshot (collectionIds = Array.from(dirtyCollectionIds)) but only remove
each collectionId from dirtyCollectionIds after a successful update (or on
success call dirtyCollectionIds.delete(collectionId)), and on error re-throw
without removing the failed and remaining IDs so they remain scheduled for
retry; use the existing logger.error and updateCollectionRarity references when
implementing the change.
🧹 Nitpick comments (2)
db/migrations/1760160000000-Data.js (1)
10-16: Consider adding composite index for(collection_id, rarity_score).You have composite indexes for
(collection_id, rarity_rank),(collection_id, rarity_tier), and(collection_id, rarity_percentile), but not for(collection_id, rarity_score). If you expect queries that filter or sort byrarity_scorewithin a collection, you should add the corresponding composite index for consistency.Proposed fix
await db.query(`CREATE INDEX "IDX_nft_entity_collection_rarity_rank" ON "nft_entity" ("collection_id", "rarity_rank") `) await db.query(`CREATE INDEX "IDX_nft_entity_collection_rarity_tier" ON "nft_entity" ("collection_id", "rarity_tier") `) await db.query(`CREATE INDEX "IDX_nft_entity_collection_rarity_percentile" ON "nft_entity" ("collection_id", "rarity_percentile") `) + await db.query(`CREATE INDEX "IDX_nft_entity_collection_rarity_score" ON "nft_entity" ("collection_id", "rarity_score") `)And in
down():async down(db) { + await db.query(`DROP INDEX "public"."IDX_nft_entity_collection_rarity_score"`) await db.query(`DROP INDEX "public"."IDX_nft_entity_collection_rarity_percentile"`)src/mappings/uniques/setAttribute.ts (1)
40-41: Consider skipping rarity marking for collection-level attribute changes.When
event.sn === undefined, the attribute is being set on the collection itself, not on an NFT. Collection-level attributes typically don't affect NFT rarity calculations (which are based on NFT trait distribution within the collection). Marking rarity dirty in this case triggers an unnecessary recalculation.Proposed fix
await context.store.save(final) - markCollectionRarityDirty(event.collectionId) + // Only mark rarity dirty for NFT attribute changes, not collection-level attributes + if (event.sn !== undefined) { + markCollectionRarityDirty(event.collectionId) + }
|
@coderabbitai review, and recheck comment you left |
|
✅ Actions performedReview triggered.
|
vikiival
left a comment
vikiival
left a comment
There was a problem hiding this comment.
The problem is that once in indexer crashes, you will lose the items in the set, and additionally, you are forced to recompute it almost every time.what I would do is to have it as a handler on the mint/setAttribute and once collection is done, I would recompute the whole rarity system
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
from what i understand/checked is that the indexer processes batches transactionally and replays from last committed state after restart, so the in-memory dirty set is rebuilt during replay rather than permanently lost. let me know if im missing something
I agree with the recompute concern but also I can see some issues with “recompute only when collection is done”
The current batch dirty-recompute approach keeps data fresher while still controlling recompute frequency.
|
|
Fair thanks for the explanation! Feel free to deploy imo |
Summary by CodeRabbit
New Features
Tests