ProtoFire Milestone 3 - Indexer Integration Guide For Explorer & Indexer-spo-extension

[cosmir17]

ProtoFire Milestone 3 - Indexer Integration Guide

Date: October 13, 2025 (Updated: October 22, 2025)
For: ProtoFire Team (Matias, Manuel, Evgeny)
From: Midnight Indexer Team (Sean, Heiko)
Purpose: Integration guide for Midnight Explorer, DUST DApp, and SPO Extension

---

⚠️ Important Update (October 22, 2025)

DUST DApp Status Change:

The CNGD (cNIGHT Generates DUST) feature is currently blocked by multiple dependencies:

1. Casey's Cardano Smart Contracts: Written, Off-chain code in progress, ⏳ Not integrated into Node
2. Ledger Update: ⏳ ETA TBD
3. Node Integration: ⏳ Depends on Ledger update + Casey's off-chain code completion
4. Indexer Deployment: ⏳ Few days after Node update
5. PREVIEW Environment: ⏳ Not ready for CNGD testing

Current Reality:

• node-dev-01: CNGD tables are EMPTY (no contracts deployed, no registration data)
• PREVIEW: Not ready for CNGD (pending dependency chain)
• Testnet: Not ready for CNGD

What This Means:

• ❌ Cannot test real CNGD registrations on any environment yet
• ✅ Indexer GraphQL API is ready (infrastructure complete)
• ✅ Continue using Blockfrost for registration checks (currently the ONLY option)
• ✅ Mock generation rates locally for demos

Timeline: Unclear until Ledger ETA is confirmed by Adam Reynolds

See updated sections below for details.

---

Overview

This guide covers the midnight-indexer GraphQL API endpoints your three deliverables require:

1. Midnight Explorer - Block/transaction exploration UI
2. DUST DApp - DUST generation status tracking
3. Midnight SPO Indexer Extension - Independent SPO service

Good News: All required endpoints are available in main branch (version 3.0.0-alpha.5+)

---

1. Midnight Explorer

What You're Using

Your Explorer queries the following endpoints from midnight-indexer:

Queries:

• block(offset: BlockOffset) - Get block by height or latest
• transactions(offset: TransactionOffset!) - Get transaction by hash
• contractAction(address: HexEncoded!, offset: ContractActionOffset) - Get contract by address

Subscriptions:

• blocks(offset: BlockOffset) - Real-time block stream via WebSocket

For Testing/Development

Use node-dev-01 for testing your Explorer:

NEXT_PUBLIC_INDEXER_URL=https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql

Why node-dev-01?

• ✅ Available now with full infrastructure
• ✅ API v3 endpoint (no redirects)
• ✅ All Explorer queries work (block, transactions, contractAction, blocks subscription)
• ✅ Has DUST features (if needed for testing)
• ⚠️ Development environment only - may be reset without notice

About Your Current .env.example:

Your .env.example may reference a testnet URL, but don't use it yet. The current testnet runs an older indexer version without DUST/cNIGHT features. When testnet is upgraded with DUST infrastructure, we'll announce the endpoint.

Example Queries

Get Latest Block

query GetLastBlock {
   block {
      height
   }
}

Subscribe to Blocks

subscription BlocksFromHeight {
   blocks(offset: { height: 100 }) {
      author
      height
      timestamp
      hash
      transactions {
         hash
      }
   }
}

Status

✅ All endpoints available in main branch (3.0.0-alpha.5+)
✅ Available on node-dev-01 for testing - Full infrastructure with Cardano integration
⚠️ Use node-dev-01 URL - https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql
📋 Testnet with DUST features - Will be announced when testnet is upgraded

---

2. DUST DApp

What You're Using

Your DUST DApp uses the dustGenerationStatus query to check if Cardano stake keys are registered and get their generation status.

Important Changes

If you're using feat/cnight-generates-dust branch, you need to migrate to main branch:

Field Name Change

Feature Branch	Main Branch
isRegistered	registered

Before (feat/cnight-generates-dust):

query GetDustGenerationStatus($cardanoStakeKeys: [HexEncoded!]!) {
   dustGenerationStatus(cardanoStakeKeys: $cardanoStakeKeys) {
      cardanoStakeKey
      dustAddress
      isRegistered          # OLD FIELD NAME
      generationRate
   }
}

After (main branch 3.0.0-alpha.5+):

query GetDustGenerationStatus($cardanoStakeKeys: [HexEncoded!]!) {
   dustGenerationStatus(cardanoStakeKeys: $cardanoStakeKeys) {
      cardanoStakeKey
      dustAddress
      registered            # NEW FIELD NAME
      nightBalance
      generationRate
      currentCapacity
   }
}

Query Definition

type DustGenerationStatus {
   cardanoStakeKey: HexEncoded!
   dustAddress: HexEncoded
   registered: Boolean!
   nightBalance: String!
   generationRate: String!
   currentCapacity: String!
}

Response Example

Registered Address:

{
   "data": {
      "dustGenerationStatus": [{
         "cardanoStakeKey": "0xabc123...",
         "dustAddress": "0xdef456...",
         "registered": true,
         "nightBalance": "1000000",
         "generationRate": "8267000000",
         "currentCapacity": "2500000000000000000"
      }]
   }
}

Unregistered Address:

{
   "data": {
      "dustGenerationStatus": [{
         "cardanoStakeKey": "0x789012...",
         "dustAddress": null,
         "registered": false,
         "nightBalance": "0",
         "generationRate": "0",
         "currentCapacity": "0"
      }]
   }
}

Limits

• Maximum 10 stake keys per request (DOS protection)
• Exceeding this limit returns error: "maximum of ten stake keys allowed"

Development Environment

Updated Status (Oct 22, 2025):

node-dev-01: https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql

• ✅ GraphQL API operational
• ✅ dustGenerationStatus query available
• ❌ CNGD tables are EMPTY (no contracts deployed, no registration data)
• ⚠️ Cannot test real CNGD registrations

PREVIEW: Environment exists but not ready for CNGD testing

• Pending Ledger → Node → Indexer dependency chain
• Timeline unclear

Why Tables Are Empty:

• No CNGD smart contracts deployed to Cardano Preview
• Node only supports dummy contract addresses (no real registrations happening)
• Casey's contracts not yet integrated into Node

What You CAN Test:

• GraphQL query syntax and response structure
• API endpoint connectivity
• Error handling for unregistered addresses

What You CANNOT Test:

• Real Cardano registration transactions
• Actual CNGD event observation
• Real generation rates or DUST tracking

Test Query (returns empty data):

curl -X POST https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql \
  -H "Content-Type: application/json" \
  -d '{
    "query": "query { dustGenerationStatus(cardanoStakeKeys: [\"0x00\"]) { registered nightBalance generationRate currentCapacity } }"
  }'

Expected response: registered: false, all values zero/null

Why Blockfrost is Currently Necessary

Updated (Oct 22, 2025):

Blockfrost API is not a workaround - it's currently the ONLY option for checking real Cardano registration status. Here's why:

The Two-Blockchain Architecture:

CNGD involves two separate blockchains:

1. Cardano: Where users register address mappings (Cardano address → Midnight DUST address)
2. Midnight: Where DUST is generated based on those registrations

Why Indexer Can't Help Yet:

The data flow requires:

1. User submits registration transaction on Cardano
2. Cardano → Midnight Node (via DB Sync observation)
3. Node processes registration → emits events on Midnight
4. Indexer observes Midnight events → populates cnight_registrations table
5. You query dustGenerationStatus to get registration status

Current Blocker:

This chain is broken because:

• Casey's Cardano smart contracts not integrated into Node yet
• Node doesn't recognize the new contract datum structure
• No registrations are being observed/processed
• Indexer CNGD tables remain empty

What Blockfrost Provides:

Blockfrost gives you direct access to Cardano blockchain state:

• Real-time UTXO lookups at mapping validator addresses
• Checks if Cardano address already has registration UTXO
• Verifies registration status before attempting new registration
• Works independently of Midnight infrastructure

When You Can Stop Using Blockfrost:

Once the dependency chain is complete:

1. ✅ Casey's contracts written → 🔄 Off-chain code completed → ⏳ Integrated into Node
2. ⏳ Ledger update deployed
3. ⏳ Node updated with new contract recognition
4. ⏳ Indexer deployed with updated Node version
5. ⏳ Environment (PREVIEW/Testnet) fully configured

Timeline: Currently unclear - blocked on Ledger ETA from Adam Reynolds

For Now:

• Continue using Blockfrost - it's the correct approach, not a workaround
• Mock generation rates locally for demos (indexer field exists but no data)
• Wait for dependency chain completion before switching to indexer

Documentation

Full technical documentation: #439 (comment)

Status

✅ GraphQL API Available in Main Branch (3.0.0-alpha.5+)
✅ API Endpoint Operational on node-dev-01
❌ NO REAL DATA on any environment (CNGD tables empty)
❌ NOT Ready on PREVIEW (pending dependency chain)
❌ NOT Ready on Testnet (no timeline yet)
⏳ Blocked By: Casey's contracts integration → Ledger update → Node update → Indexer deployment
⚠️ Action Required: Update field name from isRegistered to registered
📋 For Production: Continue using Blockfrost (currently the ONLY option)

---

3. Midnight SPO Indexer Extension

Architecture

Your SPO extension is an independent Rust service that doesn't query midnight-indexer:

• spo-indexer: Fetches SPO data from Midnight node and Cardano
• spo-api: Exposes its own GraphQL API on port 8090
• Database: Separate PostgreSQL database
• NATS: Separate pub/sub messaging

Your GraphQL API

Endpoint: http://localhost:8090/api/v1/graphql (your own API, not ours)

Queries You Expose:

1. serviceInfo - Service metadata
2. spo_identities(limit, offset) - List SPO identities
3. spo_identity_by_pool_id(pool_id_hex) - Get SPO identity
4. pool_metadata(pool_id_hex) - Get pool metadata
5. spo_composite_by_pool_id(pool_id_hex) - Combined data
6. stake_pool_operators(limit) - Top SPOs
7. spo_performance_latest(limit, offset) - Latest metrics
8. spo_performance_by_spo_sk(spo_sk_hex, limit, offset) - Performance by SPO
9. epoch_performance(epoch, limit, offset) - Epoch-specific performance
10. spo_list(limit, offset) - SPO list
11. spo_by_pool_id(pool_id_hex) - Get SPO by pool ID
12. current_epoch_info - Current epoch info
13. epoch_utilization - Epoch utilization

Integration with Midnight-Indexer

Potential Usage (to be confirmed):

• Your SPO indexer might subscribe to our blocks subscription to get real-time block data
• Would track block author field to determine which validator produced each block

If you do query midnight-indexer:

• Use endpoint: /api/v3/graphql
• Available subscription: blocks(offset: BlockOffset)

Status

✅ Independent Service - No changes needed from midnight-indexer side

---

API Version Information

Current API Version: v3

Endpoints:

• HTTP: /api/v3/graphql
• WebSocket: /api/v3/graphql/ws

Backward Compatibility:

• /api/v1/graphql → Automatically redirects to /api/v3/graphql
• /api/graphql → Automatically redirects to /api/v3/graphql

Why v3? v2 was skipped to align with Midnight ecosystem versioning.

Deployed Environments

Updated Status (Oct 22, 2025):

Development (node-dev-01)

• Endpoint: https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql
• Purpose: Development, API testing, query validation
• Infrastructure: ✅ Node and Indexer deployed
• CNGD Status: ❌ EMPTY TABLES (no contracts deployed, no registration data)
• What Works:
  • ✅ GraphQL API operational
  • ✅ Query syntax testing
  • ✅ Explorer queries (blocks, transactions, contracts)
• What Doesn't Work:
  • ❌ Real CNGD registrations
  • ❌ DUST generation tracking
  • ❌ Actual Cardano observation of registrations
• Stability: ⚠️ May be reset or updated without notice
• Version: 3.0.0-alpha.5 or later
• Use For: API testing only, NOT full CNGD integration testing

PREVIEW

• Status: ⏳ Environment exists but NOT ready for CNGD
• Blocker: Pending dependency chain (Ledger → Node → Indexer)
• Timeline: Unclear - Ledger ETA needed from Adam Reynolds
• Access: Giles Cope mentioned indexer needs to be made public (SRE work)
• When Ready: Will support full CNGD testing with real contracts
• For Now: Not available for ProtoFire testing

Testnet

• Status: ❌ NOT ready for cNIGHT/DUST features
• Reason: Same dependency chain blockers as PREVIEW
• Timeline: TBD - will be announced when available
• For Now: Use Blockfrost for production Cardano registration checks

---

Testing Your Integration

1. Explorer Testing

Testing with node-dev-01 (Recommended):

cd midnight-explorer
npm install
NEXT_PUBLIC_INDEXER_URL=https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql npm run dev

Or Local Testing:

# Terminal 1: Start midnight-indexer
cd midnight-indexer
just run-node
just run-indexer-standalone

# Terminal 2: Start Explorer
cd midnight-explorer
npm install
NEXT_PUBLIC_INDEXER_URL=http://localhost:8088/api/v3/graphql npm run dev

Verification:

• Browse to http://localhost:3000
• Check block list loads
• Search for specific block by height
• Search for transaction by hash
• Verify WebSocket subscription updates blocks in real-time

2. DUST DApp Testing

Updated (Oct 22, 2025):

Current Testing Limitations:

You CANNOT do full integration testing yet because:

• ❌ No contracts deployed on any environment
• ❌ Node doesn't recognize Casey's contract structure
• ❌ Indexer CNGD tables are empty
• ❌ No real registration data available

What You CAN Test:

1. API Connectivity and Query Syntax:

curl -X POST https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql \
  -H "Content-Type: application/json" \
  -d '{
    "query": "query { dustGenerationStatus(cardanoStakeKeys: [\"0x00\"]) { cardanoStakeKey registered nightBalance generationRate currentCapacity } }"
  }'

Expected: registered: false, all values zero/null

2. Frontend Integration:

• Test query structure and error handling
• Mock response data locally
• Build UI components with dummy data

3. Blockfrost Integration:

• Test real Cardano registration checks
• Verify UTXO lookups at validator addresses
• Build complete registration flow

GraphQL Playground:
Visit https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql for API exploration, but remember all CNGD queries return empty data.

When Full Testing Will Be Possible:

Once dependency chain completes:

1. Casey's contracts integrated into Node
2. Ledger updated and deployed
3. Node deployed with new contract recognition
4. Indexer deployed with updated Node
5. PREVIEW/Testnet configured and accessible

For Your MS5 Demo (Oct 24):

• Use Blockfrost for registration status
• Mock generation rates locally
• Demo UI/UX with dummy data

3. SPO Extension Testing

Local Testing:

cd midnight-indexer-spo-extension

# Copy env file
cp .env.example .env

# Edit .env with your configuration

# Start all services
docker compose --profile cloud up --build -d

# Check health
curl http://localhost:8090/ready

# Test GraphQL
curl -X POST http://localhost:8090/api/v1/graphql \
  -H "Content-Type: application/json" \
  -d '{"query":"{ serviceInfo { name version network } }"}'

---

Migration Checklist

For Explorer

• Update .env to use node-dev-01 URL: https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql
• Test with development environment (node-dev-01)
• Verify all queries work (block, transactions, contractAction)
• Check WebSocket subscriptions (blocks)
• Wait for announcement when testnet is upgraded with DUST features (no timeline yet)

For DUST DApp

• Switch from feat/cnight-generates-dust to main branch
• Update query field: isRegistered → registered
• Update endpoint: /api/v1/graphql → /api/v3/graphql
• Update frontend code handling the field
• Test API connectivity with node-dev-01 (returns empty data)
• Test query syntax and error handling
• Build frontend with mocked generation rates
• Continue using Blockfrost for real registration checks (currently ONLY option)
• Deploy contract to Cardano Preview - NOT possible yet (contracts not integrated)
• Perform test registration transactions - NOT possible yet (no contracts deployed)
• Verify complete flow: Cardano TX → node observation → indexer query - NOT possible yet
• Wait for announcement when PREVIEW/Testnet are ready with dependency chain complete

For SPO Extension

• No changes needed (independent service)
• Optional: Verify if querying midnight-indexer blocks subscription
• If yes, use /api/v3/graphql endpoint

---

Support and Questions

Documentation

• dustGenerationStatus API: GitHub Issue #439
• GraphQL Schema: midnight-indexer/indexer-api/graphql/schema-v3.graphql

Contact

• Technical Questions: Sean Kwak (@sean Kwak on Slack)
• Code Review: Heiko Seeberger (@heiko Seeberger on Slack)
• Project Coordination: Blythe Christopher (@Blythe Christopher on Slack)

Common Questions

Updated (Oct 22, 2025):

Q: When will dustGenerationStatus have real data?
A: When the dependency chain completes:

1. Casey's contracts: Off-chain code completed → Integrated into Node
2. Ledger update: ETA TBD (Adam Reynolds to confirm)
3. Node update: Deployed with new contract recognition
4. Indexer deployment: ~1 day after Node update
5. PREVIEW/Testnet: Configured and accessible

Currently all blockers have unclear timelines.

Q: Can I test the full CNGD integration now?
A: No. You cannot test real CNGD registrations on any environment yet because:

• No contracts deployed
• Node doesn't recognize Casey's contract structure
• Indexer CNGD tables are empty

You CAN test: API connectivity, query syntax, Blockfrost integration, frontend with mocked data.

Q: Why is Blockfrost necessary?
A: Blockfrost is currently the ONLY way to check real Cardano registration status. The indexer's dustGenerationStatus depends on:

• Cardano transactions → Node observation → Midnight events → Indexer tracking

This chain is broken until Casey's contracts are integrated and Node is updated.

Q: When can we stop using Blockfrost?
A: Once PREVIEW or Testnet is ready with the complete dependency chain. We'll announce when available. No timeline yet.

Q: Do we need to change our SPO extension code?
A: No changes needed - it's an independent service. If you're querying midnight-indexer's blocks subscription, use the v3 endpoint.

Q: What about the Oct 24 demo?
A: Use Blockfrost + mocked generation rates, as discussed. Full integration testing will come later when environments are ready.

---

Next Steps

Updated (Oct 22, 2025):

For Explorer

1. Test with node-dev-01 - Explorer functionality works

   • Use endpoint: https://indexer-rs.node-dev-01.dev.midnight.network/api/v3/graphql
   • Test block queries, transaction queries, WebSocket subscriptions
   • Explorer is NOT blocked by CNGD dependencies

2. Wait for Testnet announcement - Will be upgraded when ready

For DUST DApp

1. Update Field Names (Do Now)

   • Change isRegistered to registered
   • Update frontend code referencing the old field
   • Update endpoint to /api/v3/graphql

2. API Testing (Limited)

   • Test query syntax with node-dev-01 (returns empty data)
   • Test error handling for unregistered addresses
   • Build frontend with mocked data

3. Continue Blockfrost Integration

   • This is NOT optional - it's currently the ONLY way to check real registrations
   • Build complete registration flow with Blockfrost
   • Mock generation rates locally for display

4. MS5 Demo (Oct 24)

   • Demo with Blockfrost + mocked rates
   • Show UI/UX and user flows
   • Full integration testing - NOT possible yet

5. Wait for Dependency Chain

   • Cannot do real CNGD testing until:
     • Casey's contracts integrated
     • Ledger updated
     • Node updated and deployed
     • Indexer deployed
     • PREVIEW/Testnet ready
   • Timeline: Unclear - await announcements

For SPO Extension

1. Continue Independent Development
   • No changes needed from indexer side
   • If using blocks subscription, use /api/v3/graphql

Current Reality

• Explorer: ✅ Works on node-dev-01
• DUST DApp: ⏳ Use Blockfrost, await full integration
• SPO Extension: ✅ Independent deployment
• Full CNGD Testing: ❌ Not possible on any environment yet

---

Document Version: 2.0 (Major Update)
Original Date: October 13, 2025
Updated: October 22, 2025
Authors: Sean Kwak, Heiko Seeberger
Contact: Midnight Indexer Team

Changelog (v2.0):

• Added critical CNGD status update warning
• Updated node-dev-01 status: CNGD tables are empty
• Added PREVIEW environment information
• Clarified Blockfrost is necessary, not a workaround
• Updated testing limitations and capabilities
• Added dependency chain information
• Updated all Q&A and next steps sections
• Removed misleading claims about full integration testing

[cosmir17]

Indexer work is already done.
This page is just for documentation purpose.
Closing this ticket.