Skip to content

Midnight Indexer 4.3.2

Choose a tag to compare

@cosmir17 cosmir17 released this 15 May 19:14
· 36 commits to main since this release
v4.3.2
7b6d40e

https://github.com/midnightntwrk/midnight-indexer/releases/tag/v4.3.2

Midnight Indexer 4.3.2

Midnight Indexer 4.3.2 Release Notes

Release date: 2026-05-15
Release type: Bundle component — Midnight 1.1 (patch release on the development line)
Git tag: v4.3.2
Tree hash: 9b2de1658b375e22323e6420eef372c0bf924aec
Environment: All public networks (mainnet, preprod, preview, devnet, qanet)

Note: This release pairs with node 1.0.0 and ships as part of the Midnight 1.1 bundle. Operators on node 0.22.x should track the 4.0.x maintenance line, not this release.

⚠️ Requires an indexer reset when upgrading from the 4.0.x line to 4.3.2

The 4.3.x line pairs with node 1.0.0 / ledger 8.1 and carries accumulated schema and fee-computation changes since the 4.0.x maintenance line (node 0.22.x). Index state built on 4.0.x cannot be reused — reset the indexer and re-index from genesis on upgrade. Budget ~3–4 days at mainnet/preprod scale. Upgrades within the 4.3.x line (4.3.0 → 4.3.1 → 4.3.2) are drop-in and require no reset.

Summary

Small patch release on the development line. Adds a /live HTTP liveness endpoint to indexer-api, allowing kubernetes to detect a stuck process the existing file-based liveness check could not. Fixes a dustGenerations subscription bug where the termination signal could miss in-flight wallet entries, replacing the per-wallet cursor check with chain progress and adding a re-drain to close the race window.

Dependencies

Unchanged from 4.3.1:

  • Paired ledger: 8.1
  • Paired node: 1.0.0 (supports 0.22.x as well)
  • Bundle tracking issue: Midnight Release 1.1
  • Sibling release: indexer 4.0.x maintenance line continues for operators on node 0.22.x (latest: 4.0.2)

Docker Images

Five images publish at this tag (per .github/workflows/build-indexer-images.yaml at v4.3.2):

  • midnightntwrk/chain-indexer:4.3.2
  • midnightntwrk/indexer-api:4.3.2
  • midnightntwrk/wallet-indexer:4.3.2
  • midnightntwrk/indexer-standalone:4.3.2
  • midnightntwrk/spo-indexer:4.3.2

Audience

These release notes are intended for:

  • Operators running the Midnight Indexer against node 1.0+
  • Wallet integrators consuming the dustGenerations subscription, especially clients passing an endIndex that does not coincide with a wallet-owned generation index
  • SREs configuring kubernetes liveness probes on indexer-api
  • QA and release managers tracking the Midnight 1.1 bundle

(Operators on node 0.22.x should track the 4.0.x maintenance line, not this release.)

What Changed

Change Type PR
/live HTTP endpoint for kubernetes liveness probe New feature #1145
dustGenerations subscription terminates on chain progress, not per-wallet cursor (with race-window re-drain) Bug fix #1137

New Features

/live HTTP endpoint for kubernetes liveness probe (#1145)

Adds an HTTP /live endpoint to indexer-api alongside the existing /ready endpoint. SREs can configure this as the kubernetes liveness probe, replacing the previous file-existence check (cat /var/run/indexer-api/running) which could not detect a stuck HTTP server. With /live, kubelet auto-recovers indexer-api pods that have entered an unresponsive state by triggering a restart after the probe failure threshold.

Operator note: this is opt-in. To use it, update the deployment manifest's livenessProbe to an httpGet against /live on the api port. The file-existence probe continues to work for backward compatibility.

Breaking Changes or Required Actions

None. Drop-in upgrade for operators already on the indexer 4.x.y development line.

Known Issues

Runtime-park hang under heavy concurrent merkle-tree-update load (midnight-ledger#521)

Under heavy concurrent merkle-tree-update load, Sp::force_as_arc (called via BackendLoader::get from make_dust_commitments_collapsed_update, make_dust_generations_collapsed_update, and related collapsed-update paths) can park the entire tokio runtime indefinitely. The pod produces zero log output (no errors, no panics, no background heartbeats) until kubelet's /live probe fails and the container is SIGTERM'd.

Per @tkerber on the ledger ticket:

Note: We are moving to a mainnet release with this as a known defect. This is likely to have the following effects:

  • Indexer instances will be forcibly restarted when a deadlock/stall is detected
  • Such stalls are increasingly likely to occur under heavy load
  • An indexer restart may cause disconnection in connected wallet instances, which may lead to stalls in syncing until reconnected. This has not been fully tested.

We are moving ahead with a release without fixing this because:

  • A prior fix attempt midnight-ledger#499 did not resolve this issue
  • Evidence pinpointing the cause is very weak, and we cannot definitively attribute it.
  • Restart/reconnect was deemed an acceptable mitigation while we work to find the underlying cause.

The /live HTTP endpoint shipped in this release is the detection half of that mitigation; the kubernetes restart path closes the loop. Root cause investigation ongoing at midnight-ledger#521.


Operators tracking other open issues in the indexer should consult the priority:critical and priority:high labels in the indexer GitHub issues.

Fixed Defects

Defect Description
#1137 dustGenerations subscription hung in live-tail when the user-supplied endIndex did not coincide with a wallet-owned generation index. Termination signal moved from the per-wallet cursor (cursor > end_index strict) to chain progress (chain_first_free > end_index), aligning with the canonical pattern used by dustNullifierTransactions and shieldedNullifierTransactions. A re-drain step before yielding Progress closes the small race window between the storage drain ending and the chain-progress read. Wallets passing the chain's current dustGenerationEndIndex as endIndex now terminate cleanly.

Links and References

(Per-PR links are embedded in the What Changed and Fixed Defects tables above.)