Skip to content

fix(node-health): total_miners sums replica counts and inflates the figure Nx - #8008

Open
Vyacheslav-Tomashevskiy wants to merge 1 commit into
Scottcjn:mainfrom
Vyacheslav-Tomashevskiy:fix/node-health-total-miners
Open

fix(node-health): total_miners sums replica counts and inflates the figure Nx#8008
Vyacheslav-Tomashevskiy wants to merge 1 commit into
Scottcjn:mainfrom
Vyacheslav-Tomashevskiy:fix/node-health-total-miners

Conversation

@Vyacheslav-Tomashevskiy

Copy link
Copy Markdown
Contributor

The bug

NodeHealthMonitor.get_network_health reports the headline Total miners figure by summing each online node's count:

online = [s for s in statuses if s.status != "offline"]
total_miners = sum(s.miners or 0 for s in online)   # <-- inflates Nx

But the monitored nodes (DEFAULT_NODES, three :8088/:8099 hosts) are replicas of one shared ledger. Each /status returns the network-wide active-miner count (miners / active_miners / miner_count), not a disjoint per-node slice. The sibling tools in this repo prove the intent — they treat any divergence between nodes as an anomaly:

  • monitoring/ledger_verify.pyminer_count_match / MINER COUNT MISMATCH
  • tools/node_sync_validator.pyminer_count_mismatch

So in the healthy case every online node agrees, and summing multiplies the true count by the number of online nodes.

Concrete input → wrong output: 3 online nodes each reporting miners = 150 (a network with 150 miners) → buggy total_miners = 150 + 150 + 150 = 450; correct = 150. The operator's table (print_table) and --json output always show a fabricated 2-3x miner population.

The fix

Take the value the replicas agree on instead of summing. max is used so a node that is mid-sync and momentarily reporting fewer miners doesn't drag the figure down:

miner_counts = [s.miners for s in online if s.miners is not None]
total_miners = max(miner_counts) if miner_counts else 0

Test

Added tools/test_node_health_total_miners.py: 3 replicas → 150 (not 450); one lagging node (149) → 150; one offline → 150 with nodes_online=2; no miner data → 0 (no crash). All pass.

Verified against upstream/main @ c6501de.

RTC: RTCd1554f0f35576faf01d386a6be1c947f560dd0b7

…ure Nx

The monitored attestation nodes are replicas of one shared ledger, so each
/status reports the SAME network-wide miner count (ledger_verify and
node_sync_validator both flag any divergence as a mismatch). get_network_health
summed the per-node counts, so 3 online nodes each reporting 150 produced
total_miners=450. Take the agreed value instead (max, robust to a mid-sync node
reporting fewer). Adds a regression test.
@github-actions github-actions Bot added BCOS-L1 Beacon Certified Open Source tier BCOS-L1 (required for non-doc PRs) size/M PR: 51-200 lines labels Jul 18, 2026
@Scottcjn

Copy link
Copy Markdown
Owner

Right diagnosis (replicas of one ledger shouldn't be summed). Held pending two fixes: (1) CI is red because the pre-existing test_health_monitor.py::test_all_healthy still asserts the old summed value (6 != 13) — reconcile it. (2) max(miner_counts) silently trusts the highest divergent report, so a stale/over-reporting node inflates the count; detect/handle divergence rather than always taking max. Address both and it pays. — Sophia

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

BCOS-L1 Beacon Certified Open Source tier BCOS-L1 (required for non-doc PRs) size/M PR: 51-200 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants