Conversation
- Fixes negative rewards for post-Electra validators with >32 ETH - ConsolidatedAmounts should be fetched from NextState, not CurrentState - Resolves GitHub issue #215
- Prevents UInt64 underflow when aggregating negative rewards - Allows proper handling of negative validator rewards - Resolves GitHub issue #214
Required for math.MaxUint64 to check for uint64 overflow before accumulating transaction fees. Part of fix for issue #216.
- Check for underflow when GasPrice < baseFeePerGas - Skip transactions with priority fee > 1e21 (>1000 ETH, likely corrupted) - Check for uint64 overflow before accumulating fees - Add logging for debugging suspicious values Fixes #216
Go 1.25.6 stricter type checking revealed 1e21 exceeds uint64 max (2^64-1). Changed to 1e19 Wei (~10 ETH) as practical sanity cap for tx fees. Original 1e21 (1000 ETH) was impossible to represent in uint64 anyway.
The previous fix (eba4883) incorrectly used NextState.ConsolidatedAmounts, which contains consolidations for the NEXT epoch (not yet applied to balances). This fix: - Reverts EpochReward to use CurrentState.ConsolidatedAmounts - Adds processPendingConsolidations(CurrentState) in PreProcessBundle - CurrentState consolidations = those processed at START of current epoch, which already affected NextState.Balances Tested locally: Validator 2170445 epoch 424365 now shows correct reward (~9k gwei) instead of corrupt -64 ETH.
The previous fix (ab95a42) incorrectly assumed CurrentState.ConsolidatedAmounts would contain the consolidations affecting the current epoch's reward. However, state objects are reused between iterations, causing processPendingConsolidations to accumulate values across epochs. Changes: - Clear CurrentState.ConsolidatedAmounts at start of PreProcessBundle - Add processConsolidationsForRewardCalculation() to identify consolidations processed at the START of NextState's epoch by comparing pending lists - Only subtract consolidations that actually affected NextState.Balances This fixes validators receiving consolidations showing negative rewards.
- Add processDepositsForRewardCalculation() to identify external deposits processed at the START of NextState's epoch - Skip internal deposits (slot = 0) created by queue_excess_active_balance as these are balance restructuring, not new funds - Clear DepositedAmounts at start of PreProcessBundle (state reuse fix) - Add bounds check for NextState.Deposits array access Note: Validator 1111619 issue (negative reward during switch_to_compounding) is a deeper timing issue with how goteth reads beacon state balances, not a reward calculation bug. The beacon node returns post-epoch-processing balances instead of pre-processing balances.
Store a lightweight mapping slot -> stateRoot for epoch-boundary slots, populated from Head SSE events. This avoids the racy slot-based state resolution in Lighthouse v8.1.0+. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add setEpochBoundaryStateRoot() and takeEpochBoundaryStateRoot() to isolate the caching mechanics for epoch-boundary state roots from Head SSE events. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR merges all recent
devchanges intomaster, addressing two main areas:Lighthouse v8.1.0 SSE timing fix
Lighthouse v8.1.0 (commit cd8049a69) emits the
HeadSSE event before the block and state are fully committed. This causes GotEth to fetch stale data when following the chain head, resulting inf_proposed=false,f_timestamp=0, andf_attestations=0for affected epochs.Changes:
8e21b77,f5db50c,0e35d96,ecaa493,8a7ba8e,5e995b9)66d1730)Validated on a test machine running Lighthouse v8.1.0 + GotEth against mainnet: all blocks correctly show
f_proposed=truewith properf_timestampandf_attestationsvalues after applying the fix.Reward calculation and overflow fixes
These commits were merged to
devprior to the Lighthouse fix and address several issues found in reward/fee calculations:eba4883— fix: use NextState for consolidated amounts in EpochReward calculationConsolidated validator balances were being read from the current state instead of the next state. Since consolidations take effect at the start of the next epoch, the amounts must be sourced from
NextStateto reflect the actual post-consolidation balances.d74d19b— fix: change aggregated_rewards from UInt64 to Int64 in t_pool_summaryAggregated rewards can be negative (e.g. penalties exceeding rewards in a given epoch). The column type was UInt64 which caused underflow wrapping negative sums to huge values near
2^64.63bb131— chore: add math import for overflow protection in BlockGasFeesAdds the
mathpackage import required by the overflow protection logic.51f6713— fix: add overflow protection in BlockGasFees calculationLarge gas prices or unusual fee values could cause uint64 overflow when computing
(gasPrice - baseFee) * gasUsed, corrupting EL fee data. Added validation that gasPrice >= baseFee, a cap on maximum reasonable transaction fee, and overflow checks before accumulating totals.563ad4c— fix: adjust maxReasonableTxFee to fit uint64 rangeThe initial
maxReasonableTxFeeconstant exceeded uint64 range. Adjusted to ~184 ETH which still catches anomalous fees while fitting within bounds.ab95a42— fix: correct consolidation timing in EpochReward calculationConsolidations are processed at the start of the epoch, not at the end. The reward calculation was applying consolidation adjustments at the wrong point, causing incorrect balance diffs for validators involved in consolidations.
1aa0a77— fix: prevent double-counting consolidations in reward calculationWhen a source validator consolidates into a target, both the source's balance decrease and the target's balance increase were being counted separately. This fix ensures consolidation transfers are only accounted for once.
5461500— fix: add deposit tracking for reward calculation and fix bounds checkExternal deposits were not being subtracted from the balance diff, causing deposited ETH to appear as earned rewards. Added
processDepositsForRewardCalculation()to filter external deposits. Also fixed an array bounds check.Closes
Closes Lighthouse v8.1.0
HeadSSE timing can cause GotEth to fetch inconsistent epoch-boundary states (incorrectf_proposed,f_timestamp,f_attestations) #220 — Lighthouse v8.1.0 Head SSE timing can cause GotEth to fetch inconsistent epoch-boundary statesFixed by caching the state root from Head SSE events and using root-based state lookup with retry (
8e21b77,f5db50c,0e35d96,ecaa493,8a7ba8e,5e995b9), plus retrying block downloads on 404 (66d1730).Closes f_reward Inflated by Consolidated Amounts (Post-Electra Validators) #215 — f_reward Inflated by Consolidated Amounts (Post-Electra Validators)
Fixed by correcting consolidation timing (
ab95a42), preventing double-counting (1aa0a77), and using NextState for consolidated amounts (eba4883).Closes UInt64 Underflow in t_pool_summary.aggregated_rewards #214 — UInt64 Underflow in t_pool_summary.aggregated_rewards
Fixed by changing
aggregated_rewardsfrom UInt64 to Int64 (d74d19b), allowing negative reward sums without underflow.Closes Dashboards missing Execution Layer rewards and APR #212 — Dashboards missing Execution Layer rewards and APR
Fixed by adding overflow protection in BlockGasFees (
51f6713,563ad4c) which was producing corrupted EL fee data, and adding deposit tracking (5461500) so reward calculations correctly separate deposits from earned rewards.Files changed
pkg/analyzer/chain_analyzer.goepochBoundaryStateRootssync.Mappkg/analyzer/epoch_boundary_state_root.gopkg/analyzer/download.gopkg/analyzer/routines.gopkg/clientapi/block.gopkg/clientapi/state.goRequestBeaconStateByRootpkg/spec/block.gopkg/spec/metrics/standard.gopkg/spec/metrics/state_electra.gopkg/db/migrations/000001_init_schema.up.sqlTesting
f_proposed=falsef_proposed=truewith valid metricsBreaking Changes
None. Internal behavior change only.