Conversation
Preparing to change the Value field from uint64 to *big.Int to fix the silent uint64 overflow that wraps transaction values > 18.44 ETH. Fixes #235
uint64 max is ~18.44 ETH in wei. Any transaction value above that silently wrapped around via Go's (*big.Int).Uint64() which returns only the low 64 bits. For example, a 33.07 ETH tx was stored as 14.62 ETH because 33066915805160912602 % 2^64 = 14620171731451360986. Using *big.Int preserves the full 256-bit Ethereum value. Fixes #235
parsedTx.Value() returns *big.Int. The old code called .Uint64() which silently discards bits above 64, causing wraparound for values > 2^64 wei. Now we copy the full big.Int value with no truncation. Fixes #235
Float32 has only ~7 significant digits of precision, which is wholly inadequate for wei values (up to 10^26+). Storing as string preserves the exact decimal representation of the big.Int value without any precision loss or overflow risk. Fixes #235
The old code did float32(transaction.Value) which: 1. Truncated the uint64 to float32 (~7 digits precision) 2. The uint64 was already wrong due to big.Int overflow Now we call .String() on the *big.Int to get the exact decimal representation, which ClickHouse can store losslessly as String. Fixes #235
… String ClickHouse Float (Float32) only has ~7 significant digits, causing precision loss for wei values. String preserves the exact decimal representation of any Ethereum value without overflow or truncation. Existing Float32 data will be auto-converted to its string representation by ClickHouse during the ALTER. Note: already-corrupted values from the uint64 overflow cannot be recovered by this migration alone — they need a separate data backfill. Fixes #235
…ion loss in transaction values Transaction values > 18.44 ETH (2^64 wei) were silently truncated by Go's (*big.Int).Uint64(), then further degraded by Float32 storage. Now stored as String via *big.Int for exact precision. Fixes #235
… drops HeadChan was unbuffered (pkg/events/standard.go:36), meaning the non-blocking send in pkg/events/head.go:40-45 would silently drop head events whenever the consumer in runHead() was busy processing a previous event. A buffer of 32 slots gives the consumer a full epoch of breathing room, preventing head event drops during processing spikes. This is the first half of the fix for impossible VRS records (f_reward > f_max_reward) documented in issue #238 — dropped epoch-boundary head events caused stale state roots to be used for reward calculation.
Previously, every head event overwrote the epoch boundary state root cache (lines 83-84), even mid-epoch events. When the actual last-slot head event was dropped (due to the unbuffered HeadChan fixed in the previous commit), a stale mid-epoch state root was used for the beacon state download. This caused withdrawal double-counting in reward calculation, producing f_reward > f_max_reward records. Now the state root is only cached when event.HeadEvent.Slot matches the last slot of the epoch, ensuring DownloadState always fetches the correct epoch-boundary state. This is the second half of the fix for issue #238, which produced 64 impossible VRS records across epochs 431671, 431724, 432097, and 432474.
Two-part fix for issue #238 where 64 impossible VRS records were produced across epochs 431671, 431724, 432097, and 432474: 1. Buffer HeadChan to 32 slots to prevent silent head event drops 2. Only cache state root at actual epoch boundary slot Verified locally: epoch 431671 reprocessed with 0 impossible rewards across 955,583 validators.
Collaborator
Author
Added: Fix for impossible VRS records (issue #238)Two commits merged to
Fixes #238 — verified locally: epoch 431671 reprocessed with 0 impossible rewards across 955,583 validators. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
f_valuefromFloattoStringRoot cause
parsedTx.Value().Uint64()inpkg/spec/transactions.go:130— Go's(*big.Int).Uint64()silently returns only the low 64 bits, wrapping values > 2^64 wei (~18.44 ETH)float32(transaction.Value)inpkg/db/transactions.go:77— casting to Float32 loses precision beyond ~7 significant digitsf_value Floatin the ClickHouse schema — Float32 is inadequate for wei amountsExample: tx
0x1b7b...22719(slot 13,587,965)Overflow math:
33066915805160912602 % 2^64 = 14620171731451360986 → Float32 → 14620172000000000000Evidence the bug is real
Changes
pkg/spec/transactions.go:Valuefield changed fromuint64to*big.Int; parsing usesnew(big.Int).Set()instead of.Uint64()pkg/db/transactions.go: Column type changed fromproto.ColFloat32toproto.ColStr; serialization uses.String()instead offloat32()castpkg/db/migrations/000035_alter_tx_value_string.{up,down}.sql: ALTER COLUMN migrationTest plan
0x1b7b...22719now stores 33,066,915,805,160,912,602 wei (matches EL node exactly)Closes #235