Skip to content

Add block reexecution support for pathdb#4528

Closed
KolbyML wants to merge 3 commits intomasterfrom
kolby-nit-4548-block-reexecution-pathdb-support-2
Closed

Add block reexecution support for pathdb#4528
KolbyML wants to merge 3 commits intomasterfrom
kolby-nit-4548-block-reexecution-pathdb-support-2

Conversation

@KolbyML
Copy link
Copy Markdown
Member

@KolbyML KolbyML commented Mar 19, 2026

fixes NIT-4548

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 19, 2026

Codecov Report

❌ Patch coverage is 0% with 72 lines in your changes missing coverage. Please review.
✅ Project coverage is 31.86%. Comparing base (b9d1452) to head (9d06f40).
⚠️ Report is 121 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4528      +/-   ##
==========================================
- Coverage   32.67%   31.86%   -0.81%     
==========================================
  Files         497      497              
  Lines       58849    58899      +50     
==========================================
- Hits        19227    18767     -460     
- Misses      36245    36746     +501     
- Partials     3377     3386       +9     

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 19, 2026

❌ 22 Tests Failed:

Tests completed Failed Passed Skipped
4532 22 4510 0
View the top 3 failed tests by shortest run time
TestEndToEnd_TwoEvilValidators
Stack Traces | -0.000s run time
... [CONTENT TRUNCATED: Keeping last 20 lines]
created by github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe in goroutine 80960
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:162 +0x125

goroutine 81560 [select]:
github.com/offchainlabs/nitro/bold/assertions.(*Manager).updateLatestConfirmedMetrics(0xc002af9680, {0x1fe6500, 0xc005d64050})
	/home/runner/work/nitro/nitro/bold/assertions/confirmation.go:134 +0x146
github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe.func1()
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:163 +0x38
created by github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe in goroutine 80963
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:162 +0x125

goroutine 1874625 [select]:
github.com/offchainlabs/nitro/bold/assertions.(*Manager).keepTryingAssertionConfirmation(0xc001fa8f00, {0x1fe6500, 0xc000301e50}, {{0xc5, 0x59, 0xd8, 0x9d, 0x67, 0x90, 0xaf, ...}})
	/home/runner/work/nitro/nitro/bold/assertions/confirmation.go:71 +0x7b8
github.com/offchainlabs/nitro/bold/assertions.(*Manager).queueCanonicalAssertionsForConfirmation.func1({0x1fe6500?, 0xc000301e50?})
	/home/runner/work/nitro/nitro/bold/assertions/confirmation.go:29 +0x4c
github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe.func1()
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:163 +0x38
created by github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe in goroutine 81071
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:162 +0x125
TestEndToEnd_TwoEvilValidators/honest_essential_edges_confirmed_by_challenge_win
Stack Traces | -0.000s run time
... [CONTENT TRUNCATED: Keeping last 20 lines]
created by github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe in goroutine 80960
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:162 +0x125

goroutine 81560 [select]:
github.com/offchainlabs/nitro/bold/assertions.(*Manager).updateLatestConfirmedMetrics(0xc002af9680, {0x1fe6500, 0xc005d64050})
	/home/runner/work/nitro/nitro/bold/assertions/confirmation.go:134 +0x146
github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe.func1()
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:163 +0x38
created by github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe in goroutine 80963
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:162 +0x125

goroutine 1874625 [select]:
github.com/offchainlabs/nitro/bold/assertions.(*Manager).keepTryingAssertionConfirmation(0xc001fa8f00, {0x1fe6500, 0xc000301e50}, {{0xc5, 0x59, 0xd8, 0x9d, 0x67, 0x90, 0xaf, ...}})
	/home/runner/work/nitro/nitro/bold/assertions/confirmation.go:71 +0x7b8
github.com/offchainlabs/nitro/bold/assertions.(*Manager).queueCanonicalAssertionsForConfirmation.func1({0x1fe6500?, 0xc000301e50?})
	/home/runner/work/nitro/nitro/bold/assertions/confirmation.go:29 +0x4c
github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe.func1()
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:163 +0x38
created by github.com/offchainlabs/nitro/util/stopwaiter.(*StopWaiterSafe).LaunchThreadSafe in goroutine 81071
	/home/runner/work/nitro/nitro/util/stopwaiter/stopwaiter.go:162 +0x125
TestPruningDBSizeReduction
Stack Traces | 0.000s run time
=== RUN   TestPruningDBSizeReduction
--- FAIL: TestPruningDBSizeReduction (0.00s)

📣 Thoughts on this report? Let Codecov know! | Powered by Codecov

@KolbyML KolbyML force-pushed the kolby-nit-4548-block-reexecution-pathdb-support-2 branch from 1d2fd45 to 8397886 Compare March 19, 2026 12:29
@KolbyML KolbyML marked this pull request as ready for review March 19, 2026 13:48
Comment on lines +281 to +283
if start < requestedStart {
log.Info("BlocksReExecutor fell back to an earlier available anchor state", "requestedAnchor", requestedStart, "stateAt", start, "endBlock", currentBlock)
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice addition


func (s *BlocksReExecutor) dereferenceRoot(root common.Hash) {
if s.db == nil {
return
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should log about it (?)

Comment on lines +346 to +347
value := reflect.ValueOf(executor).Elem()
room := value.FieldByName("room").Int()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we could add some getter?

f.String(prefix+".mode", DefaultConfig.Mode, "mode to run the blocks-reexecutor on. Valid modes full and random. full - execute all the blocks in the given range. random - execute a random sample range of blocks with in a given range")
f.String(prefix+".blocks", DefaultConfig.Blocks, "json encoded list of block ranges in the form of start and end block numbers in a list of size 2")
f.Bool(prefix+".commit-state-to-disk", DefaultConfig.CommitStateToDisk, "if set, blocks-reexecutor not only re-executes blocks but it also commits their state to triedb")
f.Int(prefix+".room", DefaultConfig.Room, "number of threads to parallelize blocks re-execution")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we could add a note that this doesn't apply for pathdb?

Copy link
Copy Markdown
Contributor

@bragaigor bragaigor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines +209 to +213
if err == nil {
_ = blocksReExecutor.db.TrieDB().Reference(header.Root, common.Hash{}) // Will be dereferenced later in advanceStateUpToBlock
return sdb, func() { blocksReExecutor.dereferenceRoot(header.Root) }, nil
}
return sdb, arbitrum.NoopStateRelease, err
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: I'm so used to read err != nil that took me a while to notice err == nil so wondering if it would improve code readability and consistency with other parts to reverse and use err != nil?

if header == nil {
return nil, arbitrum.NoopStateRelease, errors.New("start header not found")
}
statedb, err := blockchain.StateAt(header.Root)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also add fallback to bc.HistoricState as in StateAndHeaderFromHeader:

	if bc.TrieDB().Scheme() == rawdb.PathScheme {
		statedb, err := bc.StateAt(header.Root)
		if err != nil {
			statedb, err = bc.HistoricState(header.Root)
			if err != nil {
				return nil, nil, err
			}
		}
		return statedb, header, nil
	}

https://github.com/OffchainLabs/go-ethereum/blob/2af9c1f710c0d637b1cd902b4056e24853c42dc5/arbitrum/apibackend.go#L560-L569

chainEnd := blockchain.CurrentBlock().Number.Uint64()
minBlocksPerThread := uint64(10000)
room := c.Room
if scheme == rawdb.PathScheme && room > 1 {
Copy link
Copy Markdown
Contributor

@magicxyyz magicxyyz Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why we need to limit re-execution to single thread?
concurrent historical state access is already supported, eg. eth_call RPC calls are executed concurrently.

Did you get any errors when trying to run it in parallel?

@KolbyML
Copy link
Copy Markdown
Member Author

KolbyML commented Mar 24, 2026

Closing this

@KolbyML KolbyML closed this Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants