Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sync process hangs during the execution stage #7498

Closed
1 task done
vbuterin opened this issue Apr 7, 2024 · 3 comments
Closed
1 task done

Sync process hangs during the execution stage #7498

vbuterin opened this issue Apr 7, 2024 · 3 comments
Labels
A-execution Related to the Execution and EVM A-staged-sync Related to staged sync (pipelines and stages) C-bug An unexpected or incorrect behavior S-stale This issue/PR is stale and will close with no further activity

Comments

@vbuterin
Copy link

vbuterin commented Apr 7, 2024

Describe the bug

The sync process gets to the execution stage, and then hangs. This has only happened once so far, and restarting the client fixed it.

Steps to reproduce

Happened during re-syncing process after having fully synced, and then gone offline for 8 hours.

Node logs

2024-04-06T01:10:08.336112Z  INFO Status connected_peers=15 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:10:33.336392Z  INFO Status connected_peers=13 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:10:58.336760Z  INFO Status connected_peers=14 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:11:23.336882Z  INFO Status connected_peers=17 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:11:48.336579Z  INFO Status connected_peers=17 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:12:13.335856Z  INFO Status connected_peers=20 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:12:38.335677Z  INFO Status connected_peers=18 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
multip2024-04-06T01:13:03.337230Z  INFO Status connected_peers=19 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:13:28.336519Z  INFO Status connected_peers=17 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:13:53.336453Z  INFO Status connected_peers=18 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:13:58.531846Z  INFO Received forkchoice updated message when syncing head_block_hash=0x84d304eacae026a107f8b62ce953b5bb9f9ad68e4e3797f70af85164f721f20f safe_block_hash=0x2dbc0676696656f4fbc92c1db6a840931f8ca240cc68ca453083f2956fb5ecbf finalized_block_hash=0xa7c9c6fdbed614c82295612dea7627baa3a9d9aa4f7c864650e820955f684912
2024-04-06T01:14:18.336391Z  INFO Status connected_peers=18 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:14:43.336050Z  INFO Status connected_peers=16 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:15:08.336612Z  INFO Status connected_peers=15 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:15:33.336814Z  INFO Status connected_peers=16 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:15:58.336864Z  INFO Status connected_peers=18 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:16:23.336437Z  INFO Status connected_peers=16 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:16:48.337436Z  INFO Status connected_peers=17 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:17:13.336170Z  INFO Status connected_peers=17 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:17:38.336423Z  INFO Status connected_peers=17 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:18:03.336597Z  INFO Status connected_peers=16 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:18:28.337291Z  INFO Status connected_peers=17 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:18:53.336472Z  INFO Status connected_peers=19 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:19:18.336502Z  INFO Status connected_peers=16 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:19:43.336384Z  INFO Status connected_peers=17 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:20:08.336178Z  INFO Status connected_peers=20 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:20:22.528953Z  INFO Received forkchoice updated message when syncing head_block_hash=0xe79442f46ef1eb497bd8e3fca8a35a64478396b79ed0c39d62996f0d74407083 safe_block_hash=0x9980e5d12dc6875de2f167726f10403bea8d265e8d15aa022da16f0ef275cf1a finalized_block_hash=0x2dbc0676696656f4fbc92c1db6a840931f8ca240cc68ca453083f2956fb5ecbf
2024-04-06T01:20:33.336361Z  INFO Status connected_peers=21 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:20:58.336452Z  INFO Status connected_peers=19 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:21:23.336176Z  INFO Status connected_peers=18 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:21:48.337014Z  INFO Status connected_peers=18 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:22:13.336540Z  INFO Status connected_peers=18 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:22:38.336164Z  INFO Status connected_peers=17 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:23:03.336756Z  INFO Status connected_peers=16 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:23:28.336525Z  INFO Status connected_peers=20 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:23:53.336429Z  INFO Status connected_peers=20 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:24:18.336374Z  INFO Status connected_peers=20 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:24:43.336180Z  INFO Status connected_peers=19 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:25:08.336788Z  INFO Status connected_peers=20 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:25:33.336277Z  INFO Status connected_peers=25 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:25:58.335834Z  INFO Status connected_peers=22 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:26:23.336867Z  INFO Status connected_peers=22 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:26:46.537151Z  INFO Received forkchoice updated message when syncing head_block_hash=0x0570b32c107d9084aa09522b8011425fb30fc48294492281a4bd3d8339283e59 safe_block_hash=0xbdc7ddee1091bf312a074905a3188e4431a60004aa95d42d13ca7a22bdebecdb finalized_block_hash=0x9980e5d12dc6875de2f167726f10403bea8d265e8d15aa022da16f0ef275cf1a
2024-04-06T01:26:48.336842Z  INFO Status connected_peers=22 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:27:13.336125Z  INFO Status connected_peers=23 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:27:38.336787Z  INFO Status connected_peers=23 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:28:03.335703Z  INFO Status connected_peers=24 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:28:28.336124Z  INFO Status connected_peers=24 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160
2024-04-06T01:28:53.336940Z  INFO Status connected_peers=21 freelist=3636443 stage=Execution checkpoint=19590133 target=19590160

Platform(s)

Linux (x86)

What version/commit are you on?

reth Version: 0.2.0-beta.4
Commit SHA: bfadc26
Build Timestamp: 2024-03-31T03:11:41.269410894Z
Build Features: jemalloc
Build Profile: release

What database version are you on?

Current database version: 2
Local database version: 2

What type of node are you running?

Archive (default)

What prune config do you use, if any?

No response

If you've built Reth from source, provide the full command you used

No response

Code of Conduct

  • I agree to follow the Code of Conduct
@vbuterin vbuterin added C-bug An unexpected or incorrect behavior S-needs-triage This issue needs to be labelled labels Apr 7, 2024
@curtisc
Copy link

curtisc commented Apr 8, 2024 •
edited
Loading

I've encountered this same issue twice on initial sync using the beta.5 release. My build profile was maxperf with jemalloc and asm-keccak build features. Same thing with mine where restarting the process gets it unstuck and it continues onwards with sync.

@emhane emhane added A-staged-sync Related to staged sync (pipelines and stages) A-execution Related to the Execution and EVM and removed S-needs-triage This issue needs to be labelled labels Apr 9, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 1, 2024

This issue is stale because it has been open for 21 days with no activity.

@github-actions github-actions bot added the S-stale This issue/PR is stale and will close with no further activity label May 1, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 9, 2024

This issue was closed because it has been inactive for 7 days since being marked as stale.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-execution Related to the Execution and EVM A-staged-sync Related to staged sync (pipelines and stages) C-bug An unexpected or incorrect behavior S-stale This issue/PR is stale and will close with no further activity
Projects
Reth Tracker
Archived in project
Milestone
No milestone
Development

No branches or pull requests
3 participants
@curtisc @vbuterin @emhane