Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Beacon Chain peers disconnect in block 35848 - using bazel #4621

Closed
Gsuslu opened this issue Jan 22, 2020 · 2 comments · Fixed by #4637
Closed

Beacon Chain peers disconnect in block 35848 - using bazel #4621

Gsuslu opened this issue Jan 22, 2020 · 2 comments · Fixed by #4637
Labels
Bug Something isn't working

Comments

@Gsuslu
Copy link

Gsuslu commented Jan 22, 2020

I did it already 3 times. The same block ... all peers disconnect.

[2020-01-22 22:38:25] INFO initial-sync: Processing block 35848/92892 - estimated time remaining 105h38m13s blocksPerSecond=0.1 peers=5/13
[2020-01-22 22:38:30] INFO p2p: Peer disconnected active=12 peer=16Uiu2HAm7V6p2HgCx3JhmxRBuiXmFL6QTjbHssUiUs7cXUe18Bkd
[2020-01-22 22:38:30] INFO p2p: Peer disconnected active=11 peer=16Uiu2HAmRMqPe1nR1yAwUEpBKzUh8ABHzcMBSRnnK1z4fkuARu8e
[2020-01-22 22:38:30] INFO p2p: Peer disconnected active=10 peer=16Uiu2HAmLpabFfbMpMyZhqEevu2hCfCuiRyc2gSWaRUVP9ozSA8j
[2020-01-22 22:38:30] INFO p2p: Peer disconnected active=9 peer=16Uiu2HAm12JGMpCDPnp3MzoJ2qAHkqESBSM9sU7Pf2ukeiQZ7KnF
[2020-01-22 22:38:30] INFO p2p: Peer disconnected active=8 peer=16Uiu2HAkz1t6hf1xcyknqASiGxWwQHqb6griqnMu5Wt2MC6yhhK9
[2020-01-22 22:38:35] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-22 22:38:35] INFO p2p: Peer disconnected active=7 peer=16Uiu2HAmEJB957TtRMKgAYYQjDomDbLt1hLJbk2jVqNYjHrmfJys
[2020-01-22 22:38:36] INFO p2p: Peer disconnected active=6 peer=16Uiu2HAmCVsokwnprH7QJyCMh9FuSVnQwV9a4hQ5RjzmZnbNyuYv
[2020-01-22 22:38:42] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-22 22:38:43] INFO p2p: Peer disconnected active=5 peer=16Uiu2HAm31CrTnSqozW2o1ovsmZ6wxRXioYBpRSLvQGyq8QtfWRP
[2020-01-22 22:38:43] INFO p2p: Peer disconnected active=4 peer=16Uiu2HAmAsyW714qpiYz9dNEoDyLa29SuzXwU4sY3fQrRtUxN1bT
[2020-01-22 22:38:43] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-22 22:38:45] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-22 22:38:45] INFO p2p: Peer disconnected active=3 peer=16Uiu2HAmJNZnTxJ2U3aHr8gzLb4xZnHmVXUPTNkQQMuFcP9RPg11
[2020-01-22 22:38:46] INFO p2p: Peer disconnected active=2 peer=16Uiu2HAm7XcCk3Z93SftzYVmGJtcJ4c2QVnokroJae5dgk3B23EM
[2020-01-22 22:38:46] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-22 22:38:53] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-22 22:38:53] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-22 22:39:01] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-22 22:39:01] WARN initial-sync: No peers; waiting for reconnect
[2020-01-22 22:39:01] INFO p2p: Peer disconnected active=1 peer=16Uiu2HAm7ruW6N9xyeJMqZwUX4wUfvMUhKRgGQadLYxaPwubVPr5
[2020-01-22 22:39:07] WARN initial-sync: No peers; waiting for reconnect
[2020-01-22 22:39:13] WARN initial-sync: No peers; waiting for reconnect
[2020-01-22 22:39:19] WARN initial-sync: No peers; waiting for reconnect
@Gsuslu Gsuslu changed the title Beacon Chain peers disconnect in block 35848 Beacon Chain peers disconnect in block 35848 - using bazel Jan 22, 2020
@Gsuslu
Copy link
Author

Gsuslu commented Jan 23, 2020

I left it all the night searching for new peers. In the final it jumped from block 35848 till block 39000.

[2020-01-23 02:14:07] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:14:13] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:14:19] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:14:25] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:14:31] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:14:37] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:14:37] INFO p2p: Peer connected active=1 peer=16Uiu2HAm31CrTnSqozW2o1ovsmZ6wxRXioYBpRSLvQGyq8QtfWRP
[2020-01-23 02:14:43] INFO initial-sync: Processing block 39355/93973 - estimated time remaining 303h26m0s blocksPerSecond=0.1 peers=1/1
[2020-01-23 02:17:44] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-23 02:17:45] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-01-23 02:18:04] INFO p2p: Peer connected active=2 peer=16Uiu2HAmG976ED4CDgvGRHJUpNKuaZjEPmkPv38cKtXXoCQcYySE
[2020-01-23 02:18:04] INFO p2p: Peer disconnected active=1 peer=16Uiu2HAmG976ED4CDgvGRHJUpNKuaZjEPmkPv38cKtXXoCQcYySE
[2020-01-23 02:18:48] INFO p2p: Peer connected active=2 peer=16Uiu2HAmLpabFfbMpMyZhqEevu2hCfCuiRyc2gSWaRUVP9ozSA8j
[2020-01-23 02:22:06] INFO p2p: Peer connected active=3 peer=16Uiu2HAmGUGV4fi3QEusCAeH9QnHyFyRSgmbWzFE6pkC89174Fv7
[2020-01-23 02:22:14] INFO p2p: Peer connected active=4 peer=16Uiu2HAmG976ED4CDgvGRHJUpNKuaZjEPmkPv38cKtXXoCQcYySE
[2020-01-23 02:22:14] INFO p2p: Peer disconnected active=3 peer=16Uiu2HAmG976ED4CDgvGRHJUpNKuaZjEPmkPv38cKtXXoCQcYySE
[2020-01-23 02:23:03] INFO p2p: Peer connected active=4 peer=16Uiu2HAmEJB957TtRMKgAYYQjDomDbLt1hLJbk2jVqNYjHrmfJys
[2020-01-23 02:24:39] INFO p2p: Peer connected active=6 peer=16Uiu2HAmM5zsvg5zuGdrqjmJ2VT1qMFJAsg9iDJsKDWXA3zyRYMQ
[2020-01-23 02:24:42] INFO p2p: Peer disconnected active=5 peer=16Uiu2HAmEJB957TtRMKgAYYQjDomDbLt1hLJbk2jVqNYjHrmfJys
[2020-01-23 02:27:07] INFO initial-sync: Processing block 39395/94035 - estimated time remaining 303h33m20s blocksPerSecond=0.1 peers=1/4
[2020-01-23 02:27:19] INFO p2p: Peer disconnected active=4 peer=16Uiu2HAmLpabFfbMpMyZhqEevu2hCfCuiRyc2gSWaRUVP9ozSA8j
[2020-01-23 02:27:19] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-23 02:27:19] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:27:19] INFO p2p: Peer disconnected active=3 peer=16Uiu2HAm31CrTnSqozW2o1ovsmZ6wxRXioYBpRSLvQGyq8QtfWRP
[2020-01-23 02:27:25] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:27:31] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:27:37] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:27:43] WARN initial-sync: No peers; waiting for reconnect

**THEN IT BEGAIN THE SAME PROBLEM AND JUMP TO 40500 **

[2020-01-23 02:38:08] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:38:14] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:38:20] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:38:22] INFO p2p: Peer connected active=3 peer=16Uiu2HAm7ruW6N9xyeJMqZwUX4wUfvMUhKRgGQadLYxaPwubVPr5
[2020-01-23 02:38:27] INFO initial-sync: Processing block 40547/94092 - estimated time remaining 297h28m20s blocksPerSecond=0.1 peers=1/3
[2020-01-23 02:38:27] INFO p2p: Peer disconnected active=2 peer=16Uiu2HAm7ruW6N9xyeJMqZwUX4wUfvMUhKRgGQadLYxaPwubVPr5
[2020-01-23 02:38:27] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-23 02:38:27] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:38:33] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:38:39] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:38:45] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:38:51] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 02:38:57] WARN initial-sync: No peers; waiting for reconnect

AGAIN NO PEERS AND LAST ERROR AFTER THIS ONE IT EGAN TO WORK AGAIN

ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-23 03:13:27] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:13:27] INFO p2p: Peer disconnected active=1 peer=16Uiu2HAm31CrTnSqozW2o1ovsmZ6wxRXioYBpRSLvQGyq8QtfWRP
[2020-01-23 03:13:33] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:13:39] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:13:40] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-23 03:13:40] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-01-23 03:13:45] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:13:51] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:13:57] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:14:03] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:14:09] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:14:13] INFO p2p: Peer connected active=2 peer=16Uiu2HAmLpabFfbMpMyZhqEevu2hCfCuiRyc2gSWaRUVP9ozSA8j
[2020-01-23 03:14:16] ERROR initial-sync: Round robing sync request failed error=no peers left to request blocks
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
github.com/prysmaticlabs/prysm/beacon-chain/sync/initial-sync.(*Service).roundRobinSync.func2.1
beacon-chain/sync/initial-sync/round_robin.go:140
runtime.goexit
GOROOT/src/runtime/asm_amd64.s:1357
[2020-01-23 03:14:16] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:14:16] INFO p2p: Peer disconnected active=1 peer=16Uiu2HAmLpabFfbMpMyZhqEevu2hCfCuiRyc2gSWaRUVP9ozSA8j
[2020-01-23 03:14:22] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:14:28] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:14:34] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:14:40] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:26:41] WARN initial-sync: No peers; waiting for reconnect
[2020-01-23 03:26:43] INFO p2p: Peer connected active=2 peer=16Uiu2HAkz1t6hf1xcyknqASiGxWwQHqb6griqnMu5Wt2MC6yhhK9
[2020-01-23 03:26:44] INFO p2p: Peer connected active=4 peer=16Uiu2HAmLvwPY32yihHJqgSrPPg59PTtp5GXJLpWn2P75PELjue2
[2020-01-23 03:26:48] INFO initial-sync: Processing block 44119/94334 - estimated time remaining 278h58m20s blocksPerSecond=0.1 peers=2/3
[2020-01-23 03:26:48] INFO initial-sync: Processing block 44120/94334 - estimated time remaining 139h29m0s blocksPerSecond=0.1 peers=2/3
[2020-01-23 03:26:48] INFO initial-sync: Processing block 44121/94334 - estimated time remaining 92h59m13s blocksPerSecond=0.1 peers=2/3
[2020-01-23 03:26:48] INFO initial-sync: Processing block 44122/94334 - estimated time remaining 69h44m20s blocksPerSecond=0.2 peers=2/3
[2020-01-23 03:26:48] INFO initial-sync: Processing block 44123/94334 - estimated time remaining 55h47m24s blocksPerSecond=0.2 peers=2/3
[2020-01-23 03:26:48] INFO initial-sync: Processing block 44124/94334 - estimated time remaining 46h29m26s blocksPerSecond=0.3 peers=2/3

@prestonvanloon
Copy link
Member

I think there may be a bug here where your client is spamming your peers then they disconnect you.

I'll try to reproduce today.

@prestonvanloon prestonvanloon added the Bug Something isn't working label Jan 23, 2020
This was referenced Jan 23, 2020
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use a client side rate limit to reduce chance of getting banned prysmaticlabs/prysm
2 participants
@prestonvanloon @Gsuslu