Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF #4517

Closed
avalyn2 opened this issue Jan 12, 2020 · 6 comments · Fixed by #4743
Closed

ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF #4517

avalyn2 opened this issue Jan 12, 2020 · 6 comments · Fixed by #4743
Labels
Bug Something isn't working

Comments

@avalyn2
Copy link

avalyn2 commented Jan 12, 2020 •
edited

thought I would share:

image
@0xKiwi
Copy link
Contributor

0xKiwi commented Jan 17, 2020

Was this an error you saw only once? Your node kept running after it?

@0xKiwi 0xKiwi added the Bug Something isn't working label Jan 17, 2020
@torfbolt
Copy link

I encountered the same here:

[2020-01-17 11:42:11] INFO initial-sync: Processing block 53909/53910 - estimated time remaining 2s blocksPerSecond=0.5 peers=1/15
[2020-01-17 11:42:11] INFO forkchoice: Executing state transition on block root=0x4f569fda... slot=53909
[2020-01-17 11:42:13] INFO blockchain: Finished applying state transition attestations=10 deposits=0 slot=53909
[2020-01-17 11:42:13] INFO initial-sync: Resync attempt complete slot=53909 synced=true
[2020-01-17 11:58:58] INFO sync: Fallen behind peers; reverting to initial sync to catch up currentEpoch=1687 peersEpoch=1685 syncedEpoch=1684
[2020-01-17 12:44:29] ERROR sync: Failed to decode stream message error=stream reset peer=16Uiu2HAm9ajSvWnFatEMBc5vMThLq83E9q475H3U2q72Tz616LDV topic=/eth2/beacon_chain/req/status/1/ssz
[2020-01-17 12:44:29] ERROR sync: Failed to decode stream message error=stream reset peer=16Uiu2HAm9ajSvWnFatEMBc5vMThLq83E9q475H3U2q72Tz616LDV topic=/eth2/beacon_chain/req/status/1/ssz
[2020-01-17 12:55:00] INFO p2p: Peer disconnected active=14 peer=16Uiu2HAkwpVdQSGdkxzbdpQ7ii4N8CA7tAFFPEeHMTGHqrSZyc9D
[2020-01-17 12:55:00] INFO p2p: Peer disconnected active=13 peer=16Uiu2HAmB9ghaaQipZiTmtMuHkSyg84cHt1NjEtzEQR6rjXoEzH9
[2020-01-17 12:55:41] INFO p2p: Peer connected active=15 peer=16Uiu2HAkwpVdQSGdkxzbdpQ7ii4N8CA7tAFFPEeHMTGHqrSZyc9D
[2020-01-17 12:55:41] INFO p2p: Peer connected active=15 peer=16Uiu2HAmB9ghaaQipZiTmtMuHkSyg84cHt1NjEtzEQR6rjXoEzH9
[2020-01-17 13:12:17] ERROR powchain: Post https://goerli.prylabs.net: EOF
[2020-01-17 13:12:22] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-17 13:12:23] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-01-17 13:29:56] INFO p2p: Peer disconnected active=14 peer=16Uiu2HAmRMqPe1nR1yAwUEpBKzUh8ABHzcMBSRnnK1z4fkuARu8e
[2020-01-17 13:31:30] INFO p2p: Peer connected active=15 peer=16Uiu2HAmRMqPe1nR1yAwUEpBKzUh8ABHzcMBSRnnK1z4fkuARu8e
[2020-01-17 13:34:51] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-17 13:34:51] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-01-17 13:38:41] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-17 13:38:41] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-01-17 13:44:41] INFO p2p: Peer connected active=16 peer=16Uiu2HAm9ajSvWnFatEMBc5vMThLq83E9q475H3U2q72Tz616LDV
[2020-01-17 13:44:47] INFO p2p: Peer disconnected active=15 peer=16Uiu2HAm9ajSvWnFatEMBc5vMThLq83E9q475H3U2q72Tz616LDV
[2020-01-17 13:51:42] INFO p2p: Peer disconnected active=14 peer=16Uiu2HAmRMqPe1nR1yAwUEpBKzUh8ABHzcMBSRnnK1z4fkuARu8e
[2020-01-17 13:55:30] INFO p2p: Peer disconnected active=13 peer=16Uiu2HAmB9ghaaQipZiTmtMuHkSyg84cHt1NjEtzEQR6rjXoEzH9
[2020-01-17 13:55:30] INFO p2p: Peer disconnected active=12 peer=16Uiu2HAkwpVdQSGdkxzbdpQ7ii4N8CA7tAFFPEeHMTGHqrSZyc9D

Afterwards the client continues to run but did not process blocks any more, there's just p2p connect & disconnect messages and more repetitions of the same error:

ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF

@avalyn2
Copy link
Author

avalyn2 commented Jan 18, 2020 •
edited

still getting those errors:

[2020-01-18 15:47:28] INFO initial-sync: Processing block 246/62337 - estimated time remaining 5m57s blocksPerSecond=173.5 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 247/62337 - estimated time remaining 5m57s blocksPerSecond=173.6 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 248/62337 - estimated time remaining 5m57s blocksPerSecond=173.6 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 249/62337 - estimated time remaining 5m57s blocksPerSecond=173.7 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 250/62337 - estimated time remaining 5m57s blocksPerSecond=173.7 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 251/62337 - estimated time remaining 5m57s blocksPerSecond=173.8 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 252/62337 - estimated time remaining 5m57s blocksPerSecond=173.8 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 253/62337 - estimated time remaining 5m57s blocksPerSecond=173.8 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 254/62337 - estimated time remaining 5m57s blocksPerSecond=173.9 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 255/62337 - estimated time remaining 5m56s blocksPerSecond=173.9 peers=2/2
[2020-01-18 15:47:28] INFO initial-sync: Processing block 256/62337 - estimated time remaining 5m56s blocksPerSecond=174.0 peers=2/2
[2020-01-18 16:23:13] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-18 16:23:13] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-01-18 16:36:15] INFO p2p: Peer connected active=2 peer=16Uiu2HAmAsyW714qpiYz9dNEoDyLa29SuzXwU4sY3fQrRtUxN1bT
[2020-01-18 16:36:16] INFO p2p: Peer connected active=4 peer=16Uiu2HAmGQXdqwKfzR4c4ejRApgbHntTSd1YZRKvRAwz3mzwhZgp
[2020-01-18 16:36:16] INFO p2p: Peer connected active=4 peer=16Uiu2HAkz1t6hf1xcyknqASiGxWwQHqb6griqnMu5Wt2MC6yhhK9
[2020-01-18 16:36:26] INFO p2p: Peer connected active=5 peer=16Uiu2HAmEBfmnE1AbnBvVjLaAHyuixgtcSq3QwCHis2vwPtZBGVV
[2020-01-18 16:37:06] INFO p2p: Peer connected active=6 peer=16Uiu2HAkvSZyPFfSonVPiF9EEPKKv65v6QcPC3sBgPVRxURWX2u3
[2020-01-18 16:37:15] INFO p2p: Peer connected active=7 peer=16Uiu2HAm31CrTnSqozW2o1ovsmZ6wxRXioYBpRSLvQGyq8QtfWRP
[2020-01-18 16:38:45] INFO p2p: Peer connected active=8 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG
[2020-01-18 16:42:52] INFO p2p: Peer connected active=9 peer=16Uiu2HAkuUGWAJbiMG3SeE8MkwfZTcox7e8unfCCtBTPVZaMLQ4A
[2020-01-18 16:44:36] INFO p2p: Peer connected active=9 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG
[2020-01-18 16:48:40] INFO p2p: Peer connected active=9 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG
[2020-01-18 16:55:08] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-18 16:55:08] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-01-18 17:22:23] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-18 17:22:24] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-01-18 17:35:57] INFO p2p: Peer connected active=9 peer=16Uiu2HAm12JGMpCDPnp3MzoJ2qAHkqESBSM9sU7Pf2ukeiQZ7KnF
[2020-01-18 17:38:23] INFO p2p: Peer connected active=9 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG
[2020-01-18 17:38:26] INFO p2p: Peer connected active=10 peer=16Uiu2HAkz1t6hf1xcyknqASiGxWwQHqb6griqnMu5Wt2MC6yhhK9
[2020-01-18 17:41:12] INFO p2p: Peer connected active=9 peer=16Uiu2HAkz1t6hf1xcyknqASiGxWwQHqb6griqnMu5Wt2MC6yhhK9
[2020-01-18 17:42:35] INFO p2p: Peer connected active=10 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG
[2020-01-18 17:47:33] INFO p2p: Peer connected active=10 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG
[2020-01-18 17:48:29] INFO p2p: Peer connected active=11 peer=16Uiu2HAmS7QJSqKDMezD2MZs1fL8pw9jwWbJdXvcuA8t4kudr9g9
[2020-01-18 17:50:53] INFO p2p: Peer connected active=11 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG
[2020-01-18 17:51:43] INFO p2p: Peer connected active=12 peer=16Uiu2HAmCZrJhRz73Y142UYRmwTz56qjLDM9KJn9MnWkE7DBcrV7
[2020-01-18 17:53:34] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-18 17:53:35] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket

@avalyn2
Copy link
Author

avalyn2 commented Jan 18, 2020 •
edited

Was this an error you saw only once? Your node kept running after it?

  1. no, saw it again other times, above the errors when I was running the node today.
  2. yes, ran continued after error:

[2020-01-18 16:23:13] ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-01-18 16:23:13] INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-01-18 16:36:15] INFO p2p: Peer connected active=2 peer=16Uiu2HAmAsyW714qpiYz9dNEoDyLa29SuzXwU4sY3fQrRtUxN1bT
[2020-01-18 16:36:16] INFO p2p: Peer connected active=4 peer=16Uiu2HAmGQXdqwKfzR4c4ejRApgbHntTSd1YZRKvRAwz3mzwhZgp
[2020-01-18 16:36:16] INFO p2p: Peer connected active=4 peer=16Uiu2HAkz1t6hf1xcyknqASiGxWwQHqb6griqnMu5Wt2MC6yhhK9
[2020-01-18 16:36:26] INFO p2p: Peer connected active=5 peer=16Uiu2HAmEBfmnE1AbnBvVjLaAHyuixgtcSq3QwCHis2vwPtZBGVV
[2020-01-18 16:37:06] INFO p2p: Peer connected active=6 peer=16Uiu2HAkvSZyPFfSonVPiF9EEPKKv65v6QcPC3sBgPVRxURWX2u3
[2020-01-18 16:37:15] INFO p2p: Peer connected active=7 peer=16Uiu2HAm31CrTnSqozW2o1ovsmZ6wxRXioYBpRSLvQGyq8QtfWRP
[2020-01-18 16:38:45] INFO p2p: Peer connected active=8 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG
[2020-01-18 16:42:52] INFO p2p: Peer connected active=9 peer=16Uiu2HAkuUGWAJbiMG3SeE8MkwfZTcox7e8unfCCtBTPVZaMLQ4A
[2020-01-18 16:44:36] INFO p2p: Peer connected active=9 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG
[2020-01-18 16:48:40] INFO p2p: Peer connected active=9 peer=16Uiu2HAmV2Rq9Pyv8GXusWrxRu9YZFBHzYaHHPLFDgzg9Gms5zyG

@Njoiyt
Copy link

Njoiyt commented Jan 18, 2020

ERROR powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF

I have been receiving this over and over today. I only get to about 4-8 thousand processed blocks.

docker run -it -v /tmp/prysm-data:/data -p 4000:4000 -p 13000:13000 gcr.io/prysmaticlabs/prysm/beacon-chain:testFix --datadir=/data --p2p-host-ip

@prestonvanloon
Copy link
Member

This can be resolved by running your own Goerli node. The goerli node that we provide is not very stable with so many websocket connections.

This error is mostly benign, your beacon chain node should attempt to reconnect and there is no penalty for not voting on eth1 data.

@rauljordan rauljordan mentioned this issue Feb 4, 2020
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Resolve Miscellaneous Bugs in Beacon Node prysmaticlabs/prysm
5 participants
@torfbolt @0xKiwi @prestonvanloon @Njoiyt @avalyn2