Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Nethermind is not syncing old blocks on Ropsten #4189

Closed
gituser opened this issue Jun 22, 2022 · 7 comments
Closed

Nethermind is not syncing old blocks on Ropsten #4189

gituser opened this issue Jun 22, 2022 · 7 comments
Assignees
@marcindsobczak

Comments

@gituser
Copy link

gituser commented Jun 22, 2022 •
edited
Loading

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. Run latest v1.13.3 nethermind version and sync ropsten
  2. As a CL I run lighthouse v2.3.1
  3. After syncing headers, state, everything Nethermind is not syncing old blocks, queue just grows until it grows to 12350688 but then nothing happens:
2022-06-22 07:17:52.4048|INFO|14|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:17:53.4035|INFO|263|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:17:54.4092|INFO|260|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:17:55.4075|INFO|121|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:17:56.4057|INFO|121|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:17:57.4073|INFO|198|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:17:58.4075|INFO|145|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:17:59.4056|INFO|128|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:00.4063|INFO|118|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:01.4064|INFO|198|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:02.4071|INFO|278|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:03.4076|INFO|133|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:04.4062|INFO|128|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:05.4055|INFO|128|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:06.4074|INFO|118|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:07.4068|INFO|162|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:08.4069|INFO|128|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:09.4074|INFO|118|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:10.4059|INFO|198|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:11.4061|INFO|162|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:12.4074|INFO|118|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:13.4060|INFO|198|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:14.4061|INFO|158|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:15.4059|INFO|158|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:16.4083|INFO|199|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:17.4082|INFO|21|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:18.4061|INFO|21|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:19.4072|INFO|118|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:20.4062|INFO|263|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:21.4057|INFO|21|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps 
2022-06-22 07:18:22.4057|INFO|160|Old Bodies          0 / 12380000 | queue 12350688 | current     0.00bps | total     0.00bps
  1. Althrough new blocks are received fine from consensus client and I can see that latest block is being advanced to the latest one.

Expected behavior
Nethermind should be synced completely.

  • OS: Ubuntu 18.04
  • Version v1.13.3

Additional context

======================== Nethermind initialization completed ========================
Version      : 1.13.3-0-0e3289f55-20220621/X64-Linux/6.0.5
This node    : enode://yyy@x.x.x.x:30303
RPC modules  : Eth, Net, Subscribe, Trace, Web3
Node address : 0x123 (do not use as an account)
Mem est tx   :     41MB
Mem est DB   :   6513MB
JSON RPC     : http://0.0.0.0:8545 ; http://127.0.0.1:8551
Genesis hash : 0x41941023680923e0fe4d74a34bdac8141f2540e3ae90623718e47d66d1ca4a2d
External IP  : x.x.x.x
Ethereum     : tcp://x.x.x.x:30303
Discovery    : udp://x.x.x.x:30303
Chainspec    : chainspec/ropsten.json
Chain ID     : Ropsten
Chain head   : 12442602 (0xc42d16...01483f)
=====================================================================================

Here is my config:

{
  "Init": {
    "ChainSpecPath": "chainspec/ropsten.json",
    "GenesisHash": "0x41941023680923e0fe4d74a34bdac8141f2540e3ae90623718e47d66d1ca4a2d",
    "BaseDbPath": "nethermind_db/ropsten",
    "LogFileName": "ropsten.logs.txt",
    "MemoryHint": 11480000000
  },
  "TxPool": {
    "Size": 1024
  },
  "Sync": {
    "FastSync": true,
    "FastBlocks": true,
    "UseGethLimitsInFastBlocks": true,
    "PivotNumber": 12380000,
    "PivotHash": "0x03075224249c46f94300f689e8a38b54929244379e046b3bae4c3dd37a1aeca1",
    "PivotTotalDifficulty": "50000820485795157",
  },
  //"EthStats": {
  //  "Server": "ws://ropsten-stats.parity.io/api"
  //},
  "Metrics": {
    "NodeName": "Ropsten"
  },
  "JsonRpc": {
    "Enabled": true,
    "Timeout": 20000,
    "Host": "0.0.0.0",
    "Port": 8545,
    "EnabledModules": [
      "Eth",
      "Subscribe",
      "Trace",
      "TxPool",
      "Web3",
      "Personal",
      "Proof",
      "Net",
      "Parity",
      "Health"
    ],
    "AdditionalRpcUrls": [
      "http://127.0.0.1:8551|http;ws|net;eth;subscribe;engine;web3;client"
    ],
    "JwtSecretFile":"/home/nethermind/.nethermind/jwt_secret",
    "IpcUnixDomainSocketPath":"/home/nethermind/.nethermind/nethermind.ipc"
  },
  "Merge": {
    "Enabled": true,
    "TerminalTotalDifficulty": "50000000000000000"
  }
}
@gituser gituser mentioned this issue Jun 22, 2022
Closed
@gituser
Copy link
Author

gituser commented Jun 22, 2022

Ping @LukaszRozmej @MarekM25 @marcindsobczak

@marcindsobczak
Copy link
Contributor

We are investigating it right now

@gituser
Copy link
Author

gituser commented Jun 22, 2022

@marcindsobczak Thank you!

Please let me know if you need any more info from my side.

@LukaszRozmej
Copy link
Member

@gituser got a positive response from @marcindsobczak that it looks fixed in master, we are doing more testing and want to release it next week. You can use experimental version for now.

@gituser
Copy link
Author

gituser commented Jun 24, 2022 •
edited
Loading

@LukaszRozmej thank you for the update.

It seems to be working with lighthouse after numerous tries of resync.
Would be nice if you could include default settings for Ropsten's config:

...
"Sync": {
...
"SnapSync": true,
...
},
...
"Merge": {
    "Enabled": true,
    "TerminalTotalDifficulty": "50000000000000000"
  }

also maybe get rid of these lines, as this server is no longer working (ropsten-stats.parity.io):

"EthStats": {
    "Server": "ws://ropsten-stats.parity.io/api"
  },

@marcindsobczak
Copy link
Contributor

marcindsobczak commented Jun 24, 2022 •
edited
Loading

Thank you for feedback. We have not enabled SnapSync as a default mode yet because of some issues with post-merge sync, but at this point I think it's good idea to make this change. Merge TTD is not needed in the config - it's already hardcoded. I will create a PR with SnapSync as default and EthStats removed
edit: PR #4209

@gituser
Copy link
Author

gituser commented Jun 27, 2022 •
edited
Loading

I'll close this issue as I was able to sync all blocks on Ropsten with Lighthouse CL with the latest Nethermind built from master (d433fcf).

@gituser gituser closed this as completed Jun 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@marcindsobczak marcindsobczak

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gituser @LukaszRozmej @marcindsobczak