Cant sync fresh node from snapshot #2231
Comments
|
Hello, I have the exact same issue, did you found a solution? Using reth and lighthouse as node clients |
|
I too have the same issue! Prysm 5.0.3 + geth version 1.13.14-stable-2bd6bd01 |
|
Are you guys using Because I was using I am not sure that my node is syncing correctly tho EDIT: Actually the error is still there, it appeared after 1h of running the node |
|
I am getting the same error with v2.3.3-6a1c1a7 |
|
Hey guys, the reason its unable to sync is because the snapshot is older and the nitro node is requesting for already expired blobs. As a way to unblock your node you can try using an archival beacon rpc provider. They will be able to momentarily provide the blobs: |
This did not work for me. |
|
@hdiass ，hi，do you have a better solution? |
|
@hdiass I am the same client and beacon chain as you are |
|
INFO [04-15|01:57:24.481] Loaded most recent local block number=193,592,599 hash=c758a4..c8df38 td=171,384,783 age=3w21h46m |
|
@nisdas @hdiass The --parent-chain.blob-client.beacon-url flag in my command was replaced by a local prysmrpc quicknode's Ethereum beacon rpc looks like it's synced a bit, and there's no problem with the start to initialize snapshot to sync。 INFO [04-15|02:14:46.197] Unindexing transactions blocks=19,587,000 txs=22,043,742 total=67,362,073 elapsed=6m1.193s |
|
@nisdas ，hi， |
|
@ZYS980327 Hey, after the arbitrum node is synced you can use your local prysm node. You only need the archival blobs if the snapshot is old |
|
@miki-bgd-011 Do you have any specific logs for this ? |
|
@nisdas INFO [04-15|04:08:48.545] latest assertion not yet in our node staker=0x0000000000000000000000000000000000000000 assertion=13191 state="{BlockHash:0x603c342a38a945b720e14601c6ae1a65257766ae93fe2aef4dd21e26b1ba77a2 SendRoot:0xdc2194034507b095e60dcffacb8a5e1d7713b2919f94e30de12b4233d8956554 Batch:587299 PosInBatch:0}" |
|
@nisdas But I don't know how to restart, I can only add --init-url and resync from the snapshot |
1 similar comment
|
@nisdas But I don't know how to restart, I can only add --init-url and resync from the snapshot |
|
@nisdas And after starting the arbitrum node, the connection to port 8547 to get the block data is denied |
|
You would just need to replace this flag |
|
@nisdas docker run -d --privileged --rm -it -v /usr/local/nitro-snap/:/usr/local/nitro-snap/ -p 0.0.0.0:8547:8547 -p 0.0.0.0:8548:8548 offchainlabs/nitro-node:v2.3.2-064fa11 --parent-chain.connection.url http://10.150.20.11:8545 --chain.id=42161 --parent-chain.blob-client.beacon-url=http://10.150.20.11:3500 --http.api=net,web3,eth,arb,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* --init.url="file:///usr/local/nitro-snap/nitro-pruned.tar" --init.download-path=/usr/local/nitro-snap/snapshot-8547.tar |
|
Is the prysm node running ? what are your prysm and arbitrum logs |
|
|
|
prysm |
|
|
|
@nisdas Always 8702476 |
|
Does your prysm node have peers ? it appears to be stuck for some reason |
|
yes, |
|
@nisdas Prysm is normal |
|
@ZYS980327 This particular slot was 3 weeks old: Just to confirm, you are just restarting the nitro node and replacing it with the local beacon rpc url ? |
|
@nisdas Yes, and --init-download-path |
|
@nisdas Okay, thanks for the answer, and about how often the official snapshot will be uploaded？ |
|
|
@limitrinno Can you take a look at your execution commands, and the configured geth and prysm startup commands or environments |
|
This doesnt work with chainstack, so the instructions in that URL for beacon chain providers is wrong. |
|
The only way i found to sync this was getting an updated snapshot. |
I just downloaded the snapshot today, but I'm still encountering the same error. I'm currently trying with Ankr's API. |
I'm currently using the latest version of the Arb Docker container on Ubuntu 20. Both sets of environment parameters and configurations are the same. The only difference is that the one experiencing issues was started synchronizing recently. |
|
I fully backfilled blob data in prysm: Still receiving the same error for slot 8702476. Any suggestions besides using an external RPC? |
|
Tried using an external RPC node:
Still getting the same error. |
|
Seems like this could very easily be fixed by the team uploading snapshots more frequently than every couple months... Has no one found a solution?? |
|
Agreed. Generally, I tend to think that this is an issue with the snapshot itself. On another node, my prysm blob directory has exactly the same entries as on this new node, and I synced succesfully from it just two or three weeks ago using the same nitro version. |
So nitro will ignore the snapshot if there already is a database @ZYS980327 . Is it possible you are running from a new directory on each restart ? |
|
@nisdas The command is the same every time, and occasionally the port number may be changed docker run -d --privileged --rm -it -v /usr/local/nitro-snap/:/usr/local/nitro-snap/ -p 0.0.0.0:8557:8557 -p 0.0.0.0:8558:8558 offchainlabs/nitro-node: v2.3.2-064fa11 --parent-chain.connection.url http://10.150.20.11:8545 --chain.id=42161 --parent-chain.blob-client.beacon-url=http://10.150.20.11:3500 --http.api=net,web3, eth,arb,debug --http.corsdomain=* --http.addr=0.0.0.0 --http.vhosts=* --init.url="file:///nitro-pruned.tar" ... |
|
@nisdas On Nitro 2.3.3, the developer said that the content that did not get blobs was optimized, but the snapshot is still the same slot that was problematic three weeks ago, so 2.3.3 is useless |
|
@ZYS980327 you cant run the image with |
|
@nisdas ok |
|
@nisdas I feel like the official snapshot should be updated soon |
|
@nisdas I see that you opened a feature request with prysm. Just for my understanding, in what way does your requested feature differ from |
|
So I've tried Chainstack and Ankr:
In both cases, it does not work: This is starting to get very frustating. |
I've also tried different APIs. Now, I'm planning to move the data from the currently running healthy node to a new node. I'm pruning it as it's too large, about a terabyte. I think directly transferring it over should be the best solution at the moment. |
|
Any update on this issue? It's almost one week since I can't synchronize one of my nodes after a cleanup. |
Nope... still unable to sync either of my nodes after countless attempts on both machines. If only the foundation would just release a new snapshot... |
|
I've just set up a server, and have the same issue. I use Lighthouse + Nethermind. I can confirm that issue cannot be resolved by using Chainstack archive node. upd: I tried Ankr, and it worked the node started to sync. It costs $10 ($10 is minimal amount allowed to top up). But I face random long sync freezes, I suppose Ankr just rate-limiting me, however their site says there were only 600 requests last hour. |
|
Hello, I used quicknode free tier and was able to sync my node in around 24h-48h.
And it worked for me, in around 24h-48h I was able to replace the quicknode beacon layer by my own beacon layer and it is running good at latest blocks. |
|
I managed to sync using Ankr. It took 24 hours, in case somebody needs archive beacon node you can use mine since I have credits there more than I needed: upd: Sorry, I had to disable the node, because I am out of credits. |
|
There is new snapshot available at https://snapshot.arbitrum.foundation/index.html that solves the issue (at least for now). |
|
seem issues: image=offchainlabs/nitro-node:v2.3.3-6a1c1a7
when sync block hight 190281129 and it can't sync!!! there have no error logs: how to fix this issues? |
|
Switched beacon endpoint to Quicknode and started syncing. Will switch back to local (non-archive) node when it's closer to head. |
Describe the bug
Cant sync fresh node from snapshot. v2.3.3-6a1c1a7
Yesterday booted a node from scratch using snapshot and it can't sync.
Eth client
geth v1.13.14
prysm v5.0.3
using checkpoint sync and
--enable-experimental-backfill
nitro args used
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: