How to sync from height on or before 9th July 2019

[ankitamadan]

I am able to sync from any historical syncable height on or after 10th July 2019 but I am not able to sync from snapshot height on or before 9th July 2019
E.g
I can sync from any height on or after this block height -> 19060575 (10th July 2019)
I cannot sync from height on or before this block height -> 18829401 (9th July 2019)

[chainwhisper]

Hello,
can you elaborate when did you first start your fullnode?
it seems like you used state-sync at July 10th and get a snapshot. As a result, you cannot get the history blocks

[ankitamadan]

Hello,

I am trying to use state_sync from a given height. Ref -> https://github.com/binance-chain/BEPs/blob/master/BEP18.md#51-take-snapshot

When I try to sync from height >= 19060575 (10th July 2019), it works.
Any height on or after 10th July 2019 works like example I tried to use state_sync and sync from different
block heights e.g 40242473, 57789244 etc. It works

But, when i try to use state_sync and try to sync from height <= 18829401 (9th July 2019), it doesn't work.

Please let me know if I can provide with any more information in regard to this query.

Could you please advise.

[ackratos]

But, when i try to use state_sync and try to sync from height <= 18829401 (9th July 2019), it doesn't work.

1. every time you use a new node to state sync right? state sync is designed for first-time startup use only. If you already synced to 19060575, you cannot sync with a different height by just tweak config and restart. What you should do is create a new node folder, put genesis and config files in and start from scratch.
2. if answer to 1 is yes, what's the error log indicates 18829401 doesn't work?

[ankitamadan]

But, when i try to use state_sync and try to sync from height <= 18829401 (9th July 2019), it doesn't work.

1. every time you use a new node to state sync right? state sync is designed for first-time startup use only. If you already synced to 19060575, you cannot sync with a different height by just tweak config and restart. What you should do is create a new node folder, put genesis and config files in and start from scratch.
2. if answer to 1 is yes, what's the error log indicates 18829401 doesn't work?

My answer to 1 is yes.

There is no error log but when I change the value of state_sync_height to 18829401 in
config.toml (Please refer snippet from my config.toml). The node starts from genesis block height instead of block height 18829401

config.toml

# As state sync is an experimental feature, this switch can totally disable it on core network nodes (validator, witness)
state_sync_reactor = true

# If this node is many days behind the tip of the chain, StateSyncHeight
# allows them to catchup quickly by downloading app state (without historical blocks)
# in parallel and start syncing block afterwards
# <0 - turn off state sync
# =0 - sync from peer's latest height
# >0 - sync from that height
state_sync_height = 18829401

logs

I[2020-02-24|22:12:59.704] Stopping gossipDataRoutine for peer          module=consensus peer="Peer{MConn{34.247.169.233:27146} b323708b0bd429a95154ff0a287f0f42f432cb62 out}"
I[2020-02-24|22:12:59.704] Stopping gossipVotesRoutine for peer         module=consensus peer="Peer{MConn{34.247.169.233:27146} b323708b0bd429a95154ff0a287f0f42f432cb62 out}"
I[2020-02-24|22:12:59.827] No new orders for any pair, give up matching module=dexkeeper 
I[2020-02-24|22:12:59.827] No order comes in for the block              module=dexkeeper 
I[2020-02-24|22:12:59.834] start to collect publish information         module=pub height=1
I[2020-02-24|22:12:59.835] start to publish                             module=pub height=1 blockTime=1555567166228734998 numOfTrades=0 numOfOrders=0 numOfProposals=0 numOfStakeUpdates=0 numOfAccounts=22
I[2020-02-24|22:12:59.836] Executed block                               module=state height=1 validTxs=0 invalidTxs=0
I[2020-02-24|22:12:59.849] Committed state                              module=state height=1 txs=0 appHash=B6E4D322EDA04F783EBDCD3D59AA908D211675F890EBC05A893C2093D101D009
I[2020-02-24|22:12:59.934] No new orders for any pair, give up matching module=dexkeeper 
I[2020-02-24|22:12:59.934] No order comes in for the block              module=dexkeeper 
I[2020-02-24|22:12:59.934] start to collect publish information         module=pub height=2
I[2020-02-24|22:12:59.934] start to publish                             module=pub height=2 blockTime=1555567622154340159 numOfTrades=0 numOfOrders=0 numOfProposals=0 numOfStakeUpdates=0 numOfAccounts=0
I[2020-02-24|22:12:59.934] Executed block                               module=state height=2 validTxs=0 invalidTxs=0
I[2020-02-24|22:12:59.953] Committed state                              module=state height=2 txs=0 appHash=04C4F444F7101F26AAFC4F80507BEB61C066354BBB4B94502E6AF7E7DBE69CA1
I[2020-02-24|22:13:00.008] No new orders for any pair, give up matching module=dexkeeper 
I[2020-02-24|22:13:00.008] No order comes in for the block              module=dexkeeper 
I[2020-02-24|22:13:00.008] start to collect publish information         module=pub height=3
I[2020-02-24|22:13:00.008] start to publish                             module=pub height=3 blockTime=1555567627739441221 numOfTrades=0 numOfOrders=0 numOfProposals=0

[ackratos]

My answer to 1 is yes.

that's wired, it should work if you create empty config and data folder.

could you please

1. grep "module=statesync" from log and paste here?
2. curl localhost:27147/peers so that we can check the data in peers you are syncing from are valid and complete
3. paste folder structure of your BNBHOME/data folder? more levels on snapshot folder if it exists

Thanks a lot!

[ackratos]

sorry i remembered back to that height, this statesync function is not online yet. This belongs to an additional feature. How keen are you on these data and what do you want used it for? Could you bear syncing from 0 to 19060575 its about 2 months' data?

[ankitamadan]

sorry i remembered back to that height, this statesync function is not online yet. This belongs to an additional feature. How keen are you on these data and what do you want used it for? Could you bear syncing from 0 to 19060575 its about 2 months' data?

All good! I can sync from 0 to 19060575.
I am syncing full node, since full node sync takes too long, I have created
4 parallel nodes to sync data. Was just keen to know why it didn't sync
for blocks on or before 18829401, is there any way we can sync we can sync on or before 18829401 block tho? Since its 2 months data, it might take more than a month to sync.
I am keen on syncing node a bit faster

[ackratos]

Since its 2 months data, it might take more than a month to sync.

That's wired, the sync speed should be 20 blocks/seconds, it should spend you about 10 days to sync.
Are you running bnbchaind on a ssd hard drive?

[ankitamadan]

I started genesis node 2 weeks before and it synced up 11m blocks. I guess 10 more days will do.
I am running docker image on virtual server vultr

configuration

4 cpu
8192 MB Memory
4000 GB Bandwidth

[voron]

@ankitamadan how did you merge nodes with different start sync block into one to get a full node from block 0 ? statesync allows to start from any block after BEP-18 feature release, thus it's possible to spin up multiple nodes in parallel to sync, for example, 1M block ranges, but how to merge multiple overlapped chains into one to get a boost compared to single node sync ?

[C-731]

Yooooo where/who can I get 4000GB bandwidth from? Whose a reputable supplier???