geth --syncmode "fast" --cache 4000 never syncs

[kzaher]

Hi,

Cleaning up ethereum directory and then executing
geth --syncmode "fast" --cache 4000 never finishes. After a day it's still holding a steady distance from blockchain highest block. The distance isn't getting smaller. It is always ~100 blocks.

If the problem was my hardware, the sync process wouldn't get within 100 blocks near top and then continuously maintain the distance.

I also have a parity client installed, and it syncs just fine in fast mode. Parity syncs in ~2 hours. Please fix geth, this is extremely frustrating.

When I look at my resource usage, there is no significant CPU, memory, disk IOPs, network bandwidth used.

Hardware:

• Macbook mid 2014
• 512 GB SSD
• 2.5 GHz Intel Core i7 (Quad core)
• 16GB RAM 1600 MHz DDR3
• 100Mbit/s symmetric network (I sometimes take it to 1Gbit/s network, but it doesn't make any difference).

geth never syncs to the latest block. There aren't any problems with my machine.

System information

Geth version: Geth/v1.8.13-stable/darwin-amd64/go1.10 (compiled from sources)
OS & Version: OSX 10.13.6 (17G65)
Commit hash : (if develop) tag v1.8.13

Expected behaviour

For it to sync properly.

Actual behaviour

It never syncs completely.

> web3.eth.getBlock('latest')
{
  difficulty: 17179869184,
  extraData: "0x11bbe8db4e347b4e8c937c1c8370e4b5ed33adb3db69cbdb7a38e1e50b1b82fa",
  gasLimit: 5000,
  gasUsed: 0,
  hash: "0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3",
  logsBloom: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  miner: "0x0000000000000000000000000000000000000000",
  mixHash: "0x0000000000000000000000000000000000000000000000000000000000000000",
  nonce: "0x0000000000000042",
  number: 0,
  parentHash: "0x0000000000000000000000000000000000000000000000000000000000000000",
  receiptsRoot: "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
  sha3Uncles: "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
  size: 540,
  stateRoot: "0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544",
  timestamp: 0,
  totalDifficulty: 17179869184,
  transactions: [],
  transactionsRoot: "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
  uncles: []
}
> web3.eth.syncing
{
  currentBlock: 6182500,
  highestBlock: 6182594,
  knownStates: 94729122,
  pulledStates: 94729121,
  startingBlock: 6182500
}
> web3.eth.getBlock(6182520)
null

I get endless streams of

INFO [08-20|19:31:08.178] Imported new state entries               count=0    elapsed=73.573ms  processed=94803463 pending=29605 retry=2   duplicate=10424 unexpected=16636
INFO [08-20|19:31:08.318] Imported new state entries               count=0    elapsed=1.048ms   processed=94803463 pending=29610 retry=382 duplicate=10588 unexpected=16854
INFO [08-20|19:31:08.454] Imported new state entries               count=1156 elapsed=3.732ms   processed=94804619 pending=29281 retry=0   duplicate=10588 unexpected=16854
INFO [08-20|19:31:08.852] Imported new state entries               count=0    elapsed=17.915ms  processed=94804619 pending=28905 retry=2   duplicate=10590 unexpected=16854
INFO [08-20|19:31:09.081] Imported new state entries               count=0    elapsed=648.026µs processed=94804619 pending=28903 retry=382 duplicate=10643 unexpected=17183
INFO [08-20|19:31:09.405] Imported new state entries               count=0    elapsed=48.319ms  processed=94804619 pending=29040 retry=2   duplicate=10643 unexpected=17185
INFO [08-20|19:31:09.477] Imported new state entries               count=0    elapsed=1.242ms   processed=94804619 pending=29048 retry=382 duplicate=10816 unexpected=17394
INFO [08-20|19:31:09.590] Imported new state entries               count=1154 elapsed=4.484ms   processed=94805773 pending=28593 retry=0   duplicate=10816 unexpected=17394
INFO [08-20|19:31:09.850] Imported new state entries               count=0    elapsed=34.708ms  processed=94805773 pending=28198 retry=2   duplicate=10818 unexpected=17394
INFO [08-20|19:31:09.912] Imported new state entries               count=0    elapsed=518.714µs processed=94805773 pending=28193 retry=382 duplicate=10863 unexpected=17731
INFO [08-20|19:31:10.536] Imported new state entries               count=0    elapsed=158.344ms processed=94805773 pending=28726 retry=2   duplicate=10863 unexpected=17733
INFO [08-20|19:31:10.615] Imported new state entries               count=1152 elapsed=3.180ms   processed=94806925 pending=28979 retry=0   duplicate=10863 unexpected=17733
I

Steps to reproduce the behaviour

• Clean up ethereum directory (~/Library/Ethereum)
• $ geth --syncmode "fast" --cache 4000

[RaghavSood]

It is syncing. A fast sync must download the state trie without downloading all historical blocks, and the state trie is currently ~200m entries. You're just shy of 100m. It will take several day to pull it all together.

[kzaher]

Hi @RaghavSood ,

thank you for the information.

I think there are multiple problems here.

1. It seems to me that ./target/release/parity --no-ancient-blocks completes similar operation in a matter of couple of hours. I understand that it's not downloading ancient blocks, and that it takes a couple of hours for geth to download just the blocks. But we are talking about an order of magnitude difference here.
2. If your business is depending on ethereum integration, and your database gets corrupted so you need to resync, just fiddling your fingers for a couple of days for fastest sync (which enables access to recent states) is obviously not an option. I would really want to use the default implementation because it seems like the safest option, but this is not reasonable.
3. If the current state snapshot is ~6GB big (at least this is what parity shows, I'm not sure is that compressed or not), and I'm giving it 4GB of RAM, then I would expect it to at least saturate my CPU or network bandwith, which doesn't happen.
4. Why does Parity use all of my cores or at least multiple ones when importing state, and geth usually uses < 1

Basically, my hardware is sitting there idle on 1Gbit/s network, so I would say that waiting for a couple of days for state to synchronize makes no sense.

5. Why is my client constantly ~100 blocks behind the top, this makes no sense to me? And ofc, web3.eth.getBlock('latest').number is 0, but when I use some concrete block numbers, I get values.

[aekasitt]

This also happens to my cloning of ethereum/client-go docker image and it causes all kinds of dead-ends when trying to sendTransaction/sendRawTransaction because all of stored wallet balances return 0.

[Marfy22]

I also have the same issue. It all started after I deleted my Geth folder to try and create a new one with less Gb than I had, something I had done in the past successfully.

I use Windows 10 Pro, and issuing the following in CMD: geth -rpc --datadir=C:\Ethereum --syncmode "fast" --cache=2048 console

System Intel i7-7700 CPU, 32Gb RAM, SAMSUNG M.2 500Gb SM961 Model.

[Marfy22]

UPDATE: Apparently, if you give it time, like many hours or a day (depends on the system I guess), it will eventually sync in full. It did on my systems (two of them)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.