Blockchain sync stucked on new 0.9.2 client

[krakerjaak]

Hey Luke, some of us on maxchat.info seem to have a problem with the new client... our client stopped syncing the blockchain a few hours ago and it looks stuck...

In my debug.log, I see all blocks seen being recognized as orphaned (see below). And the wallet is stuck syncing at block 585,613, says "Catching up..." but never actually advances.

2014-08-31 18:52:47 received block 0000000000005f83a1b17d1e0bf1abac8b6fb69315f81390771fbb549defed74
2014-08-31 18:52:47 ProcessBlock: ORPHAN BLOCK, prev=000000000005003ce48393ba5d416ec91b21d0ee92bd0b80548c443517c9f173
2014-08-31 18:52:52 received block 000000000001407caf2ce178ecc79a6f1712f95a95c8e41c015799d9c5f178b1
2014-08-31 18:52:52 ProcessBlock: ORPHAN BLOCK, prev=0000000000005f83a1b17d1e0bf1abac8b6fb69315f81390771fbb549defed74
2014-08-31 18:53:14 received block 000000000002cd2935b16e326d3d1f4cca709c03d200b0be51c6395b71b67c7e
2014-08-31 18:53:14 ProcessBlock: ORPHAN BLOCK, prev=000000000001407caf2ce178ecc79a6f1712f95a95c8e41c015799d9c5f178b1

Any idea what's happening?
Note that apparently some who also have updated to 0.9.2 are properly synced up, but many aren't...

Update 1:

Ducky1 provided this log extract from 0.9.2 wallet:

2014-08-31 19:06:51 ERROR: AcceptBlock() : rejected by synchronized checkpoint

Update 2:

When I run getmininginfo in the debug window, I get the following output:

{
"blocks" : 586224,
"currentblocksize" : 0,
"currentblocktx" : 0,
"difficulty" : 16008.66895775,
"errors" : "Warning: checkpoint on different blockchain fork, contact developers to resolve the issue",
"generate" : false,
"genproclimit" : -1,
"hashespersec" : 0,
"networkhashps" : 2191766298290,
"pooledtx" : 3,
"testnet" : false
}

[krakerjaak]

It looks like running enforcecheckpoint false in the debug window will allow the wallet to sync back up. It's like the wallet thinks it's on a forked blockchain because of a checkpoint validation issue...

[krakerjaak]

It's probably known by now but for the benefit of this thread, the culprit checkpoint seems to be identified by running "getcheckpoint", which outputs:

{
"synccheckpoint" : "0000000000024968ac0715949d997bf3781ca91e3b5d601c744f751c9df2053f",
"height" : 580073,
"timestamp" : 1409322929,
"subscribemode" : "advisory"
}

Timestamp translates to block # 580073 on Fri, 29 Aug 2014 14:35:29 GMT

[repsorp]

Confirmed
Ubuntu client version v0.9.2.0-g8c5e74e-beta

The wallet is stuck syncing at 585557

Debug log:
2014-08-31 19:40:41 receive version message: /Max:0.9.1/: version 70001, blocks=586350, us=162.210.197.234:55734, them=188.165.252.23:8668, peer=188.165.252.23:8668
2014-08-31 19:40:42 received block 00000000000454a985bb891b9639de4ac0daadf6883874c4a5f9b4de22bdbae0
2014-08-31 19:40:42 ProcessBlock: ORPHAN BLOCK, prev=0000000000026fa4eb6b5b7802629febf1062e0fc7a71a11f0aca70de4fea3d6
2014-08-31 19:40:43 received block 000000000003a69ae5a73b91d41081ce934fe9fbd33179ef92e5e9b05fbef09a
2014-08-31 19:40:43 ERROR: AcceptBlock() : rejected by synchronized checkpoint
2014-08-31 19:40:43 ERROR: ProcessBlock() : AcceptBlock FAILED
2014-08-31 19:40:43 received block 0000000000008b063b693a5f38893e7ee7e13557374312181c3c252f1840e251
2014-08-31 19:40:43 ProcessBlock: ORPHAN BLOCK, prev=000000000003a69ae5a73b91d41081ce934fe9fbd33179ef92e5e9b05fbef09a
2014-08-31 19:40:43 received block

[lukem512]

I believe the issue is caused by the checkpointing node having been set to a depth of 0 and creating a checkpoint on a fork that was later reorganised by the main chain. Please see http://maxcoinnews.net/maxcoin-0-9-2-checkpointing-troubleshooting/

[repsorp]

enforcecheckpoint false RPC command added
still doesn't sync...

[krakerjaak]

@repsorp if you run getcheckpoint command, what's the result's subscribemode attribute's value? It should be advisory if checkpoint enforcing is disabled.

[repsorp]

subscribe attribute's value is advisory and wallet's syncing now

Big thanks

Le 01/09/2014 14:27, krakerjaak a écrit :

@repsorp https://github.com/repsorp if you run |getcheckpoint|
command, what's the result's |subscribemode| attribute's value? It
should be |advisory| if checkpoint enforcing is disabled.

—
Reply to this email directly or view it on GitHub
#44 (comment).

[krakerjaak]

Closing the issue as this seems to have been confirmed as the root cause by @lukem512. I'm a little worried of the effects of many (if not most) people disabling checkpoint enforcing on their wallet... will they re-enable it after the fork at block 600,000? if they don't, what's the longer term consequences?

[reorder]

There is currently no way to stop checkpoint at 580073 from being broadcasted, so enabling the enforcing is just not an option.

[krakerjaak]

As far as I understand it, as soon as another, more recent, checkpoint is accepted by the network and broadcasted on the right blockchain it should come back to "normal". The checkpointing code seems to only validate that the last checkpoint is valid and descendant of the genesis block.

[reorder]

Nodes that have saved checkpoint 580073 will still retranslate it, so those syncing anew will still get stuck there. Looks like only key change will invalidate it.

[lukem512]

That's what I'm investigating reorder, thanks for your input

[lukem512]

I have updated the checkpoint key and added an RPC call to reset the checkpoint. I'll have to release another version. In the mean time, disabling checkpoints is the way to proceed.