[Wallet] Add RPC call "rescanblockchain <startheight> <stopheight>" #7061

jonasschnelli · 2015-11-19T15:10:15Z

A RPC rescan command is much more flexible for the following reasons:

You can define the start and end-height
It can be called during runtime
It can work in multiwallet environment

laanwj · 2015-11-19T15:14:02Z

I'd rather have it that the API is such that an explicit rescan is never needed. Wasn't there some work on a multi-import w/ timestamps?

jonasschnelli · 2015-11-19T15:19:50Z

Yes. There is a PR (see PR description). I agree that it would be better to avoid rescans at all, although it might be complicated to catch all edge-cases and a manual trigger can help in situations where someone needs to deal with multiple/complex imports (you only want to do one rescan).

And I think some people will cancel a rescan because they want to do other stuff and/or had not considered that a rescan can take a long time. Afterward calling -rescan (from genesis) is a time consuming option.

laanwj · 2015-11-19T15:35:28Z

But manually specifying a block # to rescan from is extremely fragile... it's very easy to get this wrong.

Also, rescanning doesn't interact with pruning which will be more and more common in the future.

gmaxwell · 2015-11-19T20:29:52Z

@lannwj I thought thats part of what the height parameter here was for-- addressing pruning comparability?

petertodd · 2015-11-20T22:03:29Z

How about we call this rescanfromheight instead, to make it possible to add a rescanfromtime later if users demand? Equally, some kind of RPC call that finds the first block with a nTime after a specific time might be useful here.

Do we have a way of querying what block # is the oldest non-pruned block?

petertodd · 2015-11-20T22:04:35Z

It also occurs to me that for this usecase we might instead want to have pruning not happen automatically, but rather be an on-demand thing where the user specifies the oldest time they're interested in.

gmaxwell · 2015-11-20T22:44:10Z

So the biggest negative I personally see here is that it furthers this misunderstanding that rescan is some thing users generally need to be doing. Until we added these non-rescan imports a user initiated rescan is something that never should have been needed (and indicated a serious bug we'd like to know about if it was). As a result of the -rescan argument there is now this whole cargo cult of people that rescan every time they're scarred by a shadow. I hate to further that.

But I can't deny how really useful this will be.

petertodd · 2015-11-21T00:07:38Z

@gmaxwell A possible way around that would be to make the rescan check if you have any addresses that haven't yet been scanned in that range and error out if not. (basically make it say "no rescan needed")

gmaxwell · 2015-11-22T22:13:53Z

Hm. this could also take a stop argument, allowing you to scan single blocks or avoid rescan overlap. Also, I think all the wallet re-scanning should traverse its interval backwards-- for more instant gratification; though this would dork with the wallet transaction ordering... actually import at all breaks that, I should go talk to luke-jr about that.

promag · 2015-11-23T02:07:37Z

src/wallet/rpcdump.cpp

+    //We can't rescan beyond non-pruned blocks, stop and throw an error
+    if (fPruneMode)
+    {
+        if (fPruneMode)


Oops. A rebase issue. Thanks for point out. Fixed.

pstratem · 2015-11-24T12:17:33Z

agree with gmaxwell that this should scan a range of blocks

jonasschnelli · 2015-11-24T12:22:58Z

Agree with the stop parameter. Working on a implementation....

sipa · 2015-11-24T12:29:50Z

I would still prefer an approach that imports with birthdate instead of explicit rescanning.

jonasschnelli · 2015-11-24T15:42:56Z

Added a commit that allows providing a optional parameter with a height where the rescan should stop.

@sipa: I agree that rescan height over a key/address birthday would be nice to have (see #6570). But a explicit rescan RPC call can be useful IMO. It's trivial to maintain and it can save lots of rescan-time on the user side. But agree, it has to be considered as "experts" feature.

What about implementing a threshold for autodetecting wether the parameter is a blockheight or timestamp (similar to LOCKTIME_THRESHOLD)?

promag · 2015-11-24T15:57:52Z

@jonasschnelli see #6570 (comment) regarding your last comment.

GIJensen · 2015-12-07T06:49:13Z

ACK

mrbandrews · 2016-03-14T17:59:20Z

If this is still moving forward - I tested it a bit (including pruned mode) and it looks fine to me. One suggestion is to make the start-height a required parameter so that the user specifies "rescanblockchain 1" to scan from genesis. If the start-height is < 1 or higher than current height, throw an error.
Otherwise, ACK.

promag · 2016-05-02T13:17:54Z

src/wallet/wallet.cpp

@@ -1084,6 +1087,8 @@ int CWallet::ScanForWalletTransactions(CBlockIndex* pindexStart, bool fUpdate)
                if (AddToWalletIfInvolvingMe(tx, &block, fUpdate))
                    ret++;
            }
+            if (pindex == pindexStop)


Move to while condition?

Because it's inclusive?

jonasschnelli · 2016-07-20T12:58:59Z

Rebased.
I think there are still reasons to consider that PR. At the moment, it would really be useful. Even once we have #7551 (importmulti) it could see use cases for rescanblockchain.

sipa · 2016-08-25T12:53:25Z

This seems better #7984, but I still prefer not furthering the usage of various rescans. We need APIs that don't require users to keep track of the concept of rescanning IMHO.

jonasschnelli · 2016-08-25T12:58:14Z

I agree. Ideally, there will be no need to rescan. But in practice, rescans are sometimes required (I guess everyone who gave some users support has encountered that). IMO a rpc rescan commend with an optional hight is much more flexible then -rescan as a startup argument.

…n example 43f76f6 Add missing comma from rescanblockchain (MeshCollider) Pull request description: bitcoin#7061 forgot a comma in the HelpExampleRpc() for the rescanblockchain RPC, giving an incorrect example command output: > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "rescanblockchain", "params": [100000 120000] }' -H 'content-type: text/plain;' http://127.0.0.1:8332/ Was just missed during nit-fixing. This is a trivial fix to add that comma in. Tree-SHA512: b808f32674af585a1ddb78b25621dff0387dbad79c97d65ff61d8a9a12a94e4b8ecf03eda3f281fe439bddb6c0703c39104dbb279f1718949abd930faaa9042f

…ght> <stopheight>" 7a91ceb [QA] Add RPC based rescan test (Jonas Schnelli) c77170f [Wallet] add rescanblockchain <start_height> <stop_height> RPC command (Jonas Schnelli) Pull request description: A RPC rescan command is much more flexible for the following reasons: * You can define the start and end-height * It can be called during runtime * It can work in multiwallet environment Tree-SHA512: df67177bad6ad1d08e5a621f095564524fa3eb87204c2048ef7265e77013e4b1b29f991708f807002329a507a254f35e79a4ed28a2d18d4b3da7a75d57ce0ea5