Unobfuscate chainstate data in CCoinsViewDB::GetStats

[jamesob]

Per the thread on the mailing list, we missed (at least) one use of CLevelDBWrapper when adding chainstate obfuscation.

Preferably, this PR (or a followup) will also add automated tests that prevent future bugs of this kind. Subsequently we should also introduce an abstraction that prevents this sort of leak when performing iteration with CLevelDBWrapper.

cc @domob1812 @dexX7

[dexX7]

Tested, "gettxoutsetinfo" now works as expected.

Based on a quick scan for value() it looks like the only other place where raw iterator data is used, is related to the block index DB: src/txdb.cpp#L205 (which isn't obfuscated)

[jamesob]

@dexX7 thanks for testing. I'm going to go ahead and apply the same treatment to the block index DB as well so that we won't have an issue if we decide to obfuscate it in the future.

[sipa]

jamesob: I think the obfuscation/deobfuscation should be done in Leveldbwrapper, not in the individual clients.

[jamesob]

Okay @sipa, I'm thinking I'll just subclass leveldb's Iterator for the purposes of CLevelDBWrapper, then return that type from NewIterator.

[sipa]

jamesob: see the code in #2802 for that :)

[jamesob]

@sipa ha! and like magic... wish writing code was always this easy ;). Will cherry-pick and layer on modifications if possible.

[laanwj]

Good catch @domob1812 , thanks for fix @jamesob @sipa

I'm getting an issue starting bitcoind on my node w/ obfuscation, though:
assertion "hashPrevBlock == view.GetBestBlock()" failed: file "main.cpp", line 1694, function "ConnectBlock" Abort trap (core dumped)

Not sure whether it's related to this, will investigate further. From debug.log:

2015-10-08 14:05:08 Using obfuscation key for /home/orion/.bitcoin/chainstate: 0000000000000000
...
2015-10-08 14:05:19 LoadBlockIndexDB: transaction index disabled
2015-10-08 14:05:20 Initializing databases...

Looks like my obfuscation key was lost, and it's now trying to interpret the data in raw.
debug.log unfortunately doesn't go back far enough to check what the key was before.

[laanwj]

Found the issue. Indeed has nothing to do with this pull. On my node I was using a previous version of your patch, using a different OBFUSCATE_KEY_KEY. So no, the key was not lost, just misplaced. This shouldn't affect anyone starting with master.

 // Prefixed with null character to avoid collisions with other keys
-const std::string CLevelDBWrapper::OBFUSCATE_KEY_KEY = "\000obfuscate_key";
+//
+// We must use a string constructor which specifies length so that we copy
+// past the null-terminator.
+const std::string CLevelDBWrapper::OBFUSCATE_KEY_KEY("\000obfuscate_key", 14);

[laanwj]

Tested ACK. gettxoutsetinfo works again after this.

[jamesob]

@laanwj good to hear. Thanks for testing.

[jamesob]

I've added some test coverage on the CLevelDB utilities. Provided travis is happy, I'm done adding changes.

[dexX7]

Is this always deterministic?

[dexX7]

Just to clarify: unfortunally it's not (due to the potentially different time to create the blocks + random address generation).

Even if it were, I'm not sure, if such a strong coupling is desired. You might setup the chain or a few blocks manually and use util.set_node_times + pre-generated key-pairs, but this seems pretty laborious.

A middle ground could be to check, whether the JSON keys are available (as in your previous version), and then do some sanity checks, such as res['height'] >= 0, len(res['bestblock']) == 64, [...].

[jamesob]

@sipa any comments here?

[sipa]

Code ACK, I haven't reviewed the tests.

[jamesob]

Ping on this. I think it's ready for merge.

[laanwj]

@jamesob Yes going to merge this now, I've been sidetracked by the upnp vulnerabilty the last few days.