New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[rpc] Allow fetching tx directly from specified block in getrawtransaction #10275
Conversation
src/rpc/rawtransaction.cpp
Outdated
uint256 hash; | ||
int64_t blockHeight = -1; | ||
bool inMainChain = true; | ||
auto p = request.params[0].get_str().find(':'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This functionality could come in useful.
As for the API I prefer to not do any string combining/parsing here, this makes the API less clean to work with at least in my experience. I'd prefer to add an optional (can be null or missing) fromblock
argument.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK. I've really wanted this before.
Allowing it work on orphan blocks is an interesting idea. I'm a little less sure about that-- I think it could allow it to return transactions that have never been validated, which would be somewhat surprising.
Nice feature! |
Fair enough, I'll add a blockhash argument instead. I was kind of toying with the idea of a new standard for referencing transactions which included the block height (not hash) so everyone could always find a tx presuming they had the block in question, and that thought sort of seeped in here. |
I just realized my logic was flawed on this. I am passing the block height only to the GetTransaction method, which means it will always pick the active chain at the given height. Either I throw when the block is not in the main chain (i.e. no support for orphaned blocks) or I move the height determine logic over to validation.cpp. I am leaning towards the latter, but feedback welcome. Edit: passing CBlockIndex to GetTransaction seems like a great way to do this. Going with that. |
dd3da7e
to
456d6f8
Compare
The code now works as advertised (see updated OP).
|
91f783d
to
23be83e
Compare
src/rpc/rawtransaction.cpp
Outdated
fVerbose = true; | ||
} | ||
fVerbose = (request.params[1].get_int() != 0); | ||
} else if (request.params[1].isBool()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: You can replace this and the next three lines with:
} else {
fVerbose = (request.params[1].get_bool());
}
since get_bool()
does the type testing for you and throws the JSONRPCError if the type isn't a VBOOL.
Up to you whether you think that's clearer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ohh, good point! Thanks.
Edit: I don't want it to throw for null
values though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I end up with
if (request.params[1].isNum()) {
fVerbose = (request.params[1].get_int() != 0);
} else if (!request.params[1].isNull()) {
fVerbose = (request.params[1].get_bool());
}
utACK, but I think this deserves a new functional test case. |
23be83e
to
ccb969a
Compare
@jnewbery I agree. Will get to work on that.
|
5b9b2e4
to
9f980f6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tested ACK the integration test in a6b84618ad04dccc628aa4905fced4784a62895a with a couple of nits.
test/functional/rawtransactions.py
Outdated
# make a tx by sending then generate 2 blocks; block1 has the tx in it, | ||
# presumably | ||
tx = self.nodes[2].sendtoaddress(self.nodes[1].getnewaddress(), 1) | ||
[ block1, block2 ] = self.nodes[2].generate(2) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: no need for brackets here. The following should do:
block1, block2 = self.nodes[2].generate(2)
test/functional/rawtransactions.py
Outdated
[ block1, block2 ] = self.nodes[2].generate(2) | ||
self.sync_all() | ||
# We should be able to get the raw transaction by providing the correct block | ||
assert self.nodes[0].getrawtransaction(tx, True, block1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I think it's better to assert on the actual value here (ie verify that the getrawtransaction returned the correct transaction rather than returned anything). The following should do that:
assert_equal(self.nodes[0].getrawtransaction(tx, True, block1)['txid'], tx)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jnewbery Good points, thanks. Addressed!
Is this still needed after #8704? |
@kallewoof #8704 does not require indexing either |
Yeah, sorry, I meant that this is a way to get a specific transaction if you know the block hash, whereas #8704 shows you all transactions in the entire block. You get the info, but you have to wade through stuff to get it. |
In both cases the whole block has to be loaded from disk, and parsed, and searched linearly. The difference is whether the linear search step happens on the server or client. I think both #8704 and this can be useful, but have to agree there's only superficial difference. |
The difference between the two is the serialization and transmission and parsing of ~5MB of JSON data vs a few hundred bytes of hex encoded data. That's a 1,000x difference on the client side, and the same absolute improvement on the server -- although as a multiplier it'd be less since as you note the server still has to parse the block from disk. That's a nontrivial performance difference. (Also, this would have greatly helped me in the past so another +1 from me.) |
Agree with @maaku - simply encoding the data into 5MB of json could be time-consuming, during which time the cs_main lock is held. Having a command to return just a single transaction from a block seems very useful. |
src/rpc/rawtransaction.cpp
Outdated
|
||
"\nNOTE: By default this function only works for mempool transactions. If the -txindex option is\n" | ||
"enabled, it also works for blockchain transactions.\n" | ||
"enabled, it also works for blockchain transactions. If the block hash is known, it can be provided\n" | ||
"for nodes without -txindex.\n" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should probably mention that it MUST be in that block in this case...
src/rpc/rawtransaction.cpp
Outdated
if (!blockHash.IsNull()) { | ||
BlockMap::iterator it = mapBlockIndex.find(blockHash); | ||
if (it == mapBlockIndex.end()) { | ||
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Block hash not found in chain"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove "in chain". Maybe "in database"?
src/rpc/rawtransaction.cpp
Outdated
|
||
"\nResult (if verbose is not set or set to false):\n" | ||
"\"data\" (string) The serialized, hex-encoded data for 'txid'\n" | ||
|
||
"\nResult (if verbose is set to true):\n" | ||
"{\n" | ||
" \"inMainChain\": b, (bool) Whether transaction is in the main chain or not. Only visible when specifying block hash\n" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whether the block specified is in the main chain or not... This could be false with the tx being still in the main chain!
…nsaction from a block directly. Github-Pull: bitcoin#10275 Rebased-From: 4d006e785454e20cf0000b04e175d5e142b12e3f
… accepted). Github-Pull: bitcoin#10275 Rebased-From: 5b389a88b0e9ab92f103f78c9b063f678e8afa2f
utACK 434526a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK 434526a
); | ||
|
||
LOCK(cs_main); | ||
|
||
bool in_active_chain = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Should not be initialized, to aid static analysers.
Strike that. gcc would throw a warning.
Sorry for the sloppy review. I messed up the parameter order when testing. Also gcc complains about my nit-fix. Tested ACK now:
|
utACK 434526a |
…n getrawtransaction 434526a [test] Add tests for getrawtransaction with block hash. (Karl-Johan Alm) b167951 [rpc] Allow getrawtransaction to take optional blockhash to fetch transaction from a block directly. (Karl-Johan Alm) a5f5a2c [rpc] Fix fVerbose parsing (remove excess if cases). (Karl-Johan Alm) Pull request description: [Reviewer hint: use [?w=1](https://github.com/bitcoin/bitcoin/pull/10275/files?w=1) to avoid seeing a bunch of indentation changes.] Presuming a user knows the block hash of the block containing a given transaction, this PR allows them to fetch the raw transaction, even without `-txindex`. It also enables support for getting transactions that are in orphaned blocks. Note that supplying a block hash will override mempool and txindex support in `GetTransaction`. The rationale behind this is that a transaction may be in multiple places (orphaned blocks) and if the user supplies an explicit block hash it should be adhered to. ```Bash $ # a41.. is a tx inside an orphan block ..3c6f.. -- first try getting it normally $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1 error code: -5 error message: No such mempool transaction. Use -txindex to enable blockchain transaction queries. Use gettransaction for wallet transactions. $ # now try with block hash $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1 0000000000000000003c6fe479122bfa4a9187493937af1734e1e5cd9f198ec7 { "hex": "01000000014e7e81144e42f6d65550e59b715d470c9301fd7ac189[...]90488ac00000000", "inMainChain": false, "txid": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79", "hash": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79", "size": 225, [...] } $ # another tx 6c66... in block 462000 $ ./bitcoin-cli getrawtransaction 6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735 1 00000000000000000217f2c12922e321f6d4aa933ce88005a9a493c503054a40 { "hex": "0200000004d157[...]88acaf0c0700", "inMainChain": true, "txid": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735", "hash": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735", "size": 666, [...] } $ ``` Tree-SHA512: 279be3818141edd3cc194a9ee65929331920afb30297ab2d6da07293a2d7311afee5c8b00c6457477d9f1f86e86786a9b56878ea3ee19fa2629b829d042d0cda
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Post-merge tested ACK. I have one style nit, which I can open a tidy-up PR for if people agree.
} | ||
|
||
if (!request.params[2].isNull()) { | ||
uint256 blockhash = ParseHashV(request.params[2], "parameter 3"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: these lines are confusing for me:
uint256 blockhash = ParseHashV(request.params[2], "parameter 3");
if (!blockhash.IsNull()) {
ParseHashV()
can either throw or return a uint256. The only way it'll return a null uint256 is if the input is a 64 length string of 0
s. That can't be a valid block (if someone finds a block that hashes to zero then they win Bitcoin The Game and we can all stop playing), so I think that this if condition is unnecessary - if parameter 3 is all zeroes, we should throw with the error that 0x0 is an invalid block.
I don't think this is a huge problem - if all zeroes is supplied as the block hash, we'll just drop through and search for the transaction in the {mempool , blockchain (if fTxIndex) , UTXO set} as if no block hash had been provided. However, it's a bit confusing for the code reader since it implies to me that we're expecting ParseHashV()
to return a null value if it fails to parse the hash.
fa4c16d qa: Add getrawtransaction in_active_chain=False test (MarcoFalke) Pull request description: #10275 accidentally forgot to add a test for `in_active_chain==False`. This adds a test and also removes the special casing of `blockhash.IsNull()`, which makes no sense imo. Tree-SHA512: 6c51295820b3dcd53b0b48020ab2b8c8f5864cd5061ddab2b35d35d643eb3e60ef95ff20c06c985a2e47f7080e82f27f3e00ee61c85dce627776d5ea6febee8f
For reference, This was proposed (and closed) in #2421. |
…ction (#1612) bitcoin/bitcoin#10275 bitcoin/bitcoin#9042 `GetTransaction` changes are easier to review ignoring whitespace changes (add `?w=1` to the PR's URL)
…block in getrawtransaction 434526a [test] Add tests for getrawtransaction with block hash. (Karl-Johan Alm) b167951 [rpc] Allow getrawtransaction to take optional blockhash to fetch transaction from a block directly. (Karl-Johan Alm) a5f5a2c [rpc] Fix fVerbose parsing (remove excess if cases). (Karl-Johan Alm) Pull request description: [Reviewer hint: use [?w=1](https://github.com/bitcoin/bitcoin/pull/10275/files?w=1) to avoid seeing a bunch of indentation changes.] Presuming a user knows the block hash of the block containing a given transaction, this PR allows them to fetch the raw transaction, even without `-txindex`. It also enables support for getting transactions that are in orphaned blocks. Note that supplying a block hash will override mempool and txindex support in `GetTransaction`. The rationale behind this is that a transaction may be in multiple places (orphaned blocks) and if the user supplies an explicit block hash it should be adhered to. ```Bash $ # a41.. is a tx inside an orphan block ..3c6f.. -- first try getting it normally $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1 error code: -5 error message: No such mempool transaction. Use -txindex to enable blockchain transaction queries. Use gettransaction for wallet transactions. $ # now try with block hash $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1 0000000000000000003c6fe479122bfa4a9187493937af1734e1e5cd9f198ec7 { "hex": "01000000014e7e81144e42f6d65550e59b715d470c9301fd7ac189[...]90488ac00000000", "inMainChain": false, "txid": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79", "hash": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79", "size": 225, [...] } $ # another tx 6c66... in block 462000 $ ./bitcoin-cli getrawtransaction 6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735 1 00000000000000000217f2c12922e321f6d4aa933ce88005a9a493c503054a40 { "hex": "0200000004d157[...]88acaf0c0700", "inMainChain": true, "txid": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735", "hash": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735", "size": 666, [...] } $ ``` Tree-SHA512: 279be3818141edd3cc194a9ee65929331920afb30297ab2d6da07293a2d7311afee5c8b00c6457477d9f1f86e86786a9b56878ea3ee19fa2629b829d042d0cda
…test fa4c16d qa: Add getrawtransaction in_active_chain=False test (MarcoFalke) Pull request description: bitcoin#10275 accidentally forgot to add a test for `in_active_chain==False`. This adds a test and also removes the special casing of `blockhash.IsNull()`, which makes no sense imo. Tree-SHA512: 6c51295820b3dcd53b0b48020ab2b8c8f5864cd5061ddab2b35d35d643eb3e60ef95ff20c06c985a2e47f7080e82f27f3e00ee61c85dce627776d5ea6febee8f
…block in getrawtransaction 434526a [test] Add tests for getrawtransaction with block hash. (Karl-Johan Alm) b167951 [rpc] Allow getrawtransaction to take optional blockhash to fetch transaction from a block directly. (Karl-Johan Alm) a5f5a2c [rpc] Fix fVerbose parsing (remove excess if cases). (Karl-Johan Alm) Pull request description: [Reviewer hint: use [?w=1](https://github.com/bitcoin/bitcoin/pull/10275/files?w=1) to avoid seeing a bunch of indentation changes.] Presuming a user knows the block hash of the block containing a given transaction, this PR allows them to fetch the raw transaction, even without `-txindex`. It also enables support for getting transactions that are in orphaned blocks. Note that supplying a block hash will override mempool and txindex support in `GetTransaction`. The rationale behind this is that a transaction may be in multiple places (orphaned blocks) and if the user supplies an explicit block hash it should be adhered to. ```Bash $ # a41.. is a tx inside an orphan block ..3c6f.. -- first try getting it normally $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1 error code: -5 error message: No such mempool transaction. Use -txindex to enable blockchain transaction queries. Use gettransaction for wallet transactions. $ # now try with block hash $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1 0000000000000000003c6fe479122bfa4a9187493937af1734e1e5cd9f198ec7 { "hex": "01000000014e7e81144e42f6d65550e59b715d470c9301fd7ac189[...]90488ac00000000", "inMainChain": false, "txid": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79", "hash": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79", "size": 225, [...] } $ # another tx 6c66... in block 462000 $ ./bitcoin-cli getrawtransaction 6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735 1 00000000000000000217f2c12922e321f6d4aa933ce88005a9a493c503054a40 { "hex": "0200000004d157[...]88acaf0c0700", "inMainChain": true, "txid": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735", "hash": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735", "size": 666, [...] } $ ``` Tree-SHA512: 279be3818141edd3cc194a9ee65929331920afb30297ab2d6da07293a2d7311afee5c8b00c6457477d9f1f86e86786a9b56878ea3ee19fa2629b829d042d0cda
…test fa4c16d qa: Add getrawtransaction in_active_chain=False test (MarcoFalke) Pull request description: bitcoin#10275 accidentally forgot to add a test for `in_active_chain==False`. This adds a test and also removes the special casing of `blockhash.IsNull()`, which makes no sense imo. Tree-SHA512: 6c51295820b3dcd53b0b48020ab2b8c8f5864cd5061ddab2b35d35d643eb3e60ef95ff20c06c985a2e47f7080e82f27f3e00ee61c85dce627776d5ea6febee8f
…block in getrawtransaction 434526a [test] Add tests for getrawtransaction with block hash. (Karl-Johan Alm) b167951 [rpc] Allow getrawtransaction to take optional blockhash to fetch transaction from a block directly. (Karl-Johan Alm) a5f5a2c [rpc] Fix fVerbose parsing (remove excess if cases). (Karl-Johan Alm) Pull request description: [Reviewer hint: use [?w=1](https://github.com/bitcoin/bitcoin/pull/10275/files?w=1) to avoid seeing a bunch of indentation changes.] Presuming a user knows the block hash of the block containing a given transaction, this PR allows them to fetch the raw transaction, even without `-txindex`. It also enables support for getting transactions that are in orphaned blocks. Note that supplying a block hash will override mempool and txindex support in `GetTransaction`. The rationale behind this is that a transaction may be in multiple places (orphaned blocks) and if the user supplies an explicit block hash it should be adhered to. ```Bash $ # a41.. is a tx inside an orphan block ..3c6f.. -- first try getting it normally $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1 error code: -5 error message: No such mempool transaction. Use -txindex to enable blockchain transaction queries. Use gettransaction for wallet transactions. $ # now try with block hash $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1 0000000000000000003c6fe479122bfa4a9187493937af1734e1e5cd9f198ec7 { "hex": "01000000014e7e81144e42f6d65550e59b715d470c9301fd7ac189[...]90488ac00000000", "inMainChain": false, "txid": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79", "hash": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79", "size": 225, [...] } $ # another tx 6c66... in block 462000 $ ./bitcoin-cli getrawtransaction 6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735 1 00000000000000000217f2c12922e321f6d4aa933ce88005a9a493c503054a40 { "hex": "0200000004d157[...]88acaf0c0700", "inMainChain": true, "txid": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735", "hash": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735", "size": 666, [...] } $ ``` Tree-SHA512: 279be3818141edd3cc194a9ee65929331920afb30297ab2d6da07293a2d7311afee5c8b00c6457477d9f1f86e86786a9b56878ea3ee19fa2629b829d042d0cda
…test fa4c16d qa: Add getrawtransaction in_active_chain=False test (MarcoFalke) Pull request description: bitcoin#10275 accidentally forgot to add a test for `in_active_chain==False`. This adds a test and also removes the special casing of `blockhash.IsNull()`, which makes no sense imo. Tree-SHA512: 6c51295820b3dcd53b0b48020ab2b8c8f5864cd5061ddab2b35d35d643eb3e60ef95ff20c06c985a2e47f7080e82f27f3e00ee61c85dce627776d5ea6febee8f
Add blockhash parameter to getrawtransaction Code ported manually from bitcoin/bitcoin#10275 in attempt to fix #4475
3ffa282 [RPC] Update getrawtransaction warning message (random-zebra) Pull request description: bitcoin#9520+bitcoin#10275 have already been ported (back in #812), but the warning message in `getrawtransaction` help has not been updated yet. ACKs for top commit: furszy: utACK 3ffa282 Fuzzbawls: utACK 3ffa282 Tree-SHA512: c406b4cf691517919c9911915737e9b97dcde5959856889ed5b6a501bd5073dfe9179e0663d25f34ccb2d536755e5d153b1362aecc81e445d604e65b74ac797f
[Reviewer hint: use ?w=1 to avoid seeing a bunch of indentation changes.]
Presuming a user knows the block hash of the block containing a given transaction, this PR allows them to fetch the raw transaction, even without
-txindex
. It also enables support for getting transactions that are in orphaned blocks.Note that supplying a block hash will override mempool and txindex support in
GetTransaction
. The rationale behind this is that a transaction may be in multiple places (orphaned blocks) and if the user supplies an explicit block hash it should be adhered to.