Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add listsinceblock command for retrieving all wallet transactions in blocks after the specified block #199

Merged
merged 2 commits into from
Oct 5, 2011

Conversation

cdhowie
Copy link
Contributor

@cdhowie cdhowie commented May 6, 2011

This commit adds a "listsinceblock" RPC command that will accept a block hash and list all of the wallet transactions that have occurred after that block. If no block is specified, it will list all wallet transactions. The output also includes the block ID of the latest block on the main chain.

This is designed to simplify the process of reconciling transactions it bitcoind with another database; you store the most recent block you have seen (the "latest block" result from your last reconcile) and, on the next reconcile pass this block identifier. bitcoind will then tell you about all the transactions you need to process (possibly including transactions you have already processed, if the latest block the last time you reconciled has since been orphaned).

@gavinandresen
Copy link
Contributor

How did you test that this does the right thing through a block split/reorg?

@cdhowie
Copy link
Contributor Author

cdhowie commented May 7, 2011

I didn't, since I'm not sure how I could create such a situation with the amount of computing power I have. I believe it will do the right thing since I've read the CBlockLocator sources.

@gavinandresen
Copy link
Contributor

You should be able to pretty easily simulate block splits by setting up a small "testnet-in-a-box" network -- see https://sourceforge.net/projects/bitcoin/files/Bitcoin/testnet-in-a-box/

Process would be something like:

Send a transaction or three while generating on testnet-in-a-box. Verify that the transactions are confirmed. Then shutdown that testnet-in-a-box and start a new one.

Generate a longer chain.

Then join the two chains (restart a -nolisten testnet-in-a-box node from the shorter chain, and -connect it to the longer-chain testnet-in-a-box) and make sure listsinceblock does what you expect.

In case it doesn't, you should probably save away the shorter and longer chains (just save the datadirs somewhere), so re-testing is easy.

@sipa
Copy link
Member

sipa commented May 8, 2011

CMerkleTx's (and by extension CWalletTx's) support looking up in which block they are (see CMerkleTx::GetDepthInMainChain(CBlockIndex* &)). I believe it is better to loop over all wallet transactions, and filter them by looking up where they are in the main chain, than to loop over the whole block chain (loading it from disk), and then finding wallet transactions that match it. The block chain won't be available in client-only mode anyway.

@cdhowie
Copy link
Contributor Author

cdhowie commented May 25, 2011

Gavin, I did the test today and the command does do what I expect. This was after modifying the command's implementation per sipa's suggestion.

@brandonjank
Copy link

Patched 322rc4 with this the other day. Seems to be working, great feature. Merge :D

@pixelglow
Copy link

Might be useful to have block id against each txn, not just the last block id. For example, perhaps I'm searching for all txn's with confirmations > n. If I find some txn's with confirmations <= n, I have to note them down in the sync'ed database and recheck them later. But if we got block id's back for the txn's, we could restart the scan somewhere else other than the last block to look for these confirmations. Also would be good to get some sense of time ordering of the block id's as well.

@cdhowie
Copy link
Contributor Author

cdhowie commented Jun 28, 2011

@pixelglow: I had that in at some point and Gavin asked that I remove it. :) But it's a one line change to add it.

The transactions should be listed sorted by block. So in terms of the time that the blocks were created, you can infer that from the time the transaction happened. If all you care about is which blocks came before which blocks, then you don't even have to look at the transaction times, just their order.

@pixelglow
Copy link

I suppose there are two ways of adding block id's for txn's. You could tag each txn with the block id e.g.[ { txid: 1, block: abc }, {txid: 2, block: abc} ... ] but since there are multiple txn per block and you are already sorted by block order you can do a more normalized { block: abc; txns: [ { txid: 1}, {txid : 2} ...] }. The feature in general would make it so convenient to pick an arbitrary block to get txn's forward from e.g. if you are looking for confirmed txn only.

@gavinandresen
Copy link
Contributor

Block chain re-orgs change what blocks a given transaction is in, which is why I'm dead-set against putting them in transaction information. Transactions already have confirmations, which tells you how far back in the best-block-chain to look for the block that contains that transaction.

@cdhowie
Copy link
Contributor Author

cdhowie commented Jun 29, 2011

I'm thinking that a "minimum confirmations" parameter would solve this nicely. If you only want transactions with six confirmations or more, you call "listsinceblock (block-id) 6" and all of the recent transactions with 6+ confirmations would be reported. The "lastblock" property on the result object would then contain the hash of the sixth most recent block instead of the most recent. This way, when you run the command later, transactions that had under 6 confirmations last time will be reported if they have 6+ now.

@pixelglow
Copy link

@cdhowie that is an excellent idea and solves exactly the issue I have with confirmations. I can't see any other reason why someone would need to look at earlier blocks than the last, other than checking confirmations. (But then again, I'm a newbie when it comes to bitcoin :-) ).

@cdhowie
Copy link
Contributor Author

cdhowie commented Jul 6, 2011

I've implemented the requested change; the second parameter specifies the depth of the block you want to obtain as the "lastblock" return property. The default of 1 will return the latest block ID, while 2 will return the second-latest block ID, and so on.

The command now outputs transactions with zero confirmations in addition to those in the newer blocks so that applications can act on them in some way, such as show them as pending in a UI.

Note that the second parameter does not in any way affect which transactions are returned; if you only care about transactions with N confirms, you will pass N as the second parameter, and then filter the results down to only those with N+ confirmations. This will ensure that the next time you run the command with the new lastblock value you won't miss any transactions that have now reached N confirmations. But it will allow applications that want to specially handle transactions with <N confirmations to do so as well. So this command neatly hits both use-cases.

Edit: I've also rebased against master.

…lastblock return property to be targeted to the block with the specified depth
@pixelglow
Copy link

I've integrated this into my bitcoin exchange (currently only as a JSON API to the backend) and it works quite well in replaying transactions into the PostgreSQL backend. Thanks. Can't wait to see this in the mainline.

http://forum.bitcoin.org/index.php?topic=29057.0

@gavinandresen gavinandresen merged commit 76aed01 into bitcoin:master Oct 5, 2011
dexX7 pushed a commit to dexX7/bitcoin that referenced this pull request Nov 15, 2014
sipa added a commit to sipa/bitcoin that referenced this pull request Mar 27, 2015
1897b8e Merge pull request bitcoin#229
efc571c Add simple testcases for signing with rfc6979 extra entropy.
1573a10 Add ability to pass extra entropy to rfc6979
3087bc4 Merge pull request bitcoin#228
d9b9f11 Merge pull request bitcoin#218
0065a8f Eliminate multiple-returns from secp256k1.c.
354ffa3 Make secp256k1_ec_pubkey_create reject oversized secrets.
27bc131 Silence some warnings from pedantic static analysis tools, improve compatibility with C++.
3b7ea63 Merge pull request bitcoin#221
f789c5b Merge pull request bitcoin#215
4bc273b Merge pull request bitcoin#222
137a8ec Merge pull request bitcoin#216
7c3771d Disable overlength-strings warnings.
8956111 use 128-bit hex seed
02efd06 Use RFC6979 for test PRNGs
ae55e85 Use faster byteswapping and avoid alignment-increasing casts.
443cd4b Get rid of hex format and some binary conversions
0bada0e Merge bitcoin#214: Improve signing API documentation & specification
8030d7c Improve signing API documentation & specification
7b2fc1c Merge bitcoin#213: Removed gotos, which are hard to trace and maintain.
11690d3 Removed gotos, which are hard to trace and maintain.
122a1ec Merge pull request bitcoin#205
035406d Merge pull request bitcoin#206
2d4cd53 Merge pull request bitcoin#161
34b898d Additional comments for the testing PRNG and a seeding fix.
6efd6e7 Some comments explaining some of the constants in the code.
ffccfd2 x86_64 assembly optimization for scalar_4x64
67cbdf0 Merge pull request bitcoin#207
039723d Benchmarks for all internal operations
6cc8425 Include a comment on secp256k1_ecdsa_sign explaining low-s.
f88343f Merge pull request bitcoin#203
d61e899 Add group operation counts
2473f17 Merge pull request bitcoin#202
b5bbce6 Some readme updates, e.g. removal of the GMP field.
f0d851e Merge pull request bitcoin#201
a0ea884 Merge pull request bitcoin#200
f735446 Convert the rest of the codebase to C89.
bf2e1ac Convert tests to C89. (also fixes a use of bare "inline" in field)
fc8285f Merge pull request bitcoin#199
fff412e Merge pull request bitcoin#197
4be8d6f Centralize the definition of uint128_t and use it uniformly.
d9543c9 Switch scalar code to C89.
fcc48c4 Remove the non-storage cmov
55422b6 Switch ecmult_gen to use storage types
41f8455 Use group element storage type in EC multiplications
e68d720 Add group element storage type
ff889f7 Field storage type
7137be8 Merge pull request bitcoin#196
0768bd5 Get rid of variable-length hex string conversions
e84e761 Merge pull request bitcoin#195
792bcdb Covert several more files to C89.
45cdf44 Merge pull request bitcoin#193
17db09e Merge pull request bitcoin#194
402878a fix ifdef/ifndef
25b35c7 Convert field code to strict C89 (+ long long, +__int128)
3627437 C89 nits and dead code removal.
a9f350d Merge pull request bitcoin#191
4732d26 Convert the field/group/ecdsa constant initialization to static consts
19f3e76 Remove unused secp256k1_fe_inner_{start, stop} functions
f1ebfe3 Convert the scalar constant initialization to static consts

git-subtree-dir: src/secp256k1
git-subtree-split: 1897b8e
TheBlueMatt added a commit to TheBlueMatt/bitcoin that referenced this pull request Oct 20, 2015
This contains the following two merges, as well as a few other changes:

Squashed 'src/secp256k1/' changes from 22f60a6..18f9f08

18f9f08 Pedersen commitments, borromean ring signatures, and ZK range proofs.
5161227 Add benchmark for ECDH multiplication
c40cb72 Expose API for constant time point multiplication
40adc7a Add constant-time `secp256k1_ecdh_point_multiply` for ECDH
ff9a397 Add zero/one tests to ecmult
729badf Merge pull request bitcoin#210
2d5a186 Apply effective-affine trick to precomp
4f9791a Effective affine addition in EC multiplication

git-subtree-dir: src/secp256k1
git-subtree-split: 18f9f0821c5a7795b760763c99d545af96a22775

Squashed 'src/secp256k1/' changes from b0210a9..22f60a6

22f60a6 Merge pull request bitcoin#245
61c1b1e Merge pull request bitcoin#190
d227579 Add scalar blinding and a secp256k1_context_randomize() call.
c146b4a Add bench_internal to gitignore.
9c4fb23 Add a secp256k1_fe_cmov unit test.
426fa52 Merge pull request bitcoin#243
d505a89 Merge pull request bitcoin#244
2d2707a travis: test i686 builds with gmp
cf7f702 travis: update to new build infrastructure
bb0ea50 Replace set/add with cmov in secp256k1_gej_add_ge.
f3d3519 Merge pull request bitcoin#241
5c2a4fa Fix memory leak in context unit test
14aacdc Merge pull request bitcoin#239
93226a5 secp256k1.c: Add missing DEBUG_CHECKs for sufficiently capable contexts
6099220 Merge pull request bitcoin#237
6066bb6 Fix typo: avg -> max
9688030 Merge pull request bitcoin#236
d899b5b Expose ability to deep-copy a context
3608c7f Merge pull request bitcoin#208
a9b6595 [API BREAK] Introduce explicit contexts
a0d3b89 Merge pull request bitcoin#233
9e8d89b Merge pull request bitcoin#234
65e70e7 Merge pull request bitcoin#235
5098f62 Improve documentation formatting consistency
4450e24 Add a comment about the avoidance of secret data in array indexes.
6534ee1 initialize variable
d5b53aa Merge pull request bitcoin#232
c01df1a Avoid some implicit type conversions to make C++ compilers happy.
bfe96ba Merge pull request bitcoin#231
33270bf Add a couple comments pointing to particular sections of RFC6979.
41603aa Merge pull request bitcoin#230
2632019 Brace all the if/for/while.
1897b8e Merge pull request bitcoin#229
efc571c Add simple testcases for signing with rfc6979 extra entropy.
1573a10 Add ability to pass extra entropy to rfc6979
3087bc4 Merge pull request bitcoin#228
d9b9f11 Merge pull request bitcoin#218
0065a8f Eliminate multiple-returns from secp256k1.c.
354ffa3 Make secp256k1_ec_pubkey_create reject oversized secrets.
27bc131 Silence some warnings from pedantic static analysis tools, improve compatibility with C++.
3b7ea63 Merge pull request bitcoin#221
f789c5b Merge pull request bitcoin#215
4bc273b Merge pull request bitcoin#222
137a8ec Merge pull request bitcoin#216
7c3771d Disable overlength-strings warnings.
8956111 use 128-bit hex seed
02efd06 Use RFC6979 for test PRNGs
ae55e85 Use faster byteswapping and avoid alignment-increasing casts.
443cd4b Get rid of hex format and some binary conversions
0bada0e Merge bitcoin#214: Improve signing API documentation & specification
8030d7c Improve signing API documentation & specification
7b2fc1c Merge bitcoin#213: Removed gotos, which are hard to trace and maintain.
11690d3 Removed gotos, which are hard to trace and maintain.
122a1ec Merge pull request bitcoin#205
035406d Merge pull request bitcoin#206
2d4cd53 Merge pull request bitcoin#161
34b898d Additional comments for the testing PRNG and a seeding fix.
6efd6e7 Some comments explaining some of the constants in the code.
ffccfd2 x86_64 assembly optimization for scalar_4x64
67cbdf0 Merge pull request bitcoin#207
039723d Benchmarks for all internal operations
6cc8425 Include a comment on secp256k1_ecdsa_sign explaining low-s.
f88343f Merge pull request bitcoin#203
d61e899 Add group operation counts
2473f17 Merge pull request bitcoin#202
b5bbce6 Some readme updates, e.g. removal of the GMP field.
f0d851e Merge pull request bitcoin#201
a0ea884 Merge pull request bitcoin#200
f735446 Convert the rest of the codebase to C89.
bf2e1ac Convert tests to C89. (also fixes a use of bare "inline" in field)
fc8285f Merge pull request bitcoin#199
fff412e Merge pull request bitcoin#197
4be8d6f Centralize the definition of uint128_t and use it uniformly.
d9543c9 Switch scalar code to C89.
fcc48c4 Remove the non-storage cmov
55422b6 Switch ecmult_gen to use storage types
41f8455 Use group element storage type in EC multiplications
e68d720 Add group element storage type
ff889f7 Field storage type
7137be8 Merge pull request bitcoin#196
0768bd5 Get rid of variable-length hex string conversions
e84e761 Merge pull request bitcoin#195
792bcdb Covert several more files to C89.
45cdf44 Merge pull request bitcoin#193
17db09e Merge pull request bitcoin#194
402878a fix ifdef/ifndef
25b35c7 Convert field code to strict C89 (+ long long, +__int128)
3627437 C89 nits and dead code removal.
a9f350d Merge pull request bitcoin#191
4732d26 Convert the field/group/ecdsa constant initialization to static consts
19f3e76 Remove unused secp256k1_fe_inner_{start, stop} functions
f1ebfe3 Convert the scalar constant initialization to static consts
50cc6ab Merge pull request bitcoin#178
941e221 Add tests for handling of the nonce function in signing.
10c81ff Merge pull request bitcoin#177
7688e34 Add magnitude limits to secp256k1_fe_verify to ensure that it's own tests function correctly.
4ee4f7a Merge pull request bitcoin#176
70ae0d2 Use secp256k1_fe_equal_var in secp256k1_fe_sqrt_var.
7767b4d Merge pull request bitcoin#175
9ab9335 Add a reference consistency test to ge_tests.
60571c6 Rework group tests
d26e26f Avoid constructing an invalid signature with probability 1:2^256.
b450c34 Merge pull request bitcoin#163
d57cae9 Merge pull request bitcoin#154
49ee0db Add _normalizes_to_zero_var variant
eed599d Add _fe_normalizes_to_zero method
d7174ed Weak normalization for secp256k1_fe_equal
0295f0a weak normalization
bbd5ba7 Use rfc6979 as default nonce generation function
b37fbc2 Implement SHA256 / HMAC-SHA256 / RFC6979.
c6e7f4e [API BREAK] Use a nonce-generation function instead of a nonce
cf0c48b Merge pull request bitcoin#169
603c33b Make signing fail if a too small buffer is passed.
6d16606 Merge pull request bitcoin#168
7277fd7 Remove GMP field implementation
e99c4c4 Merge pull request bitcoin#123
13278f6 Add explanation about how inversion can be avoided
ce7eb6f Optimize verification: avoid field inverse
a098f78 Merge pull request bitcoin#160
38acd01 Merge pull request bitcoin#165
6a59012 Make git ignore bench_recover when configured with benchmark enabled
1ba4a60 Configure options reorganization
3c0f246 Merge pull request bitcoin#157
808dd9b Merge pull request bitcoin#156
8dc75e9 Merge pull request bitcoin#158
28ade27 build: nuke bashisms
5190079 build: use subdir-objects for automake
8336040 build: disable benchmark by default
bccaf86 Merge pull request bitcoin#150
2a53a47 Merge pull request bitcoin#151
5f5a31f Merge pull request bitcoin#149
3907277 Merge pull request bitcoin#142
a3e0611 Enable tests in x86 travis builds
45da235 x86 builder
8bb0e93 Merge pull request bitcoin#155
971fe81 build: fix openssl detection for cross builds
f22d73e Explicitly access %0..%2 as 64-bit so we use the right registers for x32 ABI
e66d4d6 Avoid the stack in assembly and use explicit registers
cf7b2b4 Fix ECDSA message hashes to 32 bytes
056ad31 Really compile with -O3 by default
74ad63a Merge pull request bitcoin#146
9000458 Merge pull request bitcoin#145
1f46b00 build: fix __builtin_expect detection for clang
aaba2e0 Merge pull request bitcoin#136
8a0775c Merge pull request bitcoin#144
ee1eaa7 Merge pull request bitcoin#141
c88e2b8 Compile with -O3 by default
6558a26 Make the benchmarks print out stats
000bdf6 Rename bench_verify to bench_recovery
7c6fed2 Add a few more additional tests.
992e03b travis: add clang to the test matrix
b43b79a Merge pull request bitcoin#143
e06a924 Include time.h header for time().
8d11164 Add some additional tests.
3545627 Merge pull request bitcoin#118
6a9901e Merge pull request bitcoin#137
376b28b Merge pull request bitcoin#128
1728806 Merge pull request bitcoin#138
a5759c5 Check return value of malloc
39bd94d Variable time normalize
ad86bdf Merge pull request bitcoin#140
54b768c Another redundant secp256k1_fe_normalize
69dcaab Merge pull request bitcoin#139
1c29f2e Remove redundant secp256k1_fe_normalize from secp256k1_gej_add_ge_var.
2b9388b Remove unused secp256k1_fe_inv_all
f461b76 Allocate precomputation arrays on the heap
b2c9681 Make {mul,sqr}_inner use the same argument order as {mul,sqr}
6793505 Convert YASM code into inline assembly
f048615 Rewrite field assembly to match the C version
3ce74b1 Tweak precomputed table size for G

git-subtree-dir: src/secp256k1
git-subtree-split: 22f60a6
btcdrak pushed a commit to btcdrak/bitcoin that referenced this pull request Nov 28, 2015
deadalnix pushed a commit to deadalnix/bitcoin that referenced this pull request Jan 3, 2017
deadalnix pushed a commit to deadalnix/bitcoin that referenced this pull request Jan 19, 2017
fcc48c4 Remove the non-storage cmov (Pieter Wuille)
55422b6 Switch ecmult_gen to use storage types (Pieter Wuille)
41f8455 Use group element storage type in EC multiplications (Pieter Wuille)
e68d720 Add group element storage type (Pieter Wuille)
ff889f7 Field storage type (Pieter Wuille)
lateminer pushed a commit to lateminer/bitcoin that referenced this pull request Dec 9, 2017
…ional_fee_option

[issue#198] No more GUI option for "additional transaction fee" :)
lateminer pushed a commit to lateminer/bitcoin that referenced this pull request Dec 9, 2017
lateminer pushed a commit to lateminer/bitcoin that referenced this pull request Dec 9, 2017
classesjack pushed a commit to classesjack/bitcoin that referenced this pull request Jan 2, 2018
nining pushed a commit to nining/bitcoin that referenced this pull request Jan 3, 2018
rajarshimaitra pushed a commit to rajarshimaitra/bitcoin that referenced this pull request Aug 5, 2021
- peer-to-peer better than "peer 2 peer" , in acronyms p2p is ok
- English language guidelines recommend numbers up to 12 to be spelled out. 4 --> four
- minor additions/clarifications
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants