Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RPC: Improve how optional indices are managed #392

Closed
wants to merge 72 commits into from
Closed

RPC: Improve how optional indices are managed #392

wants to merge 72 commits into from

Conversation

adolg
Copy link

@adolg adolg commented Oct 29, 2017

Currently, each index like txindex needs a new command line or config file argument,
and may require a time-consuming -reindex-chainstate or -reindex to be enabled.

Bitcore/Litecore branch with -addressindex introduces a handful of new indices,
each with its own argument; if we were to limit those indices to a part of blockchain
starting with block at height N, we would have to introduce even more arguments.

This patch offers an alternative solution: for address->utxo index taken as an example,
with optional fromHeight parameter, it adds a new RPC method "indexchainstate".
By calling that method, indices (currently only addressUnspent) can be enabled/disabled
(which is similar to add/drop index in a relational database) with the initial index
being built by iterating over UTXO set, thus eliminating the need for -reindex-chainstate
in Litecore.

Other indices (even txindex) can be added in the future with their own "add index" procedure,
avoiding costly -reindex or -reindex-chainstate run.

laanwj and others added 30 commits August 14, 2017 17:30
Tree-SHA512: 7cdf0efaafeb47ec25d29fa9dc1c5caa762ee62614288fc849370d89bd64c1e8039175fe44163fb5655db10b567baa028ae514a05ebdaf06432b9698849da195
…accidental parenthesis & fix range style

30c246b Updating the release notes (minor stylistic changes) (practicalswift)

Pull request description:

  0.15 release-notes nits:
  * Remove accidental parenthesis
  * Fix redundancy ("remain" vs. previous "continue to remain")
  * Fix range style ("from X to Y" vs. previous "from X-Y")

Tree-SHA512: d3621b6cc781e82d0156a559f71b8190c0fb1aed9ae7a8173ac9c57387a1a03fb9b8a6e4c709adadf80bcc2c2eff113dd6aa8471899c0121392c8138218cdb6c
Eric Shaw Jr -> Eric Shaw (from GitHub profile)
gubatron -> Angel Leon (from GitHub profile)
jonnynewbs -> John Newbery (duplicate)
keystrike -> James Evans (duplicate)
Kibbled Jive Elk Zoo -> KibbledJiveElkZoo (duplicate)
kirit93 -> Kirit Thadaka (from GitHub profile)
NicolasDorier -> Nicolas Dorier (duplicate)
008f64f Update release-notes.md (Derek Miller)

Pull request description:

  Eric Shaw Jr -> Eric Shaw (from GitHub profile)
  gubatron -> Angel Leon (from GitHub profile)
  jonnynewbs -> John Newbery (duplicate)
  keystrike -> James Evans (duplicate)
  Kibbled Jive Elk Zoo -> KibbledJiveElkZoo (duplicate)
  kirit93 -> Kirit Thadaka (from GitHub profile)
  NicolasDorier -> Nicolas Dorier (duplicate)

Tree-SHA512: c952e72cc82fd780532488fbd10b35918c9ac686cb6352b752a04ce5694bbdb7c2d3402b3187cdf4e5325e42e98cfacf2a95b565792cfdf7efeef1d369669e36
Github-Pull: bitcoin#11056
Rebased-From: 844b73e
Tree-SHA512: f735853c3d9abbde5032cc55201b0ddc29679e681ba7d3822cbcbeee17c7185ee984b235411224cfb8ff12550de88a0f1367d55ad24bf52c5c03061330849eb2
Github-Pull: bitcoin#11083
Rebased-From: f9ca0fe
Tree-SHA512: ef94958d7de11e091db456e253fa4a188b0b0e713149bd568aa2de11a8a3c1585b34bbfb78aaac12f969533f05602ed8b1cd16275429d86a80c36bb594ce6526
Github-Pull: bitcoin#11081
Rebased-From: 07685d1
Tree-SHA512: 7f6b9ca6714d059d500531eb842c1c25edfa04ecba0ea1d9a28010fced657c066cea67d2016bbaa9b96c431a05ca9c0dcf2ba301898ecf96a65a4e01aac7fae9
- Bump "updated for"
- Fix link to boost (haenet mirror is broken)
- Upgrade boost version to 1.64

Github-Pull: bitcoin#11080
Rebased-From: 5be6e9b
Tree-SHA512: 72877bf32e4f38fb530946c065c23edd683a0da379c33fa3eee4fbeb51e2c97171f5d33ebe1e77f25c3e14779d94e8198cc8fbb6e1055a1024638458ef7cc032
Unused function. Mostly reverts c25d90f

c25d90f... was merged as part of PR 11022 but is not required.

Github-Pull: bitcoin#11044
Rebased-From: 1221f60
Tree-SHA512: da229b128bee5f124c009a1a2adfb4fa879366c81789824c426c9ce5209c835888a7e6cfeb1724551320a98cd08406a605372f84487a0d289cd6e02f9ac3ea21
Github-Pull: bitcoin#11044
Rebased-From: 67ceff4
Tree-SHA512: 850c5b1010c84e164edf24a83ae36e46309b2eb7a67854bad509265ed590ba67d5f743a8416590da6ecca85fe4bda7f20e8c3152e422638eb7898db11a416af7
The hex is already returned in TxToUniv, no need to give it out a
second independent time in getrawtransaction itself.

Github-Pull: bitcoin#11027
Rebased-From: e029c6e
Tree-SHA512: 4f7892431ddb9b59bcc59756890b97a20d046d1d4898f8a80c564223e1cde1c922da403cea4b8ae79d70741d1a9ff337f077043e5bb538cdc4d34fbe09301240
Github-Pull: bitcoin#11027
Rebased-From: 6bbdafc
Tree-SHA512: 8ed049a0945c4f56c518aef6d60efebda5cde09ef63d8e472b363821170d71fb4d16414b360824d7537fc7c7b5e5277bbbd092a20899446e97c188ab8df58c1b
Release version strings were broken in Gitian by 7522. This is a minimal fix
suitable for 0.15.

After this, we should fix up version handling for good so that gitian packages
the correct string in the release tarball, so that git is not required to get
the tag name.

Github-Pull: bitcoin#11097
Rebased-From: 4452829
Tree-SHA512: 40071b97b2207b2262aef0b4802cb231ccafbff0ed04b5945a64dceda8bdbb2ea71bf6a546e1d03533612edfdc55dedd413da967cba0b1bbd3f9139ec3572f60
getmempooolentry -> getmempoolentry (twice)
New locales:

- bn
- bs
- de_DE
- en_AU
- en_US
- es_419
- he_IL
- id
- ja_JP
- my
- nb_NO
- nl_NL
- si
- te
- th

Tree-SHA512: af4e9d0107122e3da7da8ee4952e01a731ed04f13e83ed1979ace24d29c437a83cc23453d8f0b2160829df02c2ffbff0ce749083f82c4d4695c69a3c7feeebed
9c76cf5 Trivial: Fix typo in release-notes.md (jbarcelo)

Pull request description:

  getmempooolentry -> getmempoolentry (twice)

Tree-SHA512: 4e52d4decf4a6b0ae6ef6f3b52fed2c2fafe63d72a3b5868f172c479a9463f28eaada29b6364a5e5ca5afe021155ee97cc9a23208d2a96c4ecccebccc173bb0e
Github-Pull: bitcoin#11119
Rebased-From: fa14b67
Tree-SHA512: a790a5d37a06a691f2536763a504ead5b06b94708ba07f3b94bedbb76b0201825be3dcd95343512d7a6211089be2abc3f1e0957ed27055426e6943d4bdb6fbd9
Github-Pull: bitcoin#11145
Rebased-From: a54c7b9
Tree-SHA512: d9abb9794edbf7e52450c63d183a4f28e9a3621a670b7f21af92407be096363b44a356818f2abbea11c168d123c04852bd0b60f1cd9d423aefbf4ae831f8567d
Github-Pull: bitcoin#11145
Rebased-From: 6af49dd
Tree-SHA512: 7245c6f7ed81cf3913e351ff1df7f74f4be7abe0ae8be312e4862c9875e2943b0b11136bc9999191cdf03468fdac5d41e294bb59b657605c22643b102e37c070
Github-Pull: bitcoin#11108
Rebased-From: cd0ea48
Tree-SHA512: 0fbd50b1343eec14301ee9c2fc76e48dcb1610ce1ddbf3cccbd937e2c0d1bbdfc4920fbc05e7e560c424e0ed7367d6ad3595a1bd38dea669bcbd1c838f522d8e
Tree-SHA512: d813ef923d804722bc161cc62e09ee003b386c0abf2d513ede671e9f14ac4c3df749a9f23ddcba9400fcc335dbef6540e47a212dca8ddc0a3d73aee01f098754
a461cf4 Update release-notes.md (kallewoof)

Pull request description:

  They're the same person.

Tree-SHA512: dcdcf18d57db87445ce09d5e20ea17b0cb0653e749028b2662d708e2798e38296f0598210fb196bf3255a6bfd0837724e56419c39ebe9564406f11296c40be68
58f1e54 Release Notes: Fix SHA256-with-SSE4 PR link (Janus Troelsen)

Pull request description:

Tree-SHA512: 7f3dc2a1fb1e602fa947f4a00e39f4958eabeb47800b5b85a2035568258f0679c026d7ffe377c9256a97019b174868db82fd4aaac9c218bed145ba86f56330b1
Tree-SHA512: 1dd5e19a2ece38002da1d4bcf017a0a1a3ab2dc8b31fe7b3e7ebc6fef58de4547abbff3fb6555496da1ecf1f9a0cc941f980b053a0376a776e8301de3834aac1
fad16a9 doc: Add missing contributors to release notes (MarcoFalke)

Pull request description:

  Guido Vranken via commit 9fe1f6b

Tree-SHA512: 9f5c860ed8e23ad679e8c9e9392a66b7ee25c7be84d256855ef65caf335a8e819c7146be47d90686e61450d2501ed96b04b813860725856619088a4378dbfa90
coblee and others added 17 commits September 19, 2017 18:22
# Conflicts:
#	doc/man/litecoin-cli.1
#	doc/man/litecoin-qt.1
#	doc/man/litecoin-tx.1
#	doc/man/litecoind.1
In this commit, all references to `Bitcoin` or `Bitcoin Core` are replaced with `Litecoin` and `Litecoin Core`, respectively.  

In addition, I added a copyright line for *The Litecoin Core Developers* and provided the rightful attribution to the original `Bitcoin Core` team as per their MIT license.
GPG keys for coblee, shaolinfry, thrasher and xinxi
Currently, each index like txindex needs a new command line or config file argument,
and may require a time-consuming -reindex-chainstate or -reindex to be enabled.

Bitcore/Litecore branch with -addressindex introduces a handful of new indices,
each with its own argument; if we were to limit those indices to a part of blockchain
starting with block at height N, we would have to introduce even more arguments.

This patch offers an alternative solution: for address->utxo index taken as an example,
with optional fromHeight parameter, it adds a new RPC method "indexchainstate".
By calling that method, indices (currently only addressUnspent) can be enabled/disabled
(which is similar to add/drop index in a relational database) with the initial index
being built by iterating over UTXO set, thus eliminating the need for -reindex-chainstate
in Litecore.

Other indices (even txindex) can be added in the future with their own "add index" procedure,
avoding costly -reindex or -reindex-chainstate run.
@adolg adolg changed the title Improve how optional indices are managed RPC: Improve how optional indices are managed Oct 29, 2017
lubosz pushed a commit to lubosz/litecoin that referenced this pull request Jan 13, 2018
8225239 Merge litecoin-project#433: Make the libcrypto detection fail the newer API.
12de863 Make the libcrypto detection fail the newer API.
2928420 Merge litecoin-project#427: Remove Schnorr from travis as well
8eecc4a Remove Schnorr from travis as well
a8abae7 Merge litecoin-project#310: Add exhaustive test for group functions on a low-order subgroup
b4ceedf Add exhaustive test for verification
83836a9 Add exhaustive tests for group arithmetic, signing, and ecmult on a small group
20b8877 Add exhaustive test for group functions on a low-order subgroup
80773a6 Merge litecoin-project#425: Remove Schnorr experiment
e06e878 Remove Schnorr experiment
04c8ef3 Merge litecoin-project#407: Modify parameter order of internal functions to match API parameter order
6e06696 Merge litecoin-project#411: Remove guarantees about memcmp-ability
40c8d7e Merge litecoin-project#421: Update scalar_4x64_impl.h
a922365 Merge litecoin-project#422: Restructure nonce clearing
3769783 Restructure nonce clearing
0f9e69d Restructure nonce clearing
9d67afa Update scalar_4x64_impl.h
7d15cd7 Merge litecoin-project#413: fix auto-enabled static precompuatation
00c5d2e fix auto-enabled static precompuatation
91219a1 Remove guarantees about memcmp-ability
7a49cac Merge litecoin-project#410: Add string.h include to ecmult_impl
0bbd5d4 Add string.h include to ecmult_impl
353c1bf Fix secp256k1_ge_set_table_gej_var parameter order
541b783 Fix secp256k1_ge_set_all_gej_var parameter order
7d893f4 Fix secp256k1_fe_inv_all_var parameter order
c5b32e1 Merge litecoin-project#405: Make secp256k1_fe_sqrt constant time
926836a Make secp256k1_fe_sqrt constant time
e2a8e92 Merge litecoin-project#404: Replace 3M + 4S doubling formula with 2M + 5S one
8ec49d8 Add note about 2M + 5S doubling formula
5a91bd7 Merge litecoin-project#400: A couple minor cleanups
ac01378 build: add -DSECP256K1_BUILD to benchmark_internal build flags
a6c6f99 Remove a bunch of unused stdlib #includes
65285a6 Merge litecoin-project#403: configure: add flag to disable OpenSSL tests
a9b2a5d configure: add flag to disable OpenSSL tests
b340123 Merge litecoin-project#402: Add support for testing quadratic residues
e6e9805 Add function for testing quadratic residue field/group elements.
efd953a Add Jacobi symbol test via GMP
fa36a0d Merge litecoin-project#401: ecmult_const: unify endomorphism and non-endomorphism skew cases
c6191fd ecmult_const: unify endomorphism and non-endomorphism skew cases
0b3e618 Merge litecoin-project#378: .gitignore build-aux cleanup
6042217 Merge litecoin-project#384: JNI: align shared files copyright/comments to bitcoinj's
24ad20f Merge litecoin-project#399: build: verify that the native compiler works for static precomp
b3be852 Merge litecoin-project#398: Test whether ECDH and Schnorr are enabled for JNI
aa0b1fd build: verify that the native compiler works for static precomp
eee808d Test whether ECDH and Schnorr are enabled for JNI
7b0fb18 Merge litecoin-project#366: ARM assembly implementation of field_10x26 inner (rebase of litecoin-project#173)
001f176 ARM assembly implementation of field_10x26 inner
0172be9 Merge litecoin-project#397: Small fixes for sha256
3f8b78e Fix undefs in hash_impl.h
2ab4695 Fix state size in sha256 struct
6875b01 Merge litecoin-project#386: Add some missing `VERIFY_CHECK(ctx != NULL)`
2c52b5d Merge litecoin-project#389: Cast pointers through uintptr_t under JNI
43097a4 Merge litecoin-project#390: Update bitcoin-core GitHub links
31c9c12 Merge litecoin-project#391: JNI: Only call ecdsa_verify if its inputs parsed correctly
1cb2302 Merge litecoin-project#392: Add testcase which hits additional branch in secp256k1_scalar_sqr
d2ee340 Merge litecoin-project#388: bench_ecdh: fix call to secp256k1_context_create
093a497 Add testcase which hits additional branch in secp256k1_scalar_sqr
a40c701 JNI: Only call ecdsa_verify if its inputs parsed correctly
faa2a11 Update bitcoin-core GitHub links
47b9e78 Cast pointers through uintptr_t under JNI
f36f9c6 bench_ecdh: fix call to secp256k1_context_create
bcc4881 Add some missing `VERIFY_CHECK(ctx != NULL)` for functions that use `ARG_CHECK`
6ceea2c align shared files copyright/comments to bitcoinj's
70141a8 Update .gitignore
7b549b1 Merge litecoin-project#373: build: fix x86_64 asm detection for some compilers
bc7c93c Merge litecoin-project#374: Add note about y=0 being possible on one of the sextic twists
e457018 Merge litecoin-project#364: JNI rebased
86e2d07 JNI library: cleanup, removed unimplemented code
3093576 JNI library
bd2895f Merge pull request litecoin-project#371
e72e93a Add note about y=0 being possible on one of the sextic twists
3f8fdfb build: fix x86_64 asm detection for some compilers
e5a9047 [Trivial] Remove double semicolons
c18b869 Merge pull request litecoin-project#360
3026daa Merge pull request litecoin-project#302
03d4611 Add sage verification script for the group laws
a965937 Merge pull request litecoin-project#361
83221ec Add experimental features to configure
5d4c5a3 Prevent damage_array in the signature test from going out of bounds.
419bf7f Merge pull request litecoin-project#356
03d84a4 Benchmark against OpenSSL verification

git-subtree-dir: src/secp256k1
git-subtree-split: 8225239
@losh11 losh11 force-pushed the master branch 2 times, most recently from c461710 to 8925697 Compare February 25, 2024 00:14
@losh11
Copy link
Member

losh11 commented Feb 25, 2024

Closing. Won't merge.

@losh11 losh11 closed this Feb 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet