Merge bitpay's upstream #1231

snogcel · 2016-12-29T01:33:21Z

This pull request includes the following features from https://github.com/bitpay/bitcoin:

There was a previous assumption that blockindex would be quite small. With addressindex and spentindex enabled the blockindex is much larger and the amount of cache allocated for it should also increase. Furthermore, enabling compression should decrease the amount of disk space required and less data to write/read. The default leveldb max_open_files is set to 1000, for the blockindex the default is set to 1000 with compression. The 64 value that is current is kept for the utxo database and does not enable compression. Two additional options are added here to be able to configure the values for leveldb and the block index: - `-dbmaxopenfiles` A number of files for leveldb to keep open - `-dbcompression` Boolean 0 or 1 to enable snappy leveldb compression

since there is i/o that can happen when retrieving extended input and output information, limiting the duration of the lock in getrawtransaction can help improve concurrency

usage: ./wallet-utility -datadir=<directory> help: ./wallet-utility -h

wallet-utility: extract addresses and private keys

Add options to configure block index database

logical timestamp indexing of block hashes

test: fix determinism of address index test

rpc: option to include chain info in address index results

rpc: add method to get address deltas from a block

UdjinM6

utACK

schinzelh · 2016-12-29T03:50:29Z

Are these features mission critical for 12.1 rollout?
12.1 branch is in feature freeze & bugfix mode, so I'd like to shift this to 12.2

tgflynn · 2016-12-29T04:18:55Z

nACK

This should not go into 12.1 unless there are critical fixes, IMO.

snogcel · 2016-12-29T06:09:38Z

My apologies I should have included the reasoning in the description. This is a fairly important one but more on the Evolution Backend side of things. Specifically it relates to Insight API / UI as well as bitcore-node-dash: dashpay/dashcore-node#5 (and more specifically to bitpay#29).

In short: the current release of bitcore-node-dash 3.1.3 is making reference to features that don't exist in dash core v0.12.1.x. We've merged up to https://github.com/bitpay/bitcoin/releases/tag/v0.12.1-bitcore.

Recently I noticed that insight-ui isn't pulling in "Latest Blocks" when using the latest build of bitcore-node-dash (you can see this at https://dev-test.dash.org). After some debugging I realized that is because it's making reference to an RPC command that isn't currently implemented in v0.12.1.x.

To fix that specific bug, I think only this commit would be needed: de6f29e

UdjinM6

After discussion in slack I guess, we are going to postpone this merge for later, so I'll just leave a note here to make sure we didn't forget about it, see inline comments.

UdjinM6 · 2016-12-31T09:51:25Z

src/main.h

@@ -119,6 +119,8 @@ static const bool DEFAULT_TXINDEX = true;
 static const bool DEFAULT_ADDRESSINDEX = false;
 static const bool DEFAULT_TIMESTAMPINDEX = false;
 static const bool DEFAULT_SPENTINDEX = false;
+static const unsigned int DEFAULT_DB_MAX_OPEN_FILES = 1000;


Bitcoin Core had some problems with high number of open files in the past and they still have it at 64 so this change can be dangerous actually.

Let's make sure it works on every platform in all modes (sending a bunch of txes, reindexing, syncing from network) first or change this back to current values i.e.

static const unsigned int DEFAULT_DB_MAX_OPEN_FILES = 64; static const bool DEFAULT_DB_COMPRESSION = false;

SCDeveloper · 2017-01-20T08:17:02Z

Why is this being held back until version v0.12.2.x ?

This is not just an enchancement but a bug fix also.

schinzelh · 2017-01-20T08:51:23Z

This is not a bugfix, but very specific Bitpay features - we don't want to break our stable branch 2 weeks before launch.

nmarley · 2018-11-10T15:56:29Z

I believe the features that were requested have now been merged in via other PRs -- @UdjinM6 ok to close this 2-year old PR?

UdjinM6 · 2018-11-10T16:11:42Z

@nmarley I don't think any of these changes were merged in dash core (c++) but since no one complained I guess it was fixed in dashcore-node (nodejs) somehow, right?

nmarley · 2018-11-10T16:24:39Z

Well I thought Jon wanted the indexes (e.g. address, timestamp indices) but I guess those probably already were merged previously. Looking closer, it seems like he needed an extra JSON argument to getblockhashes, but I'm also guessing this isn't needed any more (and Evo backend has changed since 2016, when it was still quite early).

nmarley · 2018-11-10T16:25:45Z

This particular PR can probably be closed regardless, and I think we should be able to open new ones for any individual things which might be needed. Scope of this is too large IMO.

UdjinM6 · 2018-11-10T16:27:06Z

Ok, closing

…llet Add label API to wallet RPC. This is one step towards dashpay#3816 ("Remove bolt-on account system") although it doesn't actually remove anything yet. These initially mirror the account functions, with the following differences: - These functions aren't DEPRECATED in the help - Help mentions 'label' instead of accounts. In the language used, labels are associated with addresses, instead of addresses associated with labels. (unlike with accounts.) - Labels have no balance - No balances in `listlabels` - `listlabels` has no minconf or watchonly argument - Like in the GUI, labels can be set on any address, not just receiving addreses - Unlike accounts, labels can be deleted. Being unable to delete them is a common annoyance (see dashpay#1231). Currently only by reassigning all addresses using `setlabel`, but an explicit call `deletelabel` which assigns all address to the default label may make sense. Thanks to Pierre Rochard for test fixes.

Add label API to wallet RPC. This is one step towards dashpay#3816 ("Remove bolt-on account system") although it doesn't actually remove anything yet. These initially mirror the account functions, with the following differences: - These functions aren't DEPRECATED in the help - Help mentions 'label' instead of accounts. In the language used, labels are associated with addresses, instead of addresses associated with labels. (unlike with accounts.) - Labels have no balance - No balances in `listlabels` - `listlabels` has no minconf or watchonly argument - Like in the GUI, labels can be set on any address, not just receiving addreses - Unlike accounts, labels can be deleted. Being unable to delete them is a common annoyance (see dashpay#1231). Currently only by reassigning all addresses using `setlabel`, but an explicit call `deletelabel` which assigns all address to the default label may make sense. Thanks to Pierre Rochard for test fixes.

Braydon Fuller and others added 20 commits July 14, 2016 13:56

rpc: minimize locking in getrawtransaction

1b36e2c

since there is i/o that can happen when retrieving extended input and output information, limiting the duration of the lock in getrawtransaction can help improve concurrency

tests: test dbwrapper options compression and maxopenfiles

69ea12c

wallet-utility: extract addresses and private keys

31f56e2

usage: ./wallet-utility -datadir=<directory> help: ./wallet-utility -h

test: wallet utility python test

efb0d55

Remove cpp test for wallet-utility

3658026

Merge pull request dashpay#24 from danjurious/bitpay-12.1

0c15767

wallet-utility: extract addresses and private keys

Merge pull request dashpay#27 from braydonf/0.12.1-bitcore-dboptions

796a274

Add options to configure block index database

test: fix determinism of address index test

05cf410

Add method to get address deltas from a block

d14a774

logical timestamp indexing of block hashes

c8458bf

Add a new line to print

36f5ee5

Merge pull request dashpay#29 from danjurious/logicalts

de6f29e

logical timestamp indexing of block hashes

Merge pull request dashpay#30 from braydonf/0.12.1-bitcore-fix-test

33e35c8

test: fix determinism of address index test

rpc: option to include chain info in address index results

491d6eb

Merge pull request dashpay#31 from braydonf/hashes

14c71a7

rpc: option to include chain info in address index results

Merge pull request dashpay#23 from braydonf/block-deltas

597c9b4

rpc: add method to get address deltas from a block

Merge remote-tracking branch 'bitpay/0.12.1-bitcore' into 0.12.1-bitcore

dd9b9b6

fix tests

b3e63c2

dashify wallet utility

9553545

UdjinM6 changed the title ~~0.12.1 bitcore-4~~ Merge bitpay's upstream Dec 29, 2016

UdjinM6 approved these changes Dec 29, 2016

View reviewed changes

UdjinM6 requested changes Dec 31, 2016

View reviewed changes

schinzelh added this to the 12.2 milestone Jan 13, 2017

schinzelh added the enhancement label Jan 13, 2017

schinzelh changed the base branch from v0.12.1.x to v0.12.2.x February 20, 2017 20:39

schinzelh force-pushed the v0.12.2.x branch from b03bbb6 to 119fe83 Compare March 7, 2017 16:03

TheSin- mentioned this pull request Jan 26, 2018

Add Patch from Dash terracoin/terracoin#13

Closed

UdjinM6 removed this from the 12.2 milestone Mar 16, 2018

UdjinM6 closed this Nov 10, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge bitpay's upstream #1231

Merge bitpay's upstream #1231

snogcel commented Dec 29, 2016

UdjinM6 left a comment

schinzelh commented Dec 29, 2016

tgflynn commented Dec 29, 2016

snogcel commented Dec 29, 2016 •

edited

UdjinM6 left a comment •

edited

UdjinM6 Dec 31, 2016

SCDeveloper commented Jan 20, 2017

schinzelh commented Jan 20, 2017

nmarley commented Nov 10, 2018

UdjinM6 commented Nov 10, 2018

nmarley commented Nov 10, 2018

nmarley commented Nov 10, 2018

UdjinM6 commented Nov 10, 2018

Merge bitpay's upstream #1231

Merge bitpay's upstream #1231

Conversation

snogcel commented Dec 29, 2016

UdjinM6 left a comment

Choose a reason for hiding this comment

schinzelh commented Dec 29, 2016

tgflynn commented Dec 29, 2016

snogcel commented Dec 29, 2016 • edited

UdjinM6 left a comment • edited

Choose a reason for hiding this comment

UdjinM6 Dec 31, 2016

Choose a reason for hiding this comment

SCDeveloper commented Jan 20, 2017

schinzelh commented Jan 20, 2017

nmarley commented Nov 10, 2018

UdjinM6 commented Nov 10, 2018

nmarley commented Nov 10, 2018

nmarley commented Nov 10, 2018

UdjinM6 commented Nov 10, 2018

snogcel commented Dec 29, 2016 •

edited

UdjinM6 left a comment •

edited