Skip to content

detach wallet from miner #5994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

jonasschnelli
Copy link
Contributor

This will interfere with #5993.
if this makes sense (conceptual), i'll rebase this after merging of @sipa's #5993 or @jtimon's #4793.

Next steps would be to add a argument pubkeyhash for generate to completely decouple.
generate could check over CMainSignals if a registered "device" can provide a pubkey in case of no given key over the rpc argument.

@jgarzik
Copy link
Contributor

jgarzik commented Apr 12, 2015

lightly tested ACK

Thanks for doing this. I coded this a while ago, then set it aside. Your version, with registration, is cleaner.

@@ -1208,7 +1208,7 @@ CAmount CWalletTx::GetCredit(const isminefilter& filter) const
}

CAmount CWalletTx::GetImmatureCredit(bool fUseCache) const
{
{
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This seems to have sneaked in?

@jonasschnelli
Copy link
Contributor Author

fixed @Diapolo's nits.

@jtimon
Copy link
Contributor

jtimon commented Apr 14, 2015

Concept ack.
Given that you're touching their interfaces already, could CreateNewBlockWithScript() ProcessBlockFound(), BitcoinMiner() and GenerateBitcoins() take const CChainParams& chainparams as parameter?

@jonasschnelli jonasschnelli force-pushed the 2015/04/miner_wallet_separation branch from 35a97d8 to d24f147 Compare April 14, 2015 12:15
@jonasschnelli
Copy link
Contributor Author

Added a commit on top to help improve global state independence by passing CChainParams for the touched functions.

@jonasschnelli jonasschnelli force-pushed the 2015/04/miner_wallet_separation branch 2 times, most recently from e898656 to e8d4c0e Compare April 14, 2015 13:15
@jtimon
Copy link
Contributor

jtimon commented Apr 14, 2015

ut ACK, it is very nice to decouple miner.o from the wallet.

@jtimon
Copy link
Contributor

jtimon commented Apr 16, 2015

Needs rebase

@jonasschnelli jonasschnelli force-pushed the 2015/04/miner_wallet_separation branch 2 times, most recently from 0825a17 to bab4348 Compare April 16, 2015 07:36
@jonasschnelli
Copy link
Contributor Author

Rebased and squashed [squashme] commit.

@jonasschnelli jonasschnelli force-pushed the 2015/04/miner_wallet_separation branch from bab4348 to 2528922 Compare April 16, 2015 08:30
@jtimon
Copy link
Contributor

jtimon commented Apr 16, 2015

I would also squash the second commit, leaving the note in the commit descrption. Specially, you never want to create CreateNewBlockWithScript.
In any case, minor nit, re-utAck.

@jonasschnelli jonasschnelli force-pushed the 2015/04/miner_wallet_separation branch from 2528922 to f4dfba8 Compare April 16, 2015 08:52
@jonasschnelli
Copy link
Contributor Author

Agreed. Squashed.

@@ -35,6 +36,8 @@ class CValidationInterface {
virtual void Inventory(const uint256 &hash) {};
virtual void ResendWalletTransactions(int64_t nBestBlockTime) {};
virtual void BlockChecked(const CBlock&, const CValidationState&) {};
virtual void GetScriptForMining(CScript &script) {};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say this needs to be able to fail (if no wallet is connected), and have the miner code deal with that case. Otherwise you can't really make it independent of the wallet (if it needs to assume a wallet that can provide a script output for mining is present).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it can fail. If no listening module provides a CScript through signal ScriptForMining the miner will continue with a empty script which should result in a unspendable coinbase? But i didn't tested that. Will do.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not want to create a block with an unspendable coinbase.

I mean the call should return a boolean, or the resulting script should at least be tested for non-emptiness, and the miner should exit if that is the case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be nice if you could use generate() in a non-wallet environment? Regarding RPC test in a post-wallet-split-off world: this could make sense.

If somone uses the internal miner (which has no productive usage IMO) in a wallet-disable mode he probably uses it for testing only.

I agree with adding a warning to the log. But throwing an error would block testing.

Copy link
Member

@sipa sipa Apr 24, 2015 via email

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Maybe it would be most efficient by adding a argument to generate() and setgenerate() RPC calls where one could provide a output script to mine to?

Adding a validation interface implementation for non-wallet-mining could be over the top.

Copy link
Member

@sipa sipa Apr 24, 2015 via email

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@sipa sipa Apr 24, 2015 via email

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonasschnelli
Copy link
Contributor Author

Updated.
Decoupled wallet (over signaling) from miner and rpcmining commands.
It still uses signaling to allow different wallet/pubscript-generation implementations, though, mining without pubScript providing listener will generate a error and stops mining.

Now it reserves and keeps a key when calling setgenerate or generating over cmd arg gen (same key for all generated blocks).

@jonasschnelli jonasschnelli force-pushed the 2015/04/miner_wallet_separation branch from f4dfba8 to 4fbff12 Compare April 29, 2015 12:25
@jtimon
Copy link
Contributor

jtimon commented Apr 30, 2015

Travis is failing for the build without wallet: https://travis-ci.org/bitcoin/bitcoin/jobs/60528797#L1557

@jonasschnelli jonasschnelli force-pushed the 2015/04/miner_wallet_separation branch from 5093ce6 to 5641fb8 Compare April 30, 2015 11:51
@jonasschnelli
Copy link
Contributor Author

@jtimon: Thanks for the report. Fixed.

@jonasschnelli
Copy link
Contributor Author

Now the miner is completely decoupled from the wallet.
There are no more #ifdef ENABLE_WALLET checks (for mining).
The only check is during runtime if there was a listening device who could provide a CScript over the signal ScriptForMining().

If one tries to use -gen without a wallet compiled in or enabled, it then will throw an exception during init phase. Current master would just ignore the cmd arg -gen if no wallet is present.

The RPC commands setgenerate, generate, getgenerate works in wallet-disabled mode but also throw a exception error: {"code":-32601,"message":"Use the generate method instead of setgenerate on this network"}

@jonasschnelli
Copy link
Contributor Author

Rebased.
Seeks testers to escape the rebase hamster wheel.

@jonasschnelli jonasschnelli force-pushed the 2015/04/miner_wallet_separation branch from 82942dc to 6a30073 Compare May 19, 2015 12:12
str4d added a commit to str4d/zcash that referenced this pull request Apr 4, 2018
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
zkbot added a commit to zcash/zcash that referenced this pull request Apr 6, 2018
Bitcoin Core refactoring and cleanups 1

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#5994
- bitcoin/bitcoin#6538
- bitcoin/bitcoin#6163
- bitcoin/bitcoin#6982
- bitcoin/bitcoin#6986
- bitcoin/bitcoin#7053
- bitcoin/bitcoin#7444
- bitcoin/bitcoin#7793
  - Excluding some comments in `txmempool.h` on code we haven't yet pulled in.
- bitcoin/bitcoin#7916
- bitcoin/bitcoin#7815

Additionally, the Equihash parameters are moved into the consensus parameters.

Part of #2074.
str4d added a commit to str4d/zcash that referenced this pull request Apr 13, 2018
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
str4d added a commit to str4d/zcash that referenced this pull request May 31, 2018
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
zkbot added a commit to zcash/zcash that referenced this pull request May 31, 2018
Bitcoin Core refactoring and cleanups 1

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#5994
- bitcoin/bitcoin#6538
- bitcoin/bitcoin#6163
- bitcoin/bitcoin#6982
- bitcoin/bitcoin#6986
- bitcoin/bitcoin#7053
- bitcoin/bitcoin#7444
- bitcoin/bitcoin#7793
  - Excluding some comments in `txmempool.h` on code we haven't yet pulled in.
- bitcoin/bitcoin#7916
- bitcoin/bitcoin#7815

Additionally, the Equihash parameters are moved into the consensus parameters.

Part of #2074.
str4d added a commit to str4d/zcash that referenced this pull request Oct 24, 2018
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
zkbot added a commit to zcash/zcash that referenced this pull request Oct 24, 2018
Bitcoin Core refactoring and cleanups 1

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#5994
- bitcoin/bitcoin#6538
- bitcoin/bitcoin#6163
- bitcoin/bitcoin#6982
- bitcoin/bitcoin#6986
- bitcoin/bitcoin#7053
- bitcoin/bitcoin#7444
- bitcoin/bitcoin#7793
  - Excluding some comments in `txmempool.h` on code we haven't yet pulled in.
- bitcoin/bitcoin#7916
- bitcoin/bitcoin#7815

Additionally, the Equihash parameters are moved into the consensus parameters.

Part of #2074.
str4d added a commit to str4d/zcash that referenced this pull request Oct 25, 2018
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
zkbot added a commit to zcash/zcash that referenced this pull request Oct 25, 2018
Bitcoin Core refactoring and cleanups 1

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#5994
- bitcoin/bitcoin#6538
- bitcoin/bitcoin#6163
- bitcoin/bitcoin#6982
- bitcoin/bitcoin#6986
- bitcoin/bitcoin#7053
- bitcoin/bitcoin#7444
- bitcoin/bitcoin#7793
  - Excluding some comments in `txmempool.h` on code we haven't yet pulled in.
- bitcoin/bitcoin#7916
- bitcoin/bitcoin#7815

Additionally, the Equihash parameters are moved into the consensus parameters.

Part of #2074.
str4d added a commit to str4d/zcash that referenced this pull request Mar 5, 2019
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
zkbot added a commit to zcash/zcash that referenced this pull request Mar 12, 2019
Detach wallet from miner

Cherry-picked from upstream PR bitcoin/bitcoin#5994.

Part of #2074.
renuzit pushed a commit to btcz/bitcoinz that referenced this pull request May 16, 2019
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
milesmanley pushed a commit to RunOnFlux/fluxd that referenced this pull request May 19, 2019
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
milesmanley pushed a commit to RunOnFlux/fluxd that referenced this pull request May 19, 2019
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
garethtdavies pushed a commit to garethtdavies/zcash-patched-for-explorer that referenced this pull request May 30, 2019
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
garethtdavies pushed a commit to garethtdavies/zcash-patched-for-explorer that referenced this pull request May 30, 2019
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
garethtdavies pushed a commit to garethtdavies/zcash-patched-for-explorer that referenced this pull request May 30, 2019
After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.
renuzit added a commit to btcz/bitcoinz that referenced this pull request May 31, 2019
* Backport size_on_disk to RPC call getblockchaininfo.

This commit is extracted from upstream PR:
bitcoin/bitcoin#11367

* Add size_on_disk test

Co-authored-by: Simon <simon@bitcartel.com>

* Rename methods to include Sprout

* Add benchmark for decrypting sapling notes

* Move reusable Sapling test setup to utiltest

* Move test SaplingNote creation to utiltest

* Add test method for generating master Sapling extended spending keys

* Include Sapling transactions in increment note witness benchmark

* Prevent header from being included multiple times

* benchmarks do not require updating network parameters

* FakeCoinsViewDB can inherit directly from CCoinsView

* Add a method for generating a test CKey

* Change to t->z transaction and create separate benchmark for sapling

* Renaming and other minor cleanup

* Improve some error messages when building a transaction fails

* redirect and update source documentation

* Update OpenSSL from 1.1.0h to 1.1.1a. #3786

* Update boost from v1.66.0 to v1.69.0. #3786

* Update Rust from v1.28.0 to v1.32.0. #3786

* Update Proton from 0.17.0 to 0.26.0. #3816, #3786

* Patch Proton for a minimal build. #3786

* Update of copyright year to 2019

* Add Sapling benchmarks to benchmark runner

* Add missing author aliases

* Correcting logo on README

* Simplify DisconnectBlock arguments/return value

DisconnectBlock currently has a complicated interface:

  Situation       Return value
                  pfClean != nullptr   pfClean == nullptr

  All good:       true                 true
  Failure:        false                false
  Unclean rewind: true                 false
                  with *pfClean=false

Change this to return a tristate enum instead. As an added bonus,
remove the ValidationState& argument which was unused.

* Add Sprout support to TransactionBuilder

* Split test in to multiple parts

* Use a custom error type if creating joinsplit descriptions fails

* detach wallet from miner

* fix GetScriptForMining() CReserveKey::keepKey() issue

* add CReserveScript to allow modular script keeping/returning

- use one CReserveScript per mining thread

* miner: rename UpdateRequestCount signal to ResetRequestCount

* test: Fetch coinbase address from coinbase UTXOs

After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after
startup does not return the address being used by the miner.

* test: Make expected_utxos optional in get_coinbase_address()

* Add comments

* Move utiltest.cpp from wallet to common

This ensures it is accessible by the test suite when the wallet is
disabled.

* Move payment disclosure code and tests into wallet

The code was already compiled as part of the wallet, but the tests were
not, meaning that the tests would fail to compile when the wallet was
disabled.

* depends: Use full path to cargo binary

The native binaries generated in the depends system are available on the path,
but system binaries are still visible. This change ensures we use cargo from
the depends system rather than whatever might be installed locally.

* depends: Generalise the rust package cross-compilation functions

* depends: Add rust-std hash for aarch64-unknown-linux-gnu

Usage on Debian / Ubuntu:

> $ sudo apt install g++-aarch64-linux-gnu
> $ HOST=aarch64-linux-gnu ./zcutil/build.sh

Currently fails to cross-compile due to later configuration issues in
the depends system that need to be worked around.

* depends: Compile bdb with --disable-atomics on aarch64

This sidesteps the problem where the atomics check tries to run a test
binary, which cannot be performed during cross compilation. We should
replace this with a better solution in future.

Part of #3710.

* (testnet) Fall back to hardcoded shielded pool balance to avoid reorgs.

* (testnet) Reject blocks that result in turnstile violations

* depends: Update .gitignore

* configure: Guess -march for libsnark OPTFLAGS instead of hard-coding

When cross-compiling, this will remove the -march flag entirely unless
the user specifies CONFIGURE_FLAGS="--with-gcc-arch=<arch>".

* (testnet/regtest) Avoid mining transactions that would violate the turnstile.

* Update nMinimumChainWork using block 497000.

* Add checkpoint for block 497000.

* Fix tallying for Sprout/Sapling value pools.

* Consolidate logic to enable turnstile auditing for testnet/regtest/mainnet.

* Use existing chainparams variable

* Add newlines to turntile log messages for miner

* Check blockhash of fallback block for Sprout value pool balance

* Added documentation warnings about DNS rebinding attacks, issue #3841

* Added responsible disclosure statement for issue #3869

* Change SproutValuePoolCheckpointEnabled to ZIP209Activated

* Only enforce Sapling turnstile if balance values have been populated.

* Update COPYRIGHT_YEAR in clientversion.h to 2019

Update COPYRIGHT_YEAR in clientversion.h to 2019

* Do not enable ZIP209 on regtest right now.

* (minor) Remove added newline.

* Rename and update comment

* (wallet) Check that the commitment matches the note plaintext provided by the sender.

* Update release notes for 2.0.4

Co-authored-by: Jack Grigg <jack@z.cash>

* Electric Coin Company

* Minor speling changes

* make-release.py: Versioning changes for 2.0.4-rc1.

* make-release.py: Updated manpages for 2.0.4-rc1.

* make-release.py: Updated release notes and changelog for 2.0.4-rc1.

* Fix typo in release notes.

* Update _COPYRIGHT_YEAR in configure.ac to 2019

Update _COPYRIGHT_YEAR in configure.ac to 2019

* fix enable-debug build DB_COINS undefined

* Fix proton patch regression. #3916

* Fix OpenSSL reproducible build regression

* Patch out proton::url deprecation as workaround for build warnings

* make-release.py: Versioning changes for 2.0.4.

* make-release.py: Updated manpages for 2.0.4.

* make-release.py: Updated release notes and changelog for 2.0.4.

* Update company name.

* add -addressindex changes for bitcore insight block explorer

* tests: adds unit test for IsPayToPublicKeyHash method

* Add Blossom to upgrade list

* init: Fix new HD seed generation for previously-encrypted wallets

Closes #3607.

* Creates checklist template for new PRs being opened and addresses Str4d's suggestion for using GitHub handles

* Adding addressindex.h to Makefile.am

Co-authored by: Daira Hopwood <daira@jacaranda.org>

* Add testnet and regtest experimental feature: -developersetpoolsizezero

* add -spentindex changes for bitcore insight block explorer

* Update boost from v1.69.0 to v1.70.0. #3947

* Add qa test for experimental feature: -developersetpoolsizezero

* Enable ZIP209 on mainnet and set fallback Sprout pool balance.

* Enable experimental feature -developersetpoolsizezero on mainnet.

* Add rpc to enable and disable Sprout to Sapling migration

* Move migration logic to ChainTip

* Documentation cleanup

* Additional locking and race condition prevention

* Refactor wait_and_assert_operationid_status to allow returning the result

* Set min depth when selecting notes to migrate

* add -timestampindex for bitcore insight block explorer

* Check for full failure message in test case

* Add migration options to conf file

Co-authored-by: Simon <simon@bitcartel.com>

* remove extra hyphen

Co-Authored-By: Eirik0 <eirik@z.cash>

* Create method for getting HD seed in RPCs

* Add rpc to get Sprout to Sapling migration status

* Fix help message

* Test migration using both the parameter and the default Sapling address

* Fix typos and update documentation

* use -valueBalance rather than vpub_new to calculate migrated amount

* Do not look at vin/vout when determining migration txs and other cleanup

* Calculate the number of confimations in the canonical way

* Do not throw an exception if HD Seed is not found when exporting wallet

* 3873 z_setmigration cli bool enable arg conversion

* make-release.py: Versioning changes for 2.0.5-rc1.

* make-release.py: Updated manpages for 2.0.5-rc1.

* make-release.py: Updated release notes and changelog for 2.0.5-rc1.

* add curl to package list for gitian lxc container

* Update chain work and checkpoint using block 525000.

* Notable changes for v2.0.5

* Add missing word to release notes

* make-release.py: Versioning changes for 2.0.5.

* make-release.py: Updated manpages for 2.0.5.

* make-release.py: Updated release notes and changelog for 2.0.5.

* Correctly account for migration transactions in the mempool

Co-authored-by: LarryRuane <larry@z.cash>

* Store the migration operation id rather than the operation iteslf

* Rename variable and add comment

* Notable changes for v2.0.5-1

* Fix summing available funds

* Add the amount migrated to the operation's result

* coinsView is required when using TransactionBuilder if there may be Sprout change

* make-release.py: Versioning changes for 2.0.5-1.

* make-release.py: Updated manpages for 2.0.5-1.

* make-release.py: Updated release notes and changelog for 2.0.5-1.

* Remove unused specifier from format string.

The extra specifier meant that a runtime error would be thrown
during Sprout to Sapling migration if `zrpcunsafe` logging
was enabled:
"tinyformat: Too many conversion specifiers in format string"

* Don't allow migration when node is syncing at launch or after waking up.

* Generalize TransactionBuilder and CreateNewContextualCMutableTransaction to allow choosing the expiry delta.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

* Repair calls to TransactionBuilder from tests.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

* Change expiry delta for migration transactions to 450 blocks.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

* Test the expiry height of migration transactions.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

* Fix cosmetic spacing issue in z_setmigration help.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

* Do not automatically remove async migration operations and return txids

* Add logging for Sprout to Sapling migration transaction generation

* Fix LogPrint statements

* Notable changes for v2.0.5-2

* Release notes wording and punctuation

* make-release.py: Versioning changes for 2.0.5-2.

* make-release.py: Updated manpages for 2.0.5-2.

* make-release.py: Updated release notes and changelog for 2.0.5-2.

* Update ZIP reference

* Set BitcoinZ version to 2.0.5

* Fixed merge bug

Let's use the chainparams passed into the function rather than the
static class instance.

* Support to reduce future block time window to 5 mins

Check timestamp is within the allowed 5 minute window to help decrease
effectiveness of timewarp attacks

* Added reorg protection

This commit implements some basic reorg protection by implementing
a rolling 10 block checkpoint. This limits the number of blocks that
a miner can mine in secret to a maximum of 10.

* Don't require flag to enable reorg projection

* Add disablereorgprotection flag

Use the -disablereorgprotection flag to disable reorg protection.

* Add checkpoints

* Zeuz Fingerz release to go live on block 364400

* ignore shutdown process when reorg > 100 blocks

* Set rolling checkpoint height to 6 blocks

* Set minimum protocol version to 77007
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants