Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update zmq to 4.3.1 #15188

Merged
merged 1 commit into from Jan 21, 2019
Merged

Update zmq to 4.3.1 #15188

merged 1 commit into from Jan 21, 2019

Conversation

rex4539
Copy link
Contributor

@rex4539 rex4539 commented Jan 17, 2019

@hebasto
Copy link
Member

hebasto commented Jan 17, 2019

Should doc/dependencies.md be updated as well?

@laanwj
Copy link
Member

laanwj commented Jan 17, 2019

adding backport label for 0.17.2 as this is a security bugfix

@practicalswift
Copy link
Contributor

Concept ACK

@fanquake
Copy link
Member

Concept ACK. I had a local branch with the same change. We'll need to review changes to libzmq.pc and related build files.

Note: At the moment, trying to ./configure Core with a brew installed zeromq (only version 4.3.1), is broken. Related PRs Homebrew/homebrew-core#35940 & Homebrew/homebrew-core#36121.

doc/dependencies.md Outdated Show resolved Hide resolved
@DrahtBot
Copy link
Contributor

Gitian builds for commit 12b3010 (master):

Gitian builds for commit 607eaa233355590ec4c2fdd532201e1a8edcd5bd (master and this pull):

@laanwj
Copy link
Member

laanwj commented Jan 18, 2019

utACK 3046e5f

@laanwj
Copy link
Member

laanwj commented Jan 18, 2019

Note: At the moment, trying to ./configure Core with a brew installed zeromq (only version 4.3.1), is broken. Related PRs Homebrew/homebrew-core#35940 & Homebrew/homebrew-core#36121.

As the gitian configure/build completes without problems, this seems to be specific to brew.

@laanwj laanwj merged commit 3046e5f into bitcoin:master Jan 21, 2019
laanwj added a commit that referenced this pull request Jan 21, 2019
3046e5f Update zmq to 4.3.1 (Dimitris Apostolou)

Pull request description:

  Addresses https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250

Tree-SHA512: 9daa6580d103dad5f1499de1e9f5e1990cb4d630982f4ff35dcba492d7c7fc6e6224556ffd01575c97d50c34011f6e03677601ce35bc396a9e2a083508a64340
fanquake pushed a commit to fanquake/bitcoin that referenced this pull request Jan 25, 2019
@fanquake
Copy link
Member

Backported in #15252.

laanwj added a commit that referenced this pull request Jan 30, 2019
8418707 Update zmq to 4.3.1 (Dimitris Apostolou)

Pull request description:

  Backports #15188 to the 0.17 branch.

  Addresses https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250.

Tree-SHA512: f8047a22430dcc92ef713e72281286c4cad2fbe3cd6880cb16b9f8c83bdc481a72539ff5fd373133ac006409ca083db68368e56df47a50798963e77fb3e579f2
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Feb 14, 2019
thrasher- pushed a commit to thrasher-/litecoin that referenced this pull request Feb 19, 2019
thrasher- pushed a commit to thrasher-/litecoin that referenced this pull request Feb 19, 2019
justinvforvendetta pushed a commit to vergecurrency/verge that referenced this pull request Feb 19, 2019
justinvforvendetta added a commit to vergecurrency/verge that referenced this pull request Feb 19, 2019
justinvforvendetta added a commit to vergecurrency/verge that referenced this pull request Feb 24, 2019
* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* fix shmypo

* Update zeromq to 4.3.1 (#874) (#875)

This is related to the BTC issue
bitcoin/bitcoin#15188
and CVE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250

* [WIP] Bugfix block submission (#877)

* Remove segwits segments from block template 🥳

* Disabling Segwit components (no pre-segwit)

* Remove segwit switch from code 🧐

* Reenable version bit calculations

* Enabling Chainparams genesis block and merkle root checks

* Improve header parsing and correction for block parsing with older nodes #865 (#870)

* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* Fix testnet chain parameters

* Refactor POW a bit for more readability

* Include Blocksignatures as part of our generated blocks
(also being transferred)

* Fix pub-/privkey prefixes for testnet
justinvforvendetta added a commit to vergecurrency/verge that referenced this pull request Feb 24, 2019
* Remove segwits segments from block template 🥳

* Disabling Segwit components (no pre-segwit)

* Remove segwit switch from code 🧐

* Reenable version bit calculations

* Enabling Chainparams genesis block and merkle root checks

* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* fix shmypo

* Update zeromq to 4.3.1 (#874) (#875)

This is related to the BTC issue
bitcoin/bitcoin#15188
and CVE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250

* Improve header parsing and correction for block parsing with older nodes #865 (#870)

* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* Fix testnet chain parameters

* Refactor POW a bit for more readability

* Include Blocksignatures as part of our generated blocks
(also being transferred)

* Fix pub-/privkey prefixes for testnet

* [WIP] Bugfix block submission (#877)

* Remove segwits segments from block template 🥳

* Disabling Segwit components (no pre-segwit)

* Remove segwit switch from code 🧐

* Reenable version bit calculations

* Enabling Chainparams genesis block and merkle root checks

* Improve header parsing and correction for block parsing with older nodes #865 (#870)

* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* Fix testnet chain parameters

* Refactor POW a bit for more readability

* Include Blocksignatures as part of our generated blocks
(also being transferred)

* Fix pub-/privkey prefixes for testnet

* Remove previous invalidation schemes and replace them with our ones

* Transfer methods into cpp file rather than inline methods

* Enabling full mining capabilities

* Fix compiling issues

* Revert "Merge branch 'bench_tests' into develop"

This reverts commit 3969482, reversing
changes made to 33ce10b.
justinvforvendetta pushed a commit to vergecurrency/verge that referenced this pull request Feb 28, 2019
* Remove segwits segments from block template 🥳

* Disabling Segwit components (no pre-segwit)

* Remove segwit switch from code 🧐

* Reenable version bit calculations

* Enabling Chainparams genesis block and merkle root checks

* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* fix shmypo

* Update zeromq to 4.3.1 (#874) (#875)

This is related to the BTC issue
bitcoin/bitcoin#15188
and CVE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250

* Improve header parsing and correction for block parsing with older nodes #865 (#870)

* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* fix init/for bench

* add chain to interfaces

* add optional header

* close def

* add walletinit to makefile

* add chain header to init

* add int chain

* update rpc util

* move rawtx to node/tx

* remove header dup

* add zmq headers

* remove zmqrpc hdr

* remove unused boost deps

* add zmqrpc

* add zmq rpc to makefile

* zmqheader

* add zmqrpc to automake

* update walletiinitinterface header

* updates to node txs

* Update init.cpp

* updates

* Update strencodings.h

* add assumptions to compat

* update banman

* update addrdb

* update addrman

* update getcheaphash

* add chain interface

* update net_processing header

* updates to node interface

* updates to zmq notification interface

* update zmq abstract notifier

* update net header

* move handlers to header

* update cclientUIinterface

* clean up, remove regtest

* update logging

* update logger on http server

* isbindany

* vaddednodes

* remove dupe

* close())

* SetLimited -> SetReachable

* add argument

* add arg

* add arg

* update to net

* fixes

* update w header parsing

* update validation header

ugh

* fixes

* fix torcontrol

* loginstance

* +}-(

hue

* pita

* remove arg

* update rest api

* update blockchain rpc header

* cblockindex updates

* more cblockindex

* Fix testnet chain parameters

* Refactor POW a bit for more readability

* Include Blocksignatures as part of our generated blocks
(also being transferred)

* Fix pub-/privkey prefixes for testnet

* [WIP] Bugfix block submission (#877)

* Remove segwits segments from block template 🥳

* Disabling Segwit components (no pre-segwit)

* Remove segwit switch from code 🧐

* Reenable version bit calculations

* Enabling Chainparams genesis block and merkle root checks

* Improve header parsing and correction for block parsing with older nodes #865 (#870)

* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* Fix testnet chain parameters

* Refactor POW a bit for more readability

* Include Blocksignatures as part of our generated blocks
(also being transferred)

* Fix pub-/privkey prefixes for testnet

* Develop (#878)

* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* fix shmypo

* Update zeromq to 4.3.1 (#874) (#875)

This is related to the BTC issue
bitcoin/bitcoin#15188
and CVE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250

* [WIP] Bugfix block submission (#877)

* Remove segwits segments from block template 🥳

* Disabling Segwit components (no pre-segwit)

* Remove segwit switch from code 🧐

* Reenable version bit calculations

* Enabling Chainparams genesis block and merkle root checks

* Improve header parsing and correction for block parsing with older nodes #865 (#870)

* Update randomized authentication for proxy connections

* Adds copyrights to torcontroller
Resolves #867

* Tor Hidden Services v3 (#872)

* Initi v3 tor hidden services

* Support hidden services (V3) for new nodes

* Remove my laziness

* Remove more logs

* Add i2p support (#873)

* add i2p support

if using i2p wrapped daemon.

it would be nice to eventually add i2pd as an option as well.

* Fix testnet chain parameters

* Refactor POW a bit for more readability

* Include Blocksignatures as part of our generated blocks
(also being transferred)

* Fix pub-/privkey prefixes for testnet

* add isbindany back to netaddress header

* update blockchain rpc

* update blockchain rpc header

* fix blockchain rpc header

* Move versionbits info out of versionbits.o

* Remove previous invalidation schemes and replace them with our ones

* Transfer methods into cpp file rather than inline methods

* Enabling full mining capabilities

* Fix compiling issues

* Revert "Merge branch 'bench_tests' into develop"

This reverts commit 3969482, reversing
changes made to 33ce10b.

* Revert "Merge branch 'bench_tests' into develop"

This reverts commit efc7e3b, reversing
changes made to b508db7.

* Fix linking issues for wallet signing

* Update README.md

* Remove the useless rest of a merge conflict 🙉
Rishabh42 pushed a commit to Rishabh42/syscoin that referenced this pull request Mar 22, 2019
uhliksk pushed a commit to fxtc/fxtc that referenced this pull request Apr 21, 2019
Rishabh42 added a commit to Rishabh42/syscoin that referenced this pull request Apr 22, 2019
* Revert "more dbg"

This reverts commit c114420.

* Revert "more dbg"

This reverts commit 20d7fa1.

* Revert "more dbg"

This reverts commit f2a4621.

* Revert "logging"

This reverts commit 067d813.

* Revert "more dbg"

This reverts commit f2a4621.

* remove dbg

* remove dbg

* add 500 byte check in rpc prepare

* wip on new ban policy

* rpc: Make unloadwallet wait for complete wallet unload

Github-Pull: bitcoin#14941
Rebased-From: c37851d

* Remove errant past from walletcreatefundedpsbt for nLocktime replaceability

Github-Pull: bitcoin#15213
Rebased-From: 85f0ca9

* Update zmq to 4.3.1

Addresses https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250

Github-Pull: bitcoin#15188
Rebased-From: 3046e5f

* [0.17] [Doc] Backport release note about PSBT doc

* add pingretries to cmasternode

* start masternode preenabled and switch to enabled on ping check from broadcast

* assetallocationbalance rpc call

* have aamany vs just aa rpc

* display retries

* reset ping retries once successful ping and also ensure mn is in winnerslist

if mn is in winnerlist + 6 signatures a ping is accepted, if mn is in winners list between 0 and 6 signatures it wont be valid but it will not DOS ban peer, if  you are outside of the list entirely you should ban the peer.

The idea is that once you get into the winners list and have enough signatures its ok to start pinging. This avoids the race condition of pingers being banned and being accepted because thre is a delay between having 0 and 6 signatures, outside of the winners list

* change to 2 to test

* remove unused vars

* accept first ping on new broadcast, dont check winners list for that

* dont expire if lastPing is empty

* move enabled after checks and fix fSentinelIsCurrent

* move force check after enabled set

* reset retries only if mn enabled

* remove ping expired/expired states, either your enabled or not

* remove calls to isexpired and issentinelpingexpired

* idb check removed from miner

* rmove status print

* remove sentinel ping activation checks

* comment code out TODO put it back after test

* dont send ping unless in winners list with 6 sigs

* comment out ping

* enabled by default until theres a ping

* add sentinel ping expired state back

* refactor

* move preenabled check above

* relay if sentinel expired

* brackets around logic

* remove dbg

* ping from sentinelping for test

* some dbg

* allow pinging during preenabled state

* Revert "some dbg"

This reverts commit e6b06fe.

* dont increment npingretries if ourmn

* fix sentinelping call, call check every 10 min

* fix common lib

* move corewrite to server

* run mnchecks before calls that use the information

* is mn is not synced set to preenabled

* ensure we lock main first in istransactionvalid

* move pingretries to masternode info

* set pingretries to mnb list

* store ping retries in setmnping

* check retries from broadcast

* use references not copy

* some helpful debug

* dbg

* remove dbg

* allow force check on masternodemanager::Check

pass false by default but pass in true in the rpc calls and istransactionvalid. Don't update npingretries if passed in true, only when its not forced do we update that variable to avoid adding it multiple times

* set sentinel valid by default on broadcast

* set max retries to 60 and maintenance to every min

* only check ping in winners list if winners list is synced

* check winnerslist on sending ping only if list is synced

* serialize ping retries

* manage local ping retries

* check entire list is synced not just mn list

* startup preenabled for 10s

* remove dbg

* send ping only if not preenabled and is synced

* only check masternodes if they are in the winnerslist (or not enabled)

* output for not being in winners list in check

* use adjusted time instead of lastping for preenabled check

* use function already created IsBroadcastedWithin

* future sig time check fix

* give atleast MASTERNODE_SENTINEL_PING_MAX_SECONDS/2 seconds to propogate

* update to MASTERNODE_MIN_MNB_SECONDS

* also adjust broadcast check

* flip logic

* fix switching from pre to enabled state

* const references

* remove mn cache's

* update proto and hrp

* set sigs to 6 again

* update get to 1.8.23

* Updated relayer to 1.0.4

* geth default data dir in syscoin data dir

* pull out boot nodes

* Updated relayer to 1.0.4

* add net specific to data dir

* remove non-witness enforcement in mempool

this was taken out before but we missed this check, non-witness should be allowed

* update min proto

* skip check on empty winners list

* show active time from gettime if no ping has been processed yet

* check for multiple instances of daemon properly

* fix result check on pidof syscoin call result

* check asset consensus checks during mempool validation

* fix tests

* rmv no wallet build

* fix build

* remove graph from lint

* Removed graph and regex from libboost

* fix build

* fix build

* fix build

* typo

* remove main lock on istransactionvalid

the main lock in masternode::check is avoided from istransactionvalid because fForce=true and we only check isoutpoint which locks main in check() is fForce==false

* compile

* expire masternode sanity check if ping retries >= max

* compile

* dbg for getblockchaininfo

* remove potential for deadlock scenarios

* remove retry param

* process mncheck first time on startup

* better process killing

* implement our asset index spec part 1

* Bumped Masternode major version

* finish asset index spec

* add listassetindexassets RPC

* 3.8gb min

* flipped logic suppoed to end firsttime

* add block hash txid mapping to checksyscoininputs

* asset index configuration parameters

* check index on listasset rpc call

* check assets rpc call aswell

* check for assetindex before using it to init db

* default to page 0

* compile

* read txids regardless if doesnt exist

* compile

* write to asset page when changing or initializing

* remove double booking of asset index, refactor into standard functions and reuse

fix missing assetindextxid in the mintsyscoin indexing routine

* fix listassetallocation logic

* re-org txid structures so they don't double up during forks

* order LIFO on results

* refactor reorg of asset index

1) only erase payload
2) when scanning if payload doesn't exist, continue
this means that some pages may not be full (that is the tradeoff but its a performance based tradeoff). The alternatve is to remove TXIDs but that is more intensive because you have to remove for every receiver/sender which requires one to scan back pages to find txid.

* add erase txid on disconnect

we still have to rollback txid's on disconnect, may be slow but for now its required

* remove unneeded code

* remove potential deadlocking scenario with updatelastpaid/check

* remove potential deadlocks from mn rpc calls

updatelastpaid has cs_main locking inside but locks another cs before that, so we have to ensure we lock main first

* minor code cleanup

* dec page if empty

* ensure page doesnt overflow

* skip check() if not in winners list instead of enabled

also ping retries only if not preenabled to avoid cases where preenabled mn's start to count up since we are allowing non enabled checks now to happen in check() we need to ensure that the mn is in the right state before ping retries is incremented.

* fix while loop issue by changing page to int64_t

* only increase ping retries if votes >= 6 and on winners list, otherwise check every mn every 60 seconds

* remove def versions not needed

* fixed typo

* check for >= 0 votes not >= 6 for pings

* fix serialization of daemon version

* remove ban scenario

* Revert "check for >= 0 votes not >= 6 for pings"

This reverts commit a6c2d79.

* update major mn version

* Revert "remove ban scenario"

This reverts commit ff53b31.

* update version

* make new mn qualify after 4 rounds

* update version and genesis hash

* reset mn version

* masternode payment updates

1) on istransactionvalid, where before it would have failed and halted processing block, we also let it go if the paid masternode is NSR
2) don't allow any NSR votes to be accepted (which helps 1 from people trying to vote for themselves even though they are NSR)
3) in check() allow up to 49 ping retries while 6 or more votes are required in the winners list for a masternode and no pings detected in last 10 minutes. Once it gets to 50 or more, it wont check for the winners list and will update every minute if no pings are detected for a masternode until 60, then NSR state ensues

* non enabled mn cannot ping

* remove node1

* update genesis and proto

* increae sleep between kills

* message for signing

* start in preenabled state

* process msgs while geth isn't synced

* accept connections unless mn and not geth synced

* watch for first time geth is synced

* Patched relayer

* Revert "Patched relayer"

This reverts commit aea436f.

* remove min proto

* update versions

* dont do auxpow for genesis

* fix collateral proposal tx check by using new gettx function with blockindex

removes need for txindex, previous version using accessbytxid wasn't working properly because for opreturn outputs it wouln't return true and if any change outputs were spent then accessbytxid on a burn tx would actually return "cant find collateral tx"

* strip out syscoin stuff from scriptPubKey before adding to vin

* fix witness related functionality

syscointxfund wouldnt add the owner output if it found a witness output since the tx fee calc would have been satisfied, instead we start our calculation from 0 everytime so it adds the outputs we need from the owner of the asset (the address passed in).

We also dont need to do any stripping as scantxoutset doesn't give us the syscoin outputs

* remove log output

* update chain params

* refactor custom outputs in favor of tx versioning

remove custom outputs and use tx versioning to understand syscoin txs. This removes an output for every syscoin tx so its more efficient and also more compatible with bitcoin related integrations like electrum which parses transactions to sign, because no custom outputs it wont need a handler to parse syscoin transactions.

Also refactored burning of sys/assets. For assets do not use cassetallocation serialized into opreturn but put the data needed to validate the burn in opreturn manually and recreate cassetallocation when the transaction is deserialized.

* Updated relayer to 1.0.5

* fix bugs

* fix assetsend/assetallocationsend and burn issues

* Add function to close all Db's and reload the databae environment

Adds a ReloadDbEnv function to BerkeleyEnvironment in order to close all Db
instances, closes the environment, resets it, and then reopens
the BerkeleyEnvironment.

Also adds a ReloadDbEnv function to BerkeleyDatabase that calls
BerkeleyEnvironment's ReloadDbEnv.

Github-Pull: bitcoin#12493
Rebased-From: 5d296ac

* After encrypting the wallet, reload the database environment

Calls ReloadDbEnv after encrypting the wallet so that the database
environment is flushed, closed, and reopened to prevent unencrypted
keys from being saved on disk.

Github-Pull: bitcoin#12493
Rebased-From: d7637c5

* Move BerkeleyEnvironment deletion from internal method to callsite

Instead of having the object destroy itself, having the caller
destroy it.

Github-Pull: bitcoin#12493
Rebased-From: a769461

* No longer shutdown after encrypting the wallet

Since the database environment is flushed, closed, and reopened during
EncryptWallet, there is no need to shut down the software anymore.

Github-Pull: bitcoin#12493
Rebased-From: c1dde3a

* wallet: Add WalletLocation utility class

Github-Pull: bitcoin#14350
Rebased-From: 01a4c09

* wallet: Refactor to use WalletLocation

Github-Pull: bitcoin#14350
Rebased-From: 65f3672

* wallet: Fix duplicate fileid

Github-Pull: bitcoin#14320
Rebased-From: 2d796fa

* tests: add test case for loading copied wallet twice

Github-Pull: bitcoin#14320
Rebased-From: 4ea7732

* wallet: Add trailing wallet.dat when detecting duplicate wallet if it's a directory.

Github-Pull: bitcoin#14552
Rebased-From: 15c93f0

* Refactor: Move m_db pointers into BerkeleyDatabase

This is a refactoring change that doesn't affect behavior. The motivation
behind the change is give BerkeleyEnvironment objects access to
BerkeleyDatabase objects so it will be possible to simplify the duplicate
wallet check and more reliably avoid opening the same databases twice.

Github-Pull: bitcoin#14552
Rebased-From: c456fbd

* wallet: Create IsDatabaseLoaded function

Github-Pull: bitcoin#14552
Rebased-From: 5912031

* Free BerkeleyEnvironment instances when not in use

Instead of adding BerkeleyEnvironment objects permanently to the g_dbenvs map,
use reference counted shared pointers and remove map entries when the last
BerkeleyEnvironment reference goes out of scope.

This change was requested by Matt Corallo <git@bluematt.me> and makes code that
sets up mock databases cleaner. The mock database environment will now go out
of scope and be reset on destruction so there is no need to call
BerkeleyEnvironment::Reset() during wallet construction to clear out prior
state.

This change does affect bitcoin behavior slightly. On startup, instead of same
wallet environments staying open throughout VerifyWallets() and OpenWallets()
calls, VerifyWallets() will open and close an environment once for each wallet,
and OpenWallets() will create its own environment(s) later.

Github-Pull: bitcoin#11911
Rebased-From: f1f4bb7

* Trivial: add doxygen-compatible comments relating to BerkeleyEnvironment

Github-Pull: bitcoin#11911
Rebased-From: 14bc2a1

* Tests: add unit tests for GetWalletEnv

Github-Pull: bitcoin#11911
Rebased-From: 88b1d95

* wallet: Close dbenv error file db.log

The error file db.log is opened by BerkeleyEnvironment instance and
should be closed after dbenv is closed.

Github-Pull: bitcoin#15297
Rebased-From: 8602a1e

* wallet: Close wallet env lock file

Close .walletlock file when a BerkeleyEnvironment is deleted.

Github-Pull: bitcoin#15297
Rebased-From: 2f8b8f4

* qa: Test .walletlock file is closed

Github-Pull: bitcoin#15297
Rebased-From: d3bf3b9

* Merge bitcoin#15575: 0.17: Backport 15297

fe95f84 qa: Test .walletlock file is closed (João Barbosa)
2e9e904 wallet: Close wallet env lock file (João Barbosa)
22cdb6c wallet: Close dbenv error file db.log (João Barbosa)
f20513b Tests: add unit tests for GetWalletEnv (Pierre Rochard)
85c6263 Trivial: add doxygen-compatible comments relating to BerkeleyEnvironment (Pierre Rochard)
f22d02f Free BerkeleyEnvironment instances when not in use (Russell Yanofsky)
0a9af2d wallet: Create IsDatabaseLoaded function (Chun Kuan Lee)
7751ea3 Refactor: Move m_db pointers into BerkeleyDatabase (Russell Yanofsky)
caf1146 wallet: Add trailing wallet.dat when detecting duplicate wallet if it's a directory. (Chun Kuan Lee)
34da2b7 tests: add test case for loading copied wallet twice (Chun Kuan Lee)
8965b6a wallet: Fix duplicate fileid (Chun Kuan Lee)
16e5759 wallet: Refactor to use WalletLocation (João Barbosa)
21693ff wallet: Add WalletLocation utility class (João Barbosa)
1c98a75 No longer shutdown after encrypting the wallet (Andrew Chow)
435df68 Move BerkeleyEnvironment deletion from internal method to callsite (Andrew Chow)
048fda2 After encrypting the wallet, reload the database environment (Andrew Chow)
f455979 Add function to close all Db's and reload the databae environment (Andrew Chow)

Pull request description:

  This PR backports the following pull requests:
   - bitcoin#12493 [wallet] Reopen CDBEnv after encryption instead of shutting down
   - bitcoin#14350 Add WalletLocation class
   - bitcoin#14320 [bugfix] wallet: Fix duplicate fileid detection
   - bitcoin#14552 wallet: detecting duplicate wallet by comparing the db filename.
   - bitcoin#11911 Free BerkeleyEnvironment instances when not in use
   - bitcoin#15297 wallet: Releases dangling files on BerkeleyEnvironment::Close

Tree-SHA512: 52d759bc4f140ca96e39b37746cc20e786741b08ddc658a87ea77fbcfbb481f1c7b75aba4fc57ca9bca8ca7154e535da1fdd650fd114873655cd85c490c79f14

* depends: Enable unicode support on dbd for Windows

* Log progress while verifying blocks at level 4.

When verifying blocks at startup, the progress is printed in 10%
increments to logs.  When -checklevel=4, however, the second half
of the verification (connecting the blocks again) does not log the
progress anymore.  (It is still computed and shown in the UI, but
not printed to logs.)

This change makes the behaviour consistent, by adding the missing
progress logging also for level-4 checks.

* Hide spendable label if priveate key is disabled

* index: Fix for indexers skipping genesis block.

* compile

* remove files

* fix parsing to support current contract format

* use blockindex to index blocks per txid

* tx version based assets

* restructure spv proof call to give list of tx's and index of tx because the internal proof isnt a merkle proof, used bitcoin-proof instead

* reorg strategy around burn sender in mint asset tx

on mint, the sender is set to burn address, remove from burn when minting and ensuring balance > 0

on disconnect, the sender is set to burn address, add to burn when disconnecting

* minor bug fixes

dont send mn related sync commands to older clients.
isstd for burn txs
serialize tx through syscoingetspvproof as non witness

* allow for easy way to call assetssend and allocationsend without object

* fix assetupdate stripping contract and fix assettxtojson

* update contract

* remove burn sig + fix ethereum abi parsing

burn signature not needed as the function signature is always the same.
ethereum abi parsing was broken waiting for final form, now that we know what that is we hook it up and fix the parsing once and for all.

* dont adjust total sipply on mints/burns

* remove powf

* Removed regtest test files for node2 and node3

* fix order of params

* fix rlp data decode to bytes on parseethmethod check

also set max headers to 1 week based on light chain syncing roughly 1 week usually
prune headers on every set call as well to avoid big db

* working spv proofs

* Fix make install

* remove wallet dependency on sys rpc functions

everything except syscointxfund

* miner fixes

* wait till geth is synced before validating

* remove availablecointypes

we don't need to worry about specific masternode amount types since we now have multi wallet support, masternodes should be set up on a seperate wallet if they want to keep their utxos intact and not spend accidentily

* revert unintentional change

* check if geth synced before mint in rpc

* dont write to indexes if miner mode or just checking

since miner does a sanity run around syscoin consensus on txs  we actually want to make sure no database writes happen during this time, just the normal balance maps which are thrown away after checksyscoininputs is finished.

* renamed bins

* move wallet masternode stuff out of cwallet

* move sys consensus code to seperate file

* update header

* remove multiple outputs and wallet check for small sys inputs

* Fixed win64 bin names

* move static decl's out of header, causing issues

put in cpp files it seems it was declaring multiple times causing a test to fail regularily

convert sorted_vector to unsorted_set for standards

* only do concurrent processing for zdag transactions (allocation send)

* save 3*MAX_ETHEREUM_TX_ROOTS before pruning and check txroot if exists

we want to validate txroot if it exists, likely your node will have up to 120k txroots (last 3 weeks approx) so when you turn node off and on it should process all txroots of last 3 weeks if it finds them.

once you are synced up it should enforce mint transactions be a week(40k  eth blocks) or under otherwise its not allowed.

this fixes the case where a txroot may be invalid and then you turn node off and on and it skips validation, it should not proceed if txroot is invalid. When syncing up pull the txroot from db and validate it if it exists, if it doesn't exist AND your already synced then it should return an error.

* rmv files

* use blockindex to find collateral tx in governance proposal validation code

this means we can use existing sys3 process for proposals not breaking syshub

* rework blockindex

-blockindex is done by default, not a config option
   - this is because governance propsoal validator code needs it now, might as well make it default
-put it in its own db
-write to it using it batch processing for performance since its usually multiple writes at a time

* remove user syscointxfund requirement

this is now built into the rpc calls (except syscoinburn where it doesnt know what address to fund it from).

removed all 3000 satoshi hardcoded dust outputs in various calls, it will find the first usable output and spend that for proof of ownership

* remove txfund req from syscoinburn

we pass in funding address into syscoinburn and remove syscointxfund step for user
uhliksk pushed a commit to FxTC-Bata-development/Bataoshi that referenced this pull request May 1, 2019
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Mar 31, 2020
Summary:
From [[bitcoin/bitcoin#15188 | PR15188]]:
```
Addresses https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250
```

From [[bitcoin/bitcoin#16927 | PR16927]]:
```
Currently in Alpine Linux (latest, 3.10) in the depends system, one of
the ZeroMQ patches won't apply cleanly because the context around the
patch has changed and Alpine's patch implementation can't handle the
diff.

Some patch implementations can't handle fuzz / too much divergence from
the original code.

This PR just tweaks the context code around the patch so that
less-sophisticated patch implementations (such as on Alpine Linux) can
apply the patch without errors.
```

Backport of core [[bitcoin/bitcoin#15188 | PR15188]] and [[bitcoin/bitcoin#16927 | PR16927]].

Depends on D5603.

Test Plan: Run the Gitian builds.

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D5607
ftrader pushed a commit to bitcoin-cash-node/bitcoin-cash-node that referenced this pull request Aug 13, 2021
Summary:
From [[bitcoin/bitcoin#15188 | PR15188]]:
```
Addresses https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250
```

From [[bitcoin/bitcoin#16927 | PR16927]]:
```
Currently in Alpine Linux (latest, 3.10) in the depends system, one of
the ZeroMQ patches won't apply cleanly because the context around the
patch has changed and Alpine's patch implementation can't handle the
diff.

Some patch implementations can't handle fuzz / too much divergence from
the original code.

This PR just tweaks the context code around the patch so that
less-sophisticated patch implementations (such as on Alpine Linux) can
apply the patch without errors.
```

Backport of core [[bitcoin/bitcoin#15188 | PR15188]] and [[bitcoin/bitcoin#16927 | PR16927]].

Depends on D5603.

Test Plan: Run the Gitian builds.

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D5607
dzutto pushed a commit to dzutto/dash that referenced this pull request Sep 9, 2021
3046e5f Update zmq to 4.3.1 (Dimitris Apostolou)

Pull request description:

  Addresses https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6250

Tree-SHA512: 9daa6580d103dad5f1499de1e9f5e1990cb4d630982f4ff35dcba492d7c7fc6e6224556ffd01575c97d50c34011f6e03677601ce35bc396a9e2a083508a64340
UdjinM6 added a commit to dashpay/dash that referenced this pull request Sep 9, 2021
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants