@presstab presstab released this Feb 11, 2019 · 78 commits to master since this release

Release Notes: v1.0.1.5

Multithreaded Zerocoin Spend Validation

batch_verify_multithread

Unit Tests: (bd8caab:presstab abe3b9f:randomzebra)
Implementation: (ec474bd:presstab)

Veil's wallet will now use multiple threads to validate zero knowledge proofs. Each spent zerocoin has one zeroknowledge proof. One transaction can have up to 20 zerocoinspends in it. It is possible to have a single block that has 100+ zerocoinspends in it. Using a single thread, it can take about 2.1 seconds (Ryzen Threadripper 1950x) to validate 40 proofs. This can be reduced to 1.1 seconds using 5 threads on the same CPU.

By default the wallet will have multi-threading of proofs set to 2 threads. If you are running the wallet on a machine with limited resources you may want to set this to 1 thread, if you are running on a machine with many cores, you may want to increase the amount of threads. You can manually set the thread count using threadbatchverify=4 (4 threads for example) in the veil.conf file.

Syncing Efficiency Improvements

Prevent full staging area (3a6277c:presstab)
When Veil is downloading the blockchain from peers, it will sync block headers first and establish the chain and blocks it needs to download. The wallet will request blocks it needs from its peers, and will do this with multiple peers. Proof of Stake requires that blocks be validated in order, and some times a block is requested from Peer 1 and a later block is requested from Peer 2. If Peer 2's block arrives before Peer 1's block, then it cannot be added to the chain yet because it is dependant on the information from Peer 1's block. In this case, the block is put into staging, and held until it can be added to the chain. This reduces the need to download the same block many times and reduces bandwidth. In previous versions of the Veil wallet, situations would arise where the staging area would become "full" and no further blocks would be downloaded, causing the syncing to become stuck.

Do not process staging and zeroknowledge proofs on the same thread (890437b:presstab)
Previously the staging area's processing of blocks would need to wait for zeroknowledge proofs to be validated, even if those proofs were not needed for a specific block, which caused syncing to be much slower.

Skip Computation In More Spots during ProcessBlock() (0fdf534:presstab)
If a block is beyond the latest hard-coded checkpoint, it skips zeroknowledge proof verification and other expensive computation checks. A few spots were not being skipped from the staging area, slowing down overall sync times.

Consensus Improvements

Ask for New Block Announcements From Multiple Peers (4a507ca:presstab)
When peers send a block header announcing that they have knowledge of a new block that has been connected to the chain, request it from up to 3 of those peers. Previously only the first peer would be asked for the block, which in many situations could be delayed by sometimes 60 seconds+ if that peer was not able to immediately respond. Now up to 3 peers will be asked for the block (if it is not already added to the chain). If redundant blocks come in, it is still only processed once. This speeds up each Veil node's ability to be on the same consensus for which chain is the main chain, and reduces the orphan rate for both PoS and PoW block creation.

Fix Reversed Operator in Consensus Algorithm Which Caused Increased Orphan Rates for v1.0.1.4 (607fe50:presstab)
A reversed operator caused v1.0.1.4 nodes to have certain scenarios where they would consider one block to have a higher score than another when the score should have been lower. This caused some orphan issues if a new block was found, and the block built onto the wrong chain.

Do not add any PoS scores into the PoW chain score (cde0433:presstab)
nChainPoW is intended to be a PoW only consideration of chain score, previously PoS was still adding to the score.

Don't Reverify Zeroknowledge Proofs (158d176:presstab)
If a zeroknowledge proof was already verified in mempool or elsewhere, don't reverify it. This saves time and allows for quicker validation of blocks and transactions.

Fix Bad Databasing and Add Ability to Reindex Zerocoin Database (71efa04:presstab e6002e6:presstab)
Previously when restarting the wallet, the application would disconnect 6 blocks and then recheck them. This is default behaviour inherited from bitcoin. Zerocoin transactions that were in these 6 blocks would be erased from the blockchain database (as should happen for a disconnected block), but the transactions were not being re-added after checking the blocks. Since all nodes do not shutdown their wallet on the same block, this did not create any issues where the chain had transactions included that should not have been, however it did cause issues with missing coins (the wallet's tracking system relies on the database to determine whether a transaction has been confirmed). To fix any bad databasing, the wallet can be started with -reindex-zdb=1.

Wallet Transactions and Balances

Abandon Stale or Invalid Wallet Transactions on Initial Load (2ef2fdd:presstab)
If a wallet has a transaction that is stale or invalid, abandon it on initial startup. This frees up any status this transaction has for that particular transaction (for example consider coins as spent or received).

Fix Minting Fees (04e8239:ncnight 29a956b:ncnight 322990a:presstab)
There were many scenarios where a Zerocoin Mint transaction had too high of a fee. This is now fixed.

Ability to Manually Lookup Wallet's Status for a Particular Owned Zerocoin (89fbff8:presstab)
Use the new lookupzerocoin RPC call to manually lookup the wallet's status for a particular zerocoin that it owns. The RPC call can find zerocoins using a serial, serial hash, pubcoin value, or pubcoin hash. This tool can be helpful for determining why the wallet is not displaying or counting a zerocoin how you expect it to be.

More Powerful Rescanning of RingCT and CT Coins (a4efc8b:presstab)
Expands the corrections that can be made by using the rescanringctwallet RPC command.

Speed Up Initial Load of Wallet (83b8e2e:presstab)
Previously loading the wallet would go through a lot of computation when loading transactions from database. This has been reduced, and significantly speeds up initial loading of the wallet.

Preferred Automint Denomination Persists Through Restarting the Wallet (b205199:furszy)
Users can now set their preferred autominting zerocoin denomination, which will persist through shutdown. This can be done using the GUI or by setting veil.conf with nautomintdenom=

Assets 17

@presstab presstab released this Jan 28, 2019 · 78 commits to master since this release

Release Notes: v1.0.1.4

  • Quicker recognition of orphans and forks.
  • Denomination filter when using spendzerocoin rpc. Allows the user to specify a specific zerocoin denomination to spend with. For example if someone wants to 10 denoms into 100's they would want to apply a 10 filter.
  • Stake miner will not stall if it is "close enough" to the chain tip.
  • Reduce stake miner's tendency to produce redundant stakes.
Assets 19

@4x13 4x13 released this Jan 25, 2019 · 78 commits to master since this release

Release Notes: v1.0.1.2

New Wallet Features and Tools

RPC/Console:

  • getwalletinfo adds wallet_seeds section, providing information about the wallet's current sub-wallet key seeds in and how many addresses or coins have been derived from each seed.

  • rescanringctwallet will scan through every RingCT and CT coin that is in the wallet's database. It will attempt to repair any metadata that is incorrect.

  • gettransaction now has an additional section debug which displays information about the wallet databases metadata for that specific transaction.

GUI

  • Coin Control now displays the wallet's available RingCT and CT outputs. Coin control is accessed on the send dialog using the Coin Control button. Users can select which output(s) they would like to use in their transaction, and they will be carried forward to the send dialog. Transactions are invalid if they contain different types of inputs (for example RingCT and Basecoin), so Coin Control will prevent different types from being selected at the same time. In this wallet release, the labels at the top of the dialog are not being updated, this functionality is still being worked on and tested.

  • Transaction Fees displayed clearly in the transaction list on the Overview Dialog.

  • RingCT Balance displayed on the Zerocoin Minting Dialog.

VIP-1 Soft Fork

Version 1.0.1 of the Veil wallet adds support for a soft fork to include support for VIP-1. This Veil Improvement Proposal adds enforcement of reduced stake weight for large denomination zerocoin denominations (100, 1000, and 10000). This reduction of stake weight (the likelihood of the coin to successfully stake) was advertised as a feature in Veil, and has been active in all wallets, but the enforcement of the reduction was not being done properly. This allowed for editing certain lines of the stake mining code out, allowing for full weight stakes. This gave the 100 denom about 10% more chance to stake than it is supposed to have, 20% for 1,000, and 30% for 10,000.

Since people are actively staking with the higher weight, Veil's default stake miner now removes the reduction as well. The Veil Project believes that if one person can edit the code out to increase their chances, then everyone should be able to have it by default. When enforcement of the decreased weighting goes into effect (successful activation of VIP-1 soft fork), Veil's default stake miner will revert back to using decreased weights as is required.

How soft forks work:
Each new block that is added to the blockchain has the ability to signal support for VIP-1. By default any wallet running on v1.0.1.2 will signal yes in the blocks that are produced from it.

A new voting period starts every 1440 blocks, which is approximately one day. If the day has 75% of the created blocks signalling VIP-1 support then it will consider the soft fork "locked in". After VIP-1 is locked in, there will be one day until the soft fork is activated.

As this is a soft fork, it is not mandatory for users to upgrade, but they should upgrade.

The softfork status/block signalling can be seen using getblockchaininfo:

  "bip9_softforks": {
    "pos_weight": {
      "status": "started",
      "bit": 2,
      "startTime": 1548161817,
      "timeout": 1579805817,
      "since": 27360,
      "statistics": {
        "period": 1440,
        "threshold": 1080,
        "elapsed": 819,
        "count": 24,
        "support": 0.0293040293040293,
        "possible": false
      }
    }
  }

Increased Spam and Denial of Service Protection

Background Information: A recent security analysis and associated research paper have identified some areas of concern for DoS attacks against many Proof of Stake based cryptocurrencies (link). The majority of these attacks focus on headers first syncing and spamming fake headers and invalid blocks. These spam attacks rely on headers because in most headers first implementations you are not able to identify the validity of proof of stake based solely on headers.

While Veil is less vulnerable than some other blockchains, there were a few areas that have been hardened against such attacks, and a few areas where Veil has decided to place more emphasis on the Proof of Work side of Veil's dual consensus system.

Long Range Headers Syncing: Veil will now only do "long range" headers syncing if the headers chain has a higher amount of PoW specific work than the client's best known header. This is important because it means that in order to create a spam side chain that might fill up the client's memory (or at least be monitored actively as a potential reorg) the chain would need to have spent money/resources/hash power on the spam chain, which causes a major barrier to entry and significant hurdle to launch such an attack.

Short Range Headers Syncing: If there is no established chain with more PoW work than the current main chain, then the Veil client will revert to short range syncing. With short range syncing Veil will only run 50 headers above the current active chain. These 50 headers may in fact be invalid (since they would only be PoS and thus unable to be validated without more information). Since the headers will be a higher dual-consensus work chain, the client will request the blocks for the chain. Once any of the blocks are seen as invalid, the headers chain will be disconnected.

Active Collection of Headers Garbage: One of the described attacks relies primarily on filling up the memory of a node by giving them too many headers. Veil's default client will now actively prune irrelevant block indexes that are not candidates for reorg once there are in excess of 1,000 non-main chain indexes.

Disk Exhaustion: Veil does full validation of stakes during AcceptBlock(), this means that a block will not be stored to disk unless it is associated with a valid stake. Since Veil does not rely on UTXO database for staking and instead relies on cryptographic accumulators (which can be easily instantiated on the fly to a certain block without impacting the state of the mainchain) Veil is able to check that a stake is in fact valid. The Veil client also keeps full indexing of spent zerocoins and the block they were spent on, which allows for quick checking of whether the zerocoin being spent was spent on the chain it is reorganizing to.

A possible disk attack could occur with the following scenario:
disk_exhaustion

An attack could exhaust disk resources by using at least 1 valid stake and continuously repackaging that stake into different variations of the same block, which from the blockchain's perspective gives unique valid blocks. As seen in Block 98-s3 and Block 98'-s3, without proper handling the same stake could be repackaged and have new invalid headers added on top to give the impression of a valid new chain to reorganize to, resulting in each modified version of Block 98 being stored to disk.

Veil will now circumvent such an attack by keeping track of stakes. If a repacked block with the same stake comes in, the block will be ignored unless a valid Proof of Work header is also on the reorg chain, creating much less likelihood for an exhaustion attack to occur without spending significant resources on the attack. This system will allow for the valid chain with Block 98'' to be reorganized to.

Assets 20
Jan 25, 2019
Bump v1.0.1.1
Jan 25, 2019
Bump v1.0.1.0

@4x13 4x13 released this Jan 12, 2019 · 161 commits to master since this release

v1.0.0.9 is a major improvement in quite a few areas from the last released build (v1.0.0.6). The development team has focused a lot of their time correcting the majority of issues that were causing orphan stakes, preventing spending of certain coins, syncing, and other issues. There are still some issues to overcome, but v1.0.0.9 will provide a much better experience. It is suggested that you use both rescanzerocoinwallet and rescanblockchain commands in the console to help restore any potential problems in the wallet.dat.

Issues addressed in this release:

  • Better invalid data handling on send dialog
  • dont branch based on potentially uninit var
  • relay zcspends that were not relayed for zero fee
  • More specific selection for ReacceptWalletTransactions() and SelectCoins()
  • Specify output type to mint from. Tweak automint coin select. Don't relay loose coinstake tx. Archive mint when failed spend and not in chain.
  • remove restore from file + ringCT & CT balance added to tooltip
  • lock/unlock icon #205
  • remove word wrapping in tooltip
  • Don't allow 0 threads on searchdeterministiczerocoins. Fix description on deterministiczerocoinstate.
  • Staking and automint trigger off of HeadersAndBlocksSynced().
  • rescanzerocoinwallet RPC call.
  • rpc findserial
  • rpc add clearing of mempool to abandontransaction
  • Better tracking of mempool zerocoins
  • better rescanblockchain
  • wallet confirmed balance + no zerocoin balance fix #217
  • balance + tooltip better detailed
  • nBalance not loaded on zcspend
  • Merge pull request #243 from veildev/gui-fixes-2
  • Add some failure checks to getblocktemplate call
  • Merge pull request #252 from blondfrogs/miner_logging Add some failure checks to getblocktemplate call
  • Fix saving stealth destination ephemeral key.
  • More efficient filtering of duplicate pubcoins and serials in CreateNewblock()
  • Fix mismatched output types.
  • Add zerocoinspend rules to IsStandard check.
  • More verbose mempool rejection logging.
  • Check mempool for existing owned pubcoins before using them.
  • Display stealth address on CT recv.
  • Make transaction records involving ZeroCoin and blinded values show up properly
  • Look for Zerocoin and Anon inputs in IsMine()
  • Remove pubcoins and serials from mempool on new block.
  • Fix some cases where transaction records were not showing up properly.
  • Cleanup. Fix zerocoinspend to CT displaying as 0.
  • Merge pull request #254 from presstab/wallet_scan - Wallet scans. Tx Records. Encryption. Mempool filtering.
  • Fix bad scope in miner check causing failure to create blocks.
  • Update Standard Inputs to skip Anon Inputs
Assets 20
Jan 11, 2019
Bump v1.0.0.8

@4x13 4x13 released this Jan 6, 2019 · 93 commits to 1.0.0 since this release

Included in this release:

  • Numerous Segfault fixes
  • Staking Fix

Known Issues:

  • Zerocoinminting is temporarily not working, and will be fixed within the next few hours.
  • Zerocoinspend works, but can often be stuck in mempool
Assets 20