Restore documentation that had been moved to zcash.readthedocs.io

doc/book/src/SUMMARY.md

@@ -3,14 +3,20 @@
 [zcashd](README.md)
 - [User Documentation](user.md)
   - [Platform Support](user/platform-support.md)
+  - [Wallet Backup](user/wallet-backup.md)
+  - [Shielding Coinbase Outputs](user/shield-coinbase.md)
+  - [Data Directory Structure](user/files.md)
   - [Metrics](user/metrics.md)
+  - [Using `zcashd` with Tor](user/tor.md)
+  - [Security Warnings](user/security-warnings.md)
   - [Deprecated Features](user/deprecation.md)
 - [Developer Documentation](dev.md)
+  - [DNS Seeders](dev/dnsseed-policy.md)
   - [Rust in `zcashd`](dev/rust.md)
-  - [Regtest tips and hints](dev/regtest.md)
+  - [Regtest Tips And Hints](dev/regtest.md)
   - [Platform Tier Policy](dev/platform-tier-policy.md)
   - [Deprecation Procedure](dev/deprecation.md)
 - [Design](design.md)
-  - [Chain state](design/chain-state.md)
-  - ["Coins" view](design/coins-view.md)
-  - [P2P data propagation](design/p2p-data-propagation.md)
+  - [Chain State](design/chain-state.md)
+  - ["Coins" View](design/coins-view.md)
+  - [P2P Data Propagation](design/p2p-data-propagation.md)

doc/book/src/dev/dnsseed-policy.md

@@ -0,0 +1,54 @@
+Expectations for DNS Seed operators
+====================================
+
+Zcash attempts to minimize the level of trust in DNS seeds,
+but DNS seeds still pose a small amount of risk for the network.
+As such, DNS seeds must be run by entities which have some minimum
+level of trust within the Zcash community.
+
+Other implementations of Zcash software may also use the same
+seeds and may be more exposed. In light of this exposure, this
+document establishes some basic expectations for operating DNS seeds.
+
+0. A DNS seed operating organization or person is expected to follow good
+host security practices, maintain control of applicable infrastructure,
+and not sell or transfer control of the DNS seed. Any hosting services
+contracted by the operator are equally expected to uphold these expectations.
+
+1. The DNS seed results must consist exclusively of fairly selected and
+functioning Zcash nodes from the public network to the best of the
+operator's understanding and capability.
+
+2. For the avoidance of doubt, the results may be randomized but must not
+single out any group of hosts to receive different results unless due to an
+urgent technical necessity and disclosed.
+
+3. The results may not be served with a DNS TTL of less than one minute.
+
+4. Any logging of DNS queries should be only that which is necessary
+for the operation of the service or urgent health of the Zcash
+network and must not be retained longer than necessary nor disclosed
+to any third party.
+
+5. Information gathered as a result of the operators node-spidering
+(not from DNS queries) may be freely published or retained, but only
+if this data was not made more complete by biasing node connectivity
+(a violation of expectation (1)).
+
+6. Operators are encouraged, but not required, to publicly document the
+details of their operating practices.
+
+7. A reachable email contact address must be published for inquiries
+related to the DNS seed operation.
+
+If these expectations cannot be satisfied the operator should discontinue
+providing services and contact the active Zcash development team as well as
+creating an issue in the [Zcash repository](https://github.com/zcash/zcash).
+
+Behavior outside of these expectations may be reasonable in some
+situations but should be discussed in public in advance.
+
+See also
+----------
+- [zcash-seeder](https://github.com/zcash/zcash-seeder) is a reference
+  implementation of a DNS seed.

doc/book/src/user/files.md

@@ -0,0 +1,24 @@
+Data Directory Files
+====================
+
+Files within the zcashd data directory (``~/.zcash/`` on Linux unless otherwise specified)
+include:
+
+| File                  | Description
+|-----------------------|--------------
+|``zcash.conf``         | contains configuration settings for zcashd
+|``zcashd.pid``         | stores the process id of zcashd while running
+|``blocks/blk000*.dat`` | block data (custom, 128 MiB per file)
+|``blocks/rev000*.dat`` | block undo data (custom)
+|``blocks/index/*``     | block index (LevelDB)
+|``chainstate/*``       | block chain state database (LevelDB)
+|``database/*``         | BDB database environment
+|``db.log``             | wallet database log file
+|``debug.log``          | contains debug information and general logging generated by zcashd
+|``fee_estimates.dat``  | stores statistics used to estimate minimum transaction fees and priorities required for confirmation
+|``peers.dat``          | peer IP address database (custom format)
+|``wallet.dat``         | personal wallet (BDB) with keys and transactions (keep private, back this up!)
+|``.cookie``            | session RPC authentication cookie (written at start when cookie authentication is used, deleted on shutdown)
+|``.lock``              | data directory lock file (empty)
+|``testnet3/*``         | contains testnet versions of these files, except ``zcash.conf``, if running ``-testnet``
+|``onion_private_key``  | cached Tor hidden service private key for ``-listenonion``

doc/book/src/user/security-warnings.md

@@ -0,0 +1,137 @@
+Security Warnings
+=================
+
+Security Audit
+--------------
+
+Zcash has been subjected to a formal third-party security review. For security
+announcements, audit results and other general security information, see
+https://z.cash/support/security.html
+
+Wallet Encryption
+-----------------
+
+Wallet encryption is disabled, for several reasons:
+
+- Encrypted wallets are unable to correctly detect shielded spends (due to the
+  nature of unlinkability of JoinSplits) and can incorrectly show larger
+  available shielded balances until the next time the wallet is unlocked. This
+  problem was not limited to failing to recognize the spend; it was possible
+  for the shown balance to increase by the amount of change from a spend,
+  without deducting the spent amount.
+
+- While encrypted wallets prevent spending of funds, they do not maintain the
+  shielding properties of JoinSplits (due to the need to detect spends). That
+  is, someone with access to an encrypted wallet.dat has full visibility of
+  your entire transaction graph (other than newly-detected spends, which suffer
+  from the earlier issue).
+
+- We were concerned about the resistance of the algorithm used to derive wallet
+  encryption keys (inherited from
+  [Bitcoin](https://bitcoin.org/en/secure-your-wallet)) to dictionary attacks
+  by a powerful attacker. If and when we re-enable wallet encryption, it is
+  likely to be with a modern passphrase-based key derivation algorithm designed
+  for greater resistance to dictionary attack, such as Argon2i.
+
+You should use full-disk encryption (or encryption of your home directory) to
+protect your wallet at rest, and should assume (even unprivileged) users who
+are running on your OS can read your wallet.dat file.
+
+Side-Channel Attacks
+--------------------
+
+This implementation of Zcash is not resistant to side-channel attacks. You
+should assume (even unprivileged) users who are running on the hardware, or who
+are physically near the hardware, that your `zcashd` process is running on
+will be able to:
+
+- Determine the values of your secret spending keys, as well as which notes you
+  are spending, by observing cache side-channels as you perform a JoinSplit
+  operation. This is due to probable side-channel leakage in the libsnark
+  proving machinery.
+
+- Determine which notes you own by observing cache side-channel information
+  leakage from the incremental witnesses as they are updated with new notes.
+
+- Determine which notes you own by observing the trial decryption process of
+  each note ciphertext on the blockchain.
+
+You should ensure no other users have the ability to execute code (even
+unprivileged) on the hardware your `zcashd` process runs on until these
+vulnerabilities are fully analyzed and fixed.
+
+REST Interface
+--------------
+
+The REST interface is a feature inherited from upstream Bitcoin.  By default,
+it is disabled. We do not recommend you enable it until it has undergone a
+security review.
+
+RPC Interface
+-------------
+
+Users should choose a strong RPC password. If no RPC username and password are
+set, zcashd will not start and will print an error message with a suggestion
+for a strong random password. If the client knows the RPC password, they have
+at least full access to the node. In addition, certain RPC commands can be
+misused to overwrite files and/or take over the account that is running zcashd.
+(In the future we may restrict these commands, but full node access – including
+the ability to spend from and export keys held by the wallet – would still be
+possible unless wallet methods are disabled.)
+
+Users should also refrain from changing the default setting that only allows
+RPC connections from localhost. Allowing connections from remote hosts would
+enable a MITM to execute arbitrary RPC commands, which could lead to compromise
+of the account running zcashd and loss of funds. For multi-user services that
+use one or more zcashd instances on the backend, the parameters passed in by
+users should be controlled to prevent confused-deputy attacks which could spend
+from any keys held by that zcashd.
+
+Block Chain Reorganization: Major Differences
+---------------------------------------------
+
+Users should be aware of new behavior in Zcash that differs significantly from
+Bitcoin: in the case of a block chain reorganization, Bitcoin's coinbase
+maturity rule helps to ensure that any reorganization shorter than the maturity
+interval will not invalidate any of the rolled-back transactions. Zcash keeps
+Bitcoin's 100-block maturity interval for generation transactions, but because
+JoinSplits must be anchored within a block, this provides more limited
+protection against transactions becoming invalidated. In the case of a block
+chain reorganization for Zcash, all JoinSplits which were anchored within the
+reorganization interval and any transactions that depend on them will become
+invalid, rolling back transactions and reverting funds to the original owner.
+The transaction rebroadcast mechanism inherited from Bitcoin will not
+successfully rebroadcast transactions depending on invalidated JoinSplits if
+the anchor needs to change. The creator of an invalidated JoinSplit, as well as
+the creators of all transactions dependent on it, must rebroadcast the
+transactions themselves.
+
+Receivers of funds from a JoinSplit can mitigate the risk of relying on funds
+received from transactions that may be rolled back by using a higher minconf
+(minimum number of confirmations).
+
+Logging z_* RPC calls
+---------------------
+
+The option `-debug=zrpc` covers logging of the z_* calls.  This will reveal
+information about private notes which you might prefer not to disclose.  For
+example, when calling `z_sendmany` to create a shielded transaction, input
+notes are consumed and new output notes are created.
+
+The option `-debug=zrpcunsafe` covers logging of sensitive information in z_*
+calls which you would only need for debugging and audit purposes.  For example,
+if you want to examine the memo field of a note being spent.
+
+Private spending keys for z addresses are never logged.
+
+Potentially-Missing Required Modifications
+------------------------------------------
+
+In addition to potential mistakes in code we added to Bitcoin Core, and
+potential mistakes in our modifications to Bitcoin Core, it is also possible
+that there were potential changes we were supposed to make to Bitcoin Core but
+didn't, either because we didn't even consider making those changes, or we ran
+out of time. We have brainstormed and documented a variety of such
+possibilities in [issue #826](https://github.com/zcash/zcash/issues/826), and
+believe that we have changed or done everything that was necessary for the
+1.0.0 launch. Users may want to review this list themselves.

doc/book/src/user/shield-coinbase.md

@@ -0,0 +1,137 @@
+# Shielding Coinbase UTXOs
+
+**Summary**
+
+Use `z_shieldcoinbase` RPC call to shield coinbase UTXOs.
+
+**Who should read this document**
+
+Miners, Mining pools, Online wallets
+
+## Background
+
+The current Zcash protocol includes a consensus rule that coinbase rewards must
+be sent to a shielded address.
+
+## User Experience Challenges
+
+A user can use the z_sendmany RPC call to shield coinbase funds, but the call
+was not designed for sweeping up many UTXOs, and offered a suboptimal user
+experience.
+
+If customers send mining pool payouts to their online wallet, the service
+provider must sort through UTXOs to correctly determine the non-coinbase UTXO
+funds that can be withdrawn or transferred by customers to another transparent
+address.
+
+## Solution
+
+The z_shieldcoinbase call makes it easy to sweep up coinbase rewards from
+multiple coinbase UTXOs across multiple coinbase reward addresses.
+
+    z_shieldcoinbase fromaddress toaddress (fee) (limit)
+
+The default fee is 0.0010000 ZEC and the default limit on the maximum number of
+UTXOs to shield is 50.
+
+## Examples
+
+Sweep up coinbase UTXOs from a transparent address you use for mining:
+
+    zcash-cli z_shieldcoinbase tMyMiningAddress zMyPrivateAddress
+
+Sweep up coinbase UTXOs from multiple transparent addresses to a shielded
+address:
+
+    zcash-cli z_shieldcoinbase "*" zMyPrivateAddress
+
+Sweep up with a fee of 1.23 ZEC:
+
+    zcash-cli z_shieldcoinbase tMyMiningAddress zMyPrivateAddress 1.23
+
+Sweep up with a fee of 0.1 ZEC and set limit on the maximum number of UTXOs to
+shield at 25:
+
+    zcash-cli z_shieldcoinbase "*" zMyPrivateAddress 0.1 25
+
+### Asynchronous Call
+
+The `z_shieldcoinbase` RPC call is an asynchronous call, so you can queue up
+multiple operations.
+
+When you invoke
+
+    zcash-cli z_shieldcoinbase tMyMiningAddress zMyPrivateAddress
+
+JSON will be returned immediately, with the following data fields populated:
+
+- operationid: a temporary id to use with `z_getoperationstatus` and
+  `z_getoperationresult` to get the status and result of the operation.
+- shieldedUTXOs: number of coinbase UTXOs being shielded
+- shieldedValue: value of coinbase UTXOs being shielded.
+- remainingUTXOs: number of coinbase UTXOs still available for shielding.
+- remainingValue: value of coinbase UTXOs still available for shielding
+
+### Locking UTXOs
+
+The `z_shieldcoinbase` call will lock any selected UTXOs. This prevents the
+selected UTXOs which are already queued up from being selected for any other
+send operation.  If the `z_shieldcoinbase` call fails, any locked UTXOs are
+unlocked.
+
+You can use the RPC call `lockunspent` to see which UTXOs have been locked.
+You can also use this call to unlock any UTXOs in the event of an unexpected
+system failure which leaves UTXOs in a locked state.
+
+### Limits, Performance and Transaction Confirmation
+
+The number of coinbase UTXOs selected for shielding can be adjusted by setting
+the limit parameter. The default value is 50.
+
+If the limit parameter is set to zero, the zcashd `mempooltxinputlimit` option
+will be used instead, where the default value for `mempooltxinputlimit` is
+zero, which means no limit.
+
+Any limit is constrained by a hard limit due to the consensus rule defining a
+maximum transaction size of 100,000 bytes.
+
+In general, the more UTXOs that are selected, the longer it takes for the
+transaction to be verified.  Due to the quadratic hashing problem, some miners
+use the `mempooltxinputlimit` option to reject transactions with a large number
+of UTXO inputs.
+
+Currently, as of November 2017, there is no commonly agreed upon limit, but as
+a rule of thumb (a form of emergent consensus) if a transaction has less than
+100 UTXO inputs, the transaction will be mined promptly by the majority of
+mining pools, but if it has many more UTXO inputs, such as 500, it might take
+several days to be mined by a miner who has higher or no limits.
+
+### Anatomy of a z_shieldcoinbase transaction
+
+The transaction created is a shielded transaction.  It consists of a single
+joinsplit, which consumes coinbase UTXOs as input, and deposits value at a
+shielded address, minus any fee.
+
+The number of coinbase UTXOs is determined by a user configured limit.
+
+If no limit is set (in the case when limit parameter and `mempooltxinputlimit`
+options are set to zero) the behaviour of z_shieldcoinbase is to consume as
+many UTXOs as possible, with `z_shieldcoinbase` constructing a transaction up
+to the size limit of 100,000 bytes.
+
+As a result, the maximum number of inputs that can be selected is:
+
+- P2PKH coinbase UTXOs ~ 662
+- 2-of-3 multisig P2SH coinbase UTXOs ~ 244.
+
+Here is an example of using `z_shieldcoinbase` on testnet to shield multi-sig coinbase UTXOs.
+
+- Block 141042 is almost ~2 MB in size (the maximum size for a block) and
+  contains 1 coinbase reward transaction and 20 transactions, each indivually
+  created by a call to z_shieldcoinbase.
+  - https://explorer.testnet.z.cash/block/0050552a78e97c89f666713c8448d49ad1d7263274422272696187dedf6c0d03
+- Drilling down into a transaction, you can see there is one joinsplit, with
+  244 inputs (vin) and 0 outputs (vout).
+  - https://explorer.testnet.z.cash/tx/cf4f3da2e434f68b6e361303403344e22a9ff9a8fda9abc180d9520d0ca6527d
+
+