Merge tag 'v1.0.5' into ondrejsika/regtest-200-9

Zcash 1.0.5

This release contains a number of bug fixes and minor usability improvements, including:

1. The chain is now fully rescanned when keys are imported that are older than the wallet. (zcash#1978)
2. The number of commitments in the note commitment tree is now displayed by `getblockchaininfo`. (zcash#1946)
3. `zcash.conf` now must exist in order to start zcashd. (zcash#2013)
4. Fixed a bug where `z_sendmany` logged incorrect txid fragments when sending from transparent addresses. (zcash#1980)
5. We integrated upstream's cookie-based RPC authentication. (zcash#1999)
6. We added a restriction to wallet export paths to protect user security. (zcash#2006)
7. `z_getoperationstatus` is now sorted chronologically. (zcash#2015)
8. Messages containing newlines are now rendered properly by the metrics UI. (zcash#1972)
9. We added more tools for benchmarking JoinSplit creation. (zcash#1953)
10. We now show serialized transaction size in `listtransactions`, more operation details in `z_getoperationstatus`, and the age of the note being spent in `z_sendmany` logging. (zcash#2001, zcash#1976, zcash#1977)
11. We now instruct users to run `fetch-params` if the parameters could not be found locally. (zcash#1979)
12. We handle exceptions better in some situations for more user-friendly error messages. (zcash#1976)

For a more complete list of changes, see our [1.0.5 milestone](https://github.com/zcash/zcash/milestone/49?closed=1).

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,42 @@
+<!--- Remove text and sections that do not apply -->
+
+This issue tracker is only for technical issues related to zcashd.
+
+General Zcash questions and/or support requests and are best directed to the [Zcash Forums](https://forum.z.cash) or [Community Rocket.Chat](https://chat.zcashcommunity.com).
+
+For reporting security vulnerabilities or for sensitive discussions with our security team, please contact [security@z.cash](mailto:security@z.cash). You can use the [GPG key](https://z.cash/gpg-pubkeys/security.asc) (fingerprint: `AF85 0445 546C 18B7 86F9  2C62 88FB 8B86 D8B5 A68C`) to send an encrypted message. The key and fingerprint are duplicated on our [Public Keys page](https://z.cash/support/pubkeys.html).
+
+### Describe the issue
+Please provide a general summary of the issue you're experiencing
+
+### Can you reliably reproduce the issue?
+#### If so, please list the steps to reproduce below:
+1. 
+2. 
+3. 
+
+### Expected behaviour
+Tell us what should happen
+
+### Actual behaviour + errors
+Tell us what happens instead including any noticable error output (any messages displayed on-screen when e.g. a crash occurred)
+
+### The version of Zcash you were using:
+Run `zcashd --version` to find out
+
+### Machine specs:
+- OS name + version:
+- CPU:
+- RAM:
+- Disk size:
+- Disk Type (HD/SDD):
+- Linux kernel version (uname -a):
+- Compiler version (gcc -version):
+
+### Any extra information that might be useful in the debugging process.
+This includes the relevant contents of `~/.zcash/debug.log`. You can paste raw text, attach the file directly in the issue or link to the text via a pastebin type site.
+Please also include any non-standard things you did during compilation (extra flags, dependency version changes etc.) if applicable.
+
+### Do you have a back up of `~/.zcash` directory and/or take a VM snapshot?
+- Backing up / making a copy of the `~/.zcash` directory might help make the problem reproducible. Please redact appropriately.
+- Taking a VM snapshot is really helpful for interactively testing fixes

.gitignore

@@ -11,6 +11,7 @@ src/qt/test/test_bitcoin-qt
 
 # Zcash utilities
 src/zcash/GenerateParams
+src/zcash/CreateJoinSplit
 
 *zcashTest.pk
 *zcashTest.vk
@@ -122,3 +123,6 @@ qa/pull-tester/test.*/*
 /doc/doxygen/
 
 libzcashconsensus.pc
+
+contrib/debian/files
+contrib/debian/substvars

Makefile.am

@@ -158,6 +158,8 @@ baseline_filtered.info: baseline.info
                       "$(abs_builddir)/depends/x86_64-unknown-linux-gnu/include/gtest/*" \
                       "$(abs_builddir)/src/gtest/*" \
                       "$(abs_builddir)/src/test/*" \
+                      "$(abs_builddir)/src/wallet/gtest/*" \
+                      "$(abs_builddir)/src/wallet/test/*" \
                       -o $@
 
 leveldb_baseline.info: baseline_filtered.info
@@ -171,6 +173,8 @@ leveldb_baseline_filtered.info: leveldb_baseline.info
                       "$(abs_builddir)/depends/x86_64-unknown-linux-gnu/include/gtest/*" \
                       "$(abs_builddir)/src/gtest/*" \
                       "$(abs_builddir)/src/test/*" \
+                      "$(abs_builddir)/src/wallet/gtest/*" \
+                      "$(abs_builddir)/src/wallet/test/*" \
                       -o $@
 
 baseline_filtered_combined.info: leveldb_baseline_filtered.info baseline_filtered.info
@@ -190,6 +194,8 @@ test_bitcoin_filtered.info: test_bitcoin.info
                       "$(abs_builddir)/depends/x86_64-unknown-linux-gnu/include/gtest/*" \
                       "$(abs_builddir)/src/gtest/*" \
                       "$(abs_builddir)/src/test/*" \
+                      "$(abs_builddir)/src/wallet/gtest/*" \
+                      "$(abs_builddir)/src/wallet/test/*" \
                       -o $@
 
 zcash-gtest.info: baseline_filtered_combined.info
@@ -206,6 +212,8 @@ zcash-gtest_filtered.info: zcash-gtest.info
                       "$(abs_builddir)/depends/x86_64-unknown-linux-gnu/include/gtest/*" \
                       "$(abs_builddir)/src/gtest/*" \
                       "$(abs_builddir)/src/test/*" \
+                      "$(abs_builddir)/src/wallet/gtest/*" \
+                      "$(abs_builddir)/src/wallet/test/*" \
                       -o $@
 
 block_test.info: test_bitcoin_filtered.info
@@ -223,6 +231,8 @@ block_test_filtered.info: block_test.info
                       "$(abs_builddir)/depends/x86_64-unknown-linux-gnu/include/gtest/*" \
                       "$(abs_builddir)/src/gtest/*" \
                       "$(abs_builddir)/src/test/*" \
+                      "$(abs_builddir)/src/wallet/gtest/*" \
+                      "$(abs_builddir)/src/wallet/test/*" \
                       -o $@
 
 test_bitcoin_coverage.info: baseline_filtered_combined.info test_bitcoin_filtered.info

README.md

@@ -1,4 +1,4 @@
-Zcash 1.0.4
+Zcash 1.0.5
 ===========
 
 What is Zcash?

configure.ac

@@ -2,12 +2,12 @@ dnl require autoconf 2.60 (AS_ECHO/AS_ECHO_N)
 AC_PREREQ([2.60])
 define(_CLIENT_VERSION_MAJOR, 1)
 define(_CLIENT_VERSION_MINOR, 0)
-define(_CLIENT_VERSION_REVISION, 4)
+define(_CLIENT_VERSION_REVISION, 5)
 define(_CLIENT_VERSION_BUILD, 50)
 define(_ZC_BUILD_VAL, m4_if(m4_eval(_CLIENT_VERSION_BUILD < 25), 1, m4_incr(_CLIENT_VERSION_BUILD), m4_eval(_CLIENT_VERSION_BUILD < 50), 1, m4_eval(_CLIENT_VERSION_BUILD - 24), m4_eval(_CLIENT_VERSION_BUILD == 50), 1, , m4_eval(_CLIENT_VERSION_BUILD - 50)))
 define(_CLIENT_VERSION_SUFFIX, m4_if(m4_eval(_CLIENT_VERSION_BUILD < 25), 1, _CLIENT_VERSION_REVISION-beta$1, m4_eval(_CLIENT_VERSION_BUILD < 50), 1, _CLIENT_VERSION_REVISION-rc$1, m4_eval(_CLIENT_VERSION_BUILD == 50), 1, _CLIENT_VERSION_REVISION, _CLIENT_VERSION_REVISION-$1)))
 define(_CLIENT_VERSION_IS_RELEASE, true)
-define(_COPYRIGHT_YEAR, 2016)
+define(_COPYRIGHT_YEAR, 2017)
 AC_INIT([Zcash],[_CLIENT_VERSION_MAJOR._CLIENT_VERSION_MINOR._CLIENT_VERSION_SUFFIX(_ZC_BUILD_VAL)],[https://github.com/zcash/zcash/issues],[zcash])
 AC_CONFIG_SRCDIR([src/main.cpp])
 AC_CONFIG_HEADERS([src/config/bitcoin-config.h])

contrib/DEBIAN/changelog → contrib/debian/changelog

@@ -1,3 +1,9 @@
+zcash (1.0.5) jessie; urgency=medium
+
+  * 1.0.5 release.
+
+ -- Zcash Company <team@z.cash>  Thu, 19 Jan 2016 19:23:40 -0700
+
 zcash (1.0.4) jessie; urgency=medium
 
   * 1.0.4 release.

contrib/DEBIAN/control → contrib/debian/control

@@ -3,17 +3,17 @@ Section: utils
 Priority: optional
 Maintainer: Zcash Company <team@z.cash>
 Homepage: https://z.cash
-Build-Depends: autoconf, automake, bsdmainutils, build-essential
- git, g++-multilib, libc6-dev, libtool
- m4, ncurses-dev, pkg-config, python
+Build-Depends: autoconf, automake, bsdmainutils, build-essential,
+ git, g++-multilib, libc6-dev, libtool,
+ m4, ncurses-dev, pkg-config, python,
  unzip, wget, zlib1g-dev
 Vcs-Git: https://github.com/zcash/zcash.git
 Vcs-Browser: https://github.com/zcash/zcash
+
 Package: zcash
-Version: 1.0.4
 Architecture: amd64
-Depends: libgomp1
-Description: An implementation of the "Zerocash" protocol.
+Depends: ${shlibs:Depends}
+Description: HTTPS for money.
  Based on Bitcoin's code, it intends to offer a far higher standard
  of privacy and anonymity through a sophisticiated zero-knowledge
  proving scheme which preserves confidentiality of transaction metadata.

contrib/DEBIAN/examples/zcash.conf → contrib/debian/examples/zcash.conf

@@ -1,4 +1,4 @@
-##
+##
 ## zcash.conf configuration file. Lines beginning with # are comments.
 ##
 

contrib/DEBIAN/manpages/zcash-cli.1 → contrib/debian/manpages/zcash-cli.1

@@ -1,9 +1,9 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.47.3.
-.TH ZCASH-CLI "1" "December 2016" "Zcash RPC client version v1.0.4" "User Commands"
+.TH ZCASH-CLI "1" "January 2017" "Zcash RPC client version v1.0.5" "User Commands"
 .SH NAME
 zcash-cli \- RPC client for the Zcash daemon
 .SH DESCRIPTION
-Zcash RPC client version v1.0.4
+Zcash RPC client version v1.0.5
 .SS "Usage:"
 .TP
 zcash\-cli [options] <command> [params]

contrib/debian/manpages/zcash-fetch-params.1

@@ -0,0 +1,28 @@
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.47.3.
+.TH ZCASH-FETCH-PARAMS "1" "January 2017" "Zcash - zcash-fetch-params" "User Commands"
+.SH NAME
+zcash-fetch-params \- Downloads the Zcash network parameters
+.SH DESCRIPTION
+Zcash \- zcash-fetch\-params
+.PP
+This script will fetch the Zcash zkSNARK parameters and verify their
+integrity with sha256sum.
+.PP
+If they already exist locally, it will exit now and do nothing else.
+.PP
+This script will fetch the Zcash zkSNARK parameters and verify their
+integrity with sha256sum.
+.PP
+If they already exist locally, it will exit now and do nothing else.
+.SH "SEE ALSO"
+The full documentation for
+.B Zcash
+is maintained as a Texinfo manual.  If the
+.B info
+and
+.B Zcash
+programs are properly installed at your site, the command
+.IP
+.B info Zcash
+.PP
+should give you access to the complete manual.

contrib/DEBIAN/manpages/zcashd.1 → contrib/debian/manpages/zcashd.1

@@ -1,9 +1,9 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.47.3.
-.TH ZCASHD "1" "December 2016" "Zcash Daemon version v1.0.4" "User Commands"
+.TH ZCASHD "1" "January 2017" "Zcash Daemon version v1.0.5" "User Commands"
 .SH NAME
 zcashd \- Network daemon for interacting with the Zcash blockchain
 .SH DESCRIPTION
-Zcash Daemon version v1.0.4
+Zcash Daemon version v1.0.5
 .SS "Usage:"
 .TP
 zcashd [options]

contrib/gitian-descriptors/gitian-linux.yml

@@ -1,5 +1,5 @@
 ---
-name: "zcash-1.0.4"
+name: "zcash-1.0.5"
 enable_cache: true
 distro: "debian"
 suites:

doc/authors.md

@@ -1,23 +1,23 @@
 Zcash Contributors
 ==================
 
-Jack Grigg (301)
-Simon Liu (207)
-Sean Bowe (176)
+Jack Grigg (323)
+Simon Liu (220)
+Sean Bowe (180)
 Taylor Hornby (65)
 Daira Hopwood (62)
 Kevin Gallagher (38)
-Jay Graber (34)
+Jay Graber (35)
+Wladimir J. van der Laan (10)
 Nathan Wilcox (10)
-Wladimir J. van der Laan (9)
 Pieter Wuille (8)
 Cory Fields (7)
+Paige Peterson (5)
 ITH4Coinomia (4)
 David Mercer (4)
 4ZEC (4)
 lpescher (3)
 Patrick Strateman (3)
-Paige Peterson (3)
 MarcoFalke (3)
 Alfie John (3)
 aniemerg (2)
@@ -36,6 +36,7 @@ Lars-Magnus Skog (1)
 Jeffrey Walton (1)
 Gaurav Rana (1)
 Ethan Heilman (1)
+Eran Tromer (1)
 Christian von Roques (1)
 Chirag Davé (1)
 Cameron Boehmer (1)

doc/payment-api.md

@@ -89,6 +89,10 @@ Asynchronous calls return an OperationStatus object which is a JSON object with
   * code : number
   * message: error message
 
+Depending on the type of asynchronous call, there may be other key-value pairs.  For example, a z_sendmany operation will also include the following in an OperationStatus object:
+
+* params : an object containing the parameters to z_sendmany
+
 Currently, as soon as you retrieve the operation status for an operation which has finished, that is it has either succeeded, failed, or been cancelled, the operation and any associated information is removed.
 
 It is currently not possible to cancel operations.

doc/release-notes/release-notes-1.0.5.md

@@ -0,0 +1,58 @@
+Eran Tromer (1):
+      CreateJoinSplit: add start_profiling() call
+
+Jack Grigg (22):
+      Extend createjoinsplit to benchmark parallel JoinSplits
+      Add total number of commitments to getblockchaininfo
+      Only enable getblocktemplate when wallet is enabled
+      Only run wallet tests when wallet is enabled
+      Add a tool for profiling the creation of JoinSplits
+      Exclude test binaries from make install
+      Scan the whole chain whenever a z-key is imported
+      Instruct users to run zcash-fetch-params if network params aren't available
+      Trigger metrics UI refresh on new messages
+      Strip out the SECURE flag in metrics UI so message style is detected
+      Handle newlines in UI messages
+      Suggest ./zcutil/fetch-params.sh as well
+      Update debug categories
+      Rename build-aux/m4/bitcoin_find_bdb48.m4 to remove version
+      Throw an error if zcash.conf is missing
+      Show a friendly message explaining why zcashd needs a zcash.conf
+      Fix gtest ordering broken by #1949
+      Debian package lint
+      Generate Debian control file to fix shlibs lint
+      Create empty zcash.conf during performance measurements
+      Create empty zcash.conf during coverage checks
+      Coverage build system tweaks
+
+Jay Graber (1):
+      Update release process to check in with users who opened resolved issues
+
+Paige Peterson (2):
+      Create ISSUE_TEMPLATE.md
+      move template to subdirectory, fix typo, include prompt under describing issue section, include uploading file directly to github ticket as option for sharing logs
+
+Sean Bowe (4):
+      Add test for IncrementalMerkleTree::size().
+      Add 'CreateJoinSplit' standalone utility to gitignore.
+      Add test for z_importkey rescanning from beginning of chain.
+      Bump version to 1.0.5.
+
+Simon Liu (13):
+      Fixes #1964 to catch general exception in z_sendmany and catch exceptions as reference-to-const.
+      Fixes #1967 by adding age of note to z_sendmany logging.
+      Fixes a bug where the unsigned transaction was logged by z_sendmany after a successful sign and send, meaning that the logged hash fragment would be different from the txid logged by "AddToWallet".  This issue occured when sending from transparent addresses, as utxo inputs must be signed.  It did not occur when sending from shielded addresses.
+      Bump COPYRIGHT_YEAR from 2016 to 2017.
+      Closes #1780. Result of z_getoperationstatus now sorted by creation time of operation
+      Remove UTF-8 BOM efbbbf from zcash.conf to avoid problems with command line tools
+      Closes #1097 so zcash-cli now displays license info like zcashd.
+      Fixes #1497 ZCA-009 by restricting data exporting to user defined folder.
+      Closes #1957 by adding tx serialization size to listtransactions output.
+      Fixes #1960: z_getoperationstatus/result now includes operation details.
+      Update walletbackup.py qa test to use -exportdir option
+      Add missing header required by std::accumulate
+      Increase timeout for z_sendmany transaction in wallet.py qa test
+
+Wladimir J. van der Laan (1):
+      rpc: Implement random-cookie based authentication
+

doc/release-process.md

@@ -35,7 +35,6 @@ previous release:
     README.md
     src/clientversion.h
     configure.ac
-    contrib/DEBIAN/control
     contrib/gitian-descriptors/gitian-linux.yml
 
     Build and commit to update versions, and then perform the following commands:
@@ -112,8 +111,14 @@ Notify the Zcash DevOps engineer/sysadmin that the release has been tagged. They
 
 Then, verify that nodes can connect to the testnet server, and update the guide on the wiki to ensure the correct hostname is listed in the recommended zcash.conf.
 
-## F. Update the Beta Guide
+## F. Update the 1.0 User Guide
+
 ## G. Publish the release announcement (blog, zcash-dev, slack)
+
+### G1. Check in with users who opened issues that were resolved in the release
+
+Contact all users who opened `user support` issues that were resolved in the release, and ask them if the release fixes or improves their issue.
+
 ## H. Make and deploy deterministic builds
 
 - Run the [Gitian deterministic build environment](https://github.com/zcash/zcash-gitian)

qa/pull-tester/rpc-tests.sh

@@ -15,6 +15,7 @@ testScripts=(
     'wallet_protectcoinbase.py'
     'wallet.py'
     'wallet_nullifiers.py'
+    'wallet_1941.py'
     'listtransactions.py'
     'mempool_resurrect_test.py'
     'txn_doublespend.py'

qa/pull-tester/run-bitcoind-for-test.sh.in

@@ -7,6 +7,7 @@ ZCASH_LOAD_TIMEOUT=500
 DATADIR="@abs_top_builddir@/.zcash"
 rm -rf "$DATADIR"
 mkdir -p "$DATADIR"/regtest
+touch "$DATADIR/zcash.conf"
 touch "$DATADIR/regtest/debug.log"
 tail -q -n 1 -F "$DATADIR/regtest/debug.log" | grep -m 1 -q "Done loading" &
 WAITER=$!

qa/rpc-tests/wallet.py

@@ -301,7 +301,7 @@ def run_test (self):
         opids = []
         opids.append(myopid)
 
-        timeout = 120
+        timeout = 300
         status = None
         for x in xrange(1, timeout):
             results = self.nodes[2].z_getoperationresult(opids)