Experimental feature: remote proving service #2120

str4d · 2017-02-22T01:26:02Z

This implements proving service support in z_sendmany and adds two new binaries:

zcash-proving-service: listens on (one or both of) ZMQ and WebSocket ports for witness data, and returns the corresponding proof.
ExampleProvingServiceClient (in src/zcash): creates five new JSDescriptions using the proving service to calculate the proofs, and validates each proof.

To build: CONFIGURE_FLAGS="--with-proving-service-daemon" ./zcutil/build.sh

This is an experimental feature, and the network protocol is likely to change. Until the protocol has been specified in a ZIP, do not rely on it for interoperation.

Part of #1113. Closes #2066.

str4d · 2017-02-22T01:33:31Z

Side note: I expect we will move, rename, or remove zcash-prover-client, as it is only an example. What we will want is to bundle its contents into the z_sendmany backend.

str4d · 2017-03-10T01:23:27Z

Renamed the binaries and added a conditional compilation flag.

str4d · 2017-03-10T02:50:44Z

Adjusted the wire protocol to take a list of JSProofWitness objects, made the number of JSDescriptions in the example client configurable, and added an env var for setting ./configure flags so that we can easily build this on the CI (per #1113 (comment)).

zkbot · 2017-03-11T00:40:10Z

☔ The latest upstream changes (presumably #1636) made this pull request unmergeable. Please resolve the merge conflicts.

str4d · 2017-03-15T23:31:15Z

@nathan-at-least as-is, this protocol still enables DP-Z with trusted proving services. I know you wanted what we land as DP-T to not support trusted proving services, but given that there will be a client library in front, is it sufficient for that library to provide a DP-T-only API?

zkbot · 2017-03-21T04:32:07Z

☔ The latest upstream changes (presumably #2183) made this pull request unmergeable. Please resolve the merge conflicts.

str4d · 2017-03-21T05:24:41Z

@nathan-at-least I can't remember whether you wanted to bump this from 1.0.8; if yes, then the merge conflict above should resolve itself once the --disable-rust commit from #2183 is reverted.

daira · 2017-03-21T05:57:08Z

@arcalinea and I bumped this from 1.0.8, and @nathan-at-least concurred on #zcash-dev.

As far as I understand, that commit (802ea76) will not be reverted for 1.0.9, but payment offloading is in the 1.0.9 milestone, so this conflict does need to be resolved.

str4d · 2017-03-21T21:37:54Z

configure.ac

@@ -203,6 +203,12 @@ AC_ARG_WITH([daemon],
  [build_bitcoind=$withval],
  [build_bitcoind=yes])

+AC_ARG_WITH([proving-service],
+  [AS_HELP_STRING([--with-proving-service],
+  [build zcash-proving-service (default is no)])],


Non-blocking: Just spotted this should be default=no 😝

str4d · 2017-04-05T22:50:16Z

Rebased on master.

str4d · 2017-12-21T11:36:30Z

Addressed comments by @arielgabizon and @bitcartel.

bitcartel · 2017-12-21T16:55:48Z

@zkbot retry

str4d · 2017-12-21T17:35:34Z

@zkbot try

zkbot · 2017-12-21T17:35:38Z

⌛ Trying commit 1bfe584 with merge 2ab81dd...

Experimental feature: remote proving service This implements proving service support in `z_sendmany` and adds two new binaries: - `zcash-proving-service`: listens on (one or both of) ZMQ and WebSocket ports for witness data, and returns the corresponding proof. - `ExampleProvingServiceClient` (in `src/zcash`): creates five new `JSDescription`s using the proving service to calculate the proofs, and validates each proof. To build: `CONFIGURE_FLAGS="--with-proving-service-daemon" ./zcutil/build.sh` This is an experimental feature, and the network protocol is likely to change. Until the protocol has been specified in a ZIP, do not rely on it for interoperation. Part of #1113. Closes #2066.

zkbot · 2017-12-21T18:50:34Z

☀️ Test successful - pr-try
State: approved= try=True

arielgabizon · 2017-12-21T21:56:26Z

I'm not getting the zcash-proving-service executable when compiling as instructed.

str4d · 2017-12-22T00:59:57Z

@arielgabizon the file should be generated as src/zcash-proving-service. Check the build output to ensure that it is enabled in configure, and compiled / linked?

str4d · 2017-12-22T09:47:44Z

Unfortunately, this still does not have any ACKs, and there are now only three work days until the 1.0.14 release, so I'm bumping this to 1.0.15.

arielgabizon · 2017-12-22T11:06:28Z

My mistake I did have zcash-proving-service

What exactly do I copy as the public key?

+YR%:NhHPZOa6=8RrjVYX82s%@.nVAh]?/Re{yog gave an error

arielgabizon · 2017-12-22T11:08:15Z

I should be able to ack today/monday, if I manage to run the code.
Also I strongly feel JSProofWitness should be changed to JSWitness

daira · 2017-12-21T21:25:28Z

src/zcash/JoinSplit.hpp

+            phi(phi), rt(rt), h_sig(h_sig),
+            inputs(inputs), outputs(outputs),
+            vpub_old(vpub_old), vpub_new(vpub_new) { }
+};


I really like this being an explicit class, I think it makes things clearer.

daira · 2017-12-21T21:30:06Z

src/zcash/JoinSplit.hpp

@@ -59,7 +83,7 @@ class JoinSplit {
                         const uint256& pubKeyHash
                        );

-    virtual ZCProof prove(
+    virtual JSProofWitness<NumInputs, NumOutputs> witness(


No opinion. The spec doesn't have a name for this (it's not the same as the primary input or the auxiliary input, because those have more redundancy).

daira · 2017-12-21T22:10:55Z

build-aux/m4/ax_boost_regex.m4

+#   Copying and distribution of this file, with or without modification, are
+#   permitted in any medium without royalty provided the copyright notice
+#   and this notice are preserved. This file is offered as-is, without any
+#   warranty.


Add this to contrib/debian/copyright:

Files: build-aux/m4/ax_boost_regex.m4 Copyright: 2008 Thomas Porschberg <thomas@randspringer.de>; 2008 Michael Tindal License: GNU-All-permissive-License

Actually there should be similar entries for all of the files under build-aux/m4:

Files: build-aux/m4/ax_boost_base.m4 Copyright: 2008 Thomas Porschberg <thomas@randspringer.de>; 2009 Peter Adolphs License: GNU-All-permissive-License Files: build-aux/m4/ax_boost_chrono.m4 Copyright: 2012 Xiyue Deng <manphiz@gmail.com> License: GNU-All-permissive-License Files: build-aux/m4/ax_boost_filesystem.m4 Copyright: 2008 Thomas Porschberg <thomas@randspringer.de>; 2009 Michael Tindal; 2009 Roman Rybalko <libtorrent@romanr.info> License: GNU-All-permissive-License Files: build-aux/m4/ax_boost_program_options.m4; build-aux/m4/ax_boost_unit_test_framework.m4 Copyright: 2008 Thomas Porschberg <thomas@randspringer.de> License: GNU-All-permissive-License Files: build-aux/m4/ax_boost_system.m4 Copyright: 2008 Thomas Porschberg <thomas@randspringer.de>; 2008 Michael Tindal; 2008 Daniel Casimiro <dan.casimiro@gmail.com> License: GNU-All-permissive-License Files: build-aux/m4/ax_cxx_compile_stdcxx.m4 Copyright: 2008 Benjamin Kosnik <bkoz@redhat.com>; 2012 Zack Weinberg <zackw@panix.com>; 2013 Roy Stogner <roystgnr@ices.utexas.edu>; 2014, 2015 Google Inc. contributed by Alexey Sokolov <sokolov@google.com>; 2015 Paul Norman <penorman@mac.com>; 2015 Moritz Klammler <moritz@klammler.eu> License: GNU-All-permissive-License Files: build-aux/m4/ax_gcc_func_attribute.m4 Copyright: 2013 Gabriele Svelto <gabriele.svelto@gmail.com> License: GNU-All-permissive-License Files: build-aux/m4/l_atomic.m4 Copyright: 2004-2017 Tim Kosse License: GPLv2 Files: build-aux/m4/ax_check_compile_flag.m4; build-aux/m4/ax_check_link_flag.m4; build-aux/m4/ax_check_preproc_flag.m4 Copyright: 2008 Guido U. Draheim <guidod@gmx.de>; 2011 Maarten Bosmans <mkbosmans@gmail.com> License: GPLv3-with-Autoconf-Macro-exception Files: build-aux/m4/ax_openmp.m4 Copyright: 2008 Steven G. Johnson <stevenj@alum.mit.edu>; 2015 John W. Peterson <jwpeterson@gmail.com>; 2016 Nick R. Papior <nickpapior@gmail.com> License: GPLv3-with-Autoconf-Macro-exception Files: build-aux/m4/ax_pthread.m4 Copyright: 2008 Steven G. Johnson <stevenj@alum.mit.edu>; 2011 Daniel Richard G. <skunk@iSKUNK.ORG> License: GPLv3-with-Autoconf-Macro-exception License: GPLv3-with-Autoconf-Macro-exception This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. . This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. . You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>. . As a special exception, the respective Autoconf Macro's copyright owner gives unlimited permission to copy, distribute and modify the configure scripts that are the output of Autoconf when processing the Macro. You need not follow the terms of the GNU General Public License when using or distributing such scripts, even though portions of the text of the Macro appear in them. The GNU General Public License (GPL) does govern all other use of the material that constitutes the Autoconf Macro. . This special exception to the GPL applies to versions of the Autoconf Macro released by the Autoconf Archive. When you make and distribute a modified version of the Autoconf Macro, you may extend this special exception to the GPL to apply to your modified version as well.

build-aux/m4/l_atomic.m4 comes from Filezilla which is GPLv2. This is a problem, which I will file another ticket for.

That other ticket is #2827.

daira · 2017-12-22T19:34:18Z

depends/packages/azmq.mk

+$(package)_download_path=https://github.com/zeromq/$(package)/archive/
+$(package)_file_name=$(package)-$($(package)_version).tar.gz
+$(package)_download_file=v$($(package)_version).tar.gz
+$(package)_sha256_hash=c204c731bcb7810ca3a2c5515e88974ef2ff8d0589e60a897dc238b369180e7b


~~I may be misinterpreting, but this recently fixed ticket looks like a memory safety bug: zeromq/azmq#111~~

Not to worry, that was before the 1.0 release.

daira · 2017-12-22T19:35:31Z

depends/packages/azmq.mk

+$(package)_download_path=https://github.com/zeromq/$(package)/archive/
+$(package)_file_name=$(package)-$($(package)_version).tar.gz
+$(package)_download_file=v$($(package)_version).tar.gz
+$(package)_sha256_hash=c204c731bcb7810ca3a2c5515e88974ef2ff8d0589e60a897dc238b369180e7b


zeromq/azmq#129 is a reliability bug.

daira · 2017-12-22T19:47:56Z

src/primitives/transaction.cpp

@@ -22,7 +22,7 @@ JSDescription::JSDescription(ZCJoinSplit& params,
 {
    boost::array<libzcash::Note, ZC_NUM_JS_OUTPUTS> notes;

-    auto witness = params.witness(
+    witness = params.witness(


Does this change make the witness stay in memory longer (than needed)?

daira · 2017-12-22T19:49:36Z

src/zcash-proving-service.cpp

+
+using namespace libzcash;
+
+std::string DEFAULT_ZMQ_BIND_ADDRESS = "tcp://*:8234";


Is this binding to all interfaces? Do we want to do that by default?

daira · 2017-12-22T20:20:13Z

src/zcash/ProvingServiceClient.cpp

+
+    zmq::message_t reply;
+    LogPrint("zrpc", "%s: Waiting for proofs…\n", log_id);
+    // Add 5% margin to timeout for overhead


Is this enough margin? I'd suggest 10% plus 5 seconds.

daira · 2017-12-22T20:29:26Z

src/wallet/asyncrpcoperation_sendmany.cpp

            &esk); // parameter expects pointer to esk, so pass in address
-    {
+
+    if (generateProof) {
        auto verifier = libzcash::ProofVerifier::Strict();


Was that fixed?

daira

utACK

saleemrashid · 2018-06-02T16:16:28Z

From what I can tell, this doesn't support trustless (i.e. without having the private keys) Z proving, right?

Is this still a limitation in Sapling or could this be updated to support it after Sapling activates?

Edit: Disregard this, I have found the DP-Z proposal here #2171 (comment) 😄

Edit 2: On second thoughts, that looks like an abandoned feature?

str4d · 2020-08-07T13:58:01Z

This is definitely an abandoned feature; Sapling is fast enough that this isn't needed in practice, and if it were in some context (which likely wouldn't be zcashd), we'd write that as a separate Rust binary.

A fun experiment, consigned to the dustbin of history 😄

str4d added I-SECURITY Problems and improvements related to security. SPV support labels Feb 22, 2017

str4d mentioned this pull request Feb 22, 2017

Payment Offloading protocol for light wallets. #1113

Open

str4d mentioned this pull request Feb 22, 2017

ZIP: Delegated Proving T (DPT) Protocol zcash/zips#104

Closed

str4d force-pushed the 1113-dpt branch from 0cb0747 to 0d4a039 Compare March 10, 2017 01:20

str4d force-pushed the 1113-dpt branch 2 times, most recently from 3436993 to b94d175 Compare March 10, 2017 02:41

str4d mentioned this pull request Mar 10, 2017

Add ability to pass in autoconf options to build.sh #2066

Closed

str4d changed the title ~~[WIP] Standalone proving service and demo client~~ Standalone proving service and demo client Mar 10, 2017

str4d added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 10, 2017

str4d added this to the 1.0.8 milestone Mar 10, 2017

str4d force-pushed the 1113-dpt branch from b94d175 to e54ade1 Compare March 14, 2017 21:32

str4d mentioned this pull request Mar 21, 2017

Introduce Rust/librustzcash into the zcash build system #2183

Merged

daira assigned str4d Mar 21, 2017

str4d modified the milestones: 1.0.9, 1.0.8 Mar 21, 2017

str4d commented Mar 21, 2017

View reviewed changes

str4d force-pushed the 1113-dpt branch from e54ade1 to e7a1a7e Compare April 5, 2017 22:50

str4d mentioned this pull request Apr 6, 2017

The arguments to zcutil/build.sh should not need to be in a specific order #2190

Open

str4d requested review from ebfull and nathan-at-least April 10, 2017 22:49

str4d modified the milestones: 1.0.14, 1.0.15 Dec 22, 2017

str4d mentioned this pull request Dec 22, 2017

Improve serialization of MerklePath #2831

Closed

daira reviewed Dec 22, 2017

View reviewed changes

daira approved these changes Dec 22, 2017

View reviewed changes

str4d modified the milestones: 1.0.15, 1.1.0 Feb 8, 2018

str4d removed this from the v1.1.0 milestone Mar 20, 2018

mms710 added spring_cleaning PR_cleanup labels Dec 18, 2018

str4d closed this Aug 7, 2020

str4d deleted the 1113-dpt branch August 7, 2020 13:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experimental feature: remote proving service #2120

Experimental feature: remote proving service #2120

str4d commented Feb 22, 2017 •

edited

Loading

str4d commented Feb 22, 2017

str4d commented Mar 10, 2017

str4d commented Mar 10, 2017

zkbot commented Mar 11, 2017

str4d commented Mar 15, 2017

zkbot commented Mar 21, 2017

str4d commented Mar 21, 2017

daira commented Mar 21, 2017 •

edited

Loading

str4d Mar 21, 2017

str4d commented Apr 5, 2017

str4d commented Dec 21, 2017

bitcartel commented Dec 21, 2017

str4d commented Dec 21, 2017

zkbot commented Dec 21, 2017

zkbot commented Dec 21, 2017

arielgabizon commented Dec 21, 2017

str4d commented Dec 22, 2017

str4d commented Dec 22, 2017

arielgabizon commented Dec 22, 2017

arielgabizon commented Dec 22, 2017

daira Dec 21, 2017

daira Dec 21, 2017

daira Dec 21, 2017

daira Dec 22, 2017

daira Dec 22, 2017

daira Dec 22, 2017

daira Dec 22, 2017

daira Dec 22, 2017

daira Dec 22, 2017

daira Dec 22, 2017

daira left a comment

saleemrashid commented Jun 2, 2018 •

edited

Loading

str4d commented Aug 7, 2020


		using namespace libzcash;

		std::string DEFAULT_ZMQ_BIND_ADDRESS = "tcp://*:8234";

Experimental feature: remote proving service #2120

Experimental feature: remote proving service #2120

Conversation

str4d commented Feb 22, 2017 • edited Loading

str4d commented Feb 22, 2017

str4d commented Mar 10, 2017

str4d commented Mar 10, 2017

zkbot commented Mar 11, 2017

str4d commented Mar 15, 2017

zkbot commented Mar 21, 2017

str4d commented Mar 21, 2017

daira commented Mar 21, 2017 • edited Loading

Choose a reason for hiding this comment

str4d commented Apr 5, 2017

str4d commented Dec 21, 2017

bitcartel commented Dec 21, 2017

str4d commented Dec 21, 2017

zkbot commented Dec 21, 2017

zkbot commented Dec 21, 2017

arielgabizon commented Dec 21, 2017

str4d commented Dec 22, 2017

str4d commented Dec 22, 2017

arielgabizon commented Dec 22, 2017

arielgabizon commented Dec 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daira left a comment

Choose a reason for hiding this comment

saleemrashid commented Jun 2, 2018 • edited Loading

str4d commented Aug 7, 2020

str4d commented Feb 22, 2017 •

edited

Loading

daira commented Mar 21, 2017 •

edited

Loading

saleemrashid commented Jun 2, 2018 •

edited

Loading