Ensure that testnet uses consistent zkSNARK parameters.

[nathan-at-least]

The API for libzerocash generate parameters lazily if not initialized, and we've been relying on that, in which case different nodes on the testnet network will be using different parameters and won't interoperate.

See if this is a problem and, if so, correct it.

Here's an example recent test code that examines parameter generation: Electric-Coin-Company/libzerocash@e02fc0a

Note my preferred style: alter the libzerocash API to remove lazy parameter generation and force valid parameters to always be passed to, eg, proving key / verifying key constructors, and then have a different standalone function to generate new parameters, and finally a binary that calls that function and spits out the raw parameters or a C header fragment that defines them as static constants.

[tromer]

I strongly agree.

And I have several additional related notes about the ZerocashParams class; will open a separate issue and branch about those..

[nathan-at-least]

Hm... @defuse and I were just figuring out how to do test parameter generation or distribution for the alpha and realized how in the long run, this is a big issue. Here are some related issues:

• Peers may not realize if they are using different parameters, currently. See: Ensure parameters match in the peer protocol. #409
• We need either to distribute a giant file to users, or have them deterministically generate their own. The latter only works for test parameters and exposes the toxic secrets to everyone. This also implies if we generate test parameters on a centralized box in an attempt to destroy those toxic secrets, there's the security risk of leaks and the resulting compromise of testnet proofs/verifications. See: needs ticket

[nathan-at-least]

For the alpha, we're planning to generate parameters on one of our dev machines, upload the result to a static web server, then add instructions for installing them into the correctly named paths.

[defuse]

I modified libzerocash to remove lazy generation and make saving/loading of keys easier. However, loading is taking forever. I'm watching the test I wrote run, and it's currently in:

#0  0x000000000048f4a0 in libsnark::Fp_model<4l, libsnark::alt_bn128_modulus_q>::mul_reduce (this=0x7ffecfa449f0, other=...) at depinst/include/libsnark/algebra/fields/fp.tcc:97
#1  0x00007f1073979c55 in libsnark::Fp_model<4l, libsnark::alt_bn128_modulus_q>::squared() const () from depinst/lib/libsnark.so
#2  0x00007f107397a9db in libsnark::Fp_model<4l, libsnark::alt_bn128_modulus_q>::sqrt() const () from depinst/lib/libsnark.so
#3  0x00007f1073978def in libsnark::operator>>(std::istream&, libsnark::alt_bn128_G1&) () from depinst/lib/libsnark.so

churning away doing some sort of math operations. But even after leaving it overnight, it's still there.

[tromer]

I modified libzerocash to remove lazy generation

Hurray!

loading is taking forever

Note libsnark's NO_PT_COMPRESSION flag. With point compression enabled, which is the default, the size of the proving key is roughly halved, but deserializing it requires heavy math to (recompute the omitted coordinate).

[tromer]

BTW, the math-heavy deserialization is currently single-threaded. In principle it is trivial to parallelize it, because each EC point is reconstructed independently of the rest. However, this is awkward to implement with the current de/serialization code.

[defuse]

@tromer Thanks! I'll try turning on that flag for libsnark.

The proving key starts with:

2
2
4
659788
481522
10
11

which means we're loading 481,522 points. By the magic of GDB I printed out the current value of i in the sparse vector operator>> and got 27. It's been going for half an hour now, so that point compression really is slow!

[tromer]

Something's wrong, it's not that slow... I don't recall specifically for libzerocash, but in larger libsnark tests, deserialization takes no more than a few minutes on decent platform.

[defuse]

After disabling point compression, a new error:

(enter) Loading Proving Key
terminate called after throwing an instance of 'std::length_error'
  what():  vector::_M_default_append
Aborted (core dumped)

I've seen this before in #226. Will try to get this to link statically.

[defuse]

Actually, I'm going to re-enable point compression and then find out what's using up all the time via GDB.

[defuse]

I just noticed this comment in the libsnark source code:

 Fp3_model sqrt() const; // HAS TO BE A SQUARE (else does not terminate)

I bet that's what's happening.

[defuse]

So it's hanging in the sqrt() call in the point decompression for i=27 (always the same one). I bet whatever it's taking the square root of isn't a square.

[defuse]

It could be the same bug causing both symptoms: alt_bn128_coeff_b (3) is one of the parameters initialized by init_public_params(). If it's not getting initialized per the bug in #226, it makes sense that tX2*tX + alt_bn128_coeff_b is not going to be square.

[defuse]

(gdb) x/4b 0x718c00
0x718c00 <_ZN8libsnark17alt_bn128_coeff_bE>:    -41 40  -83 80
(gdb) print *(alt_bn128_Fq *)0x718c00
No symbol "alt_bn128_Fq" in current context.
(gdb) print *(libsnark::alt_bn128_Fq *)0x718c00
$3 = {mont_repr = {static N = <optimized out>, data = {8797723225643362519, 2263834496217719225, 3696305541684646532, 3035258219084094862}}, static num_limbs = <optimized out>, 
  static mod = <optimized out>, static num_bits = <optimized out>, static euler = <optimized out>, static s = <optimized out>, static t = <optimized out>, static t_minus_1_over_2 = <optimized out>, 
  static nqr = <optimized out>, static nqr_to_t = <optimized out>, static multiplicative_generator = <optimized out>, static root_of_unity = <optimized out>, static inv = <optimized out>, 
  static Rsquared = <optimized out>, static Rcubed = <optimized out>}

(I don't know enough about the code to know if that's the correct value)

[defuse]

Ok, both disassembles have the same address, so I don't think the sqrt() thing is caused by the #226 bug.

Evidence:

   0x00000000004b2958 <+9191>:  mov    %rax,0x270fa1(%rip)        # 0x723900 <_ZN8libsnark17alt_bn128_coeff_bE>

   0x00000000004a9ad8 <+259>:   mov    $0x723900,%edx
   0x00000000004a9add <+264>:   mov    %rcx,%rsi
   0x00000000004a9ae0 <+267>:   mov    %rax,%rdi
   0x00000000004a9ae3 <+270>:   callq  0x48862a <libsnark::Fp_model<4l, libsnark::alt_bn128_modulus_q>::operator+(libsnark::Fp_model<4l, libsnark::alt_bn128_modulus_q> const&) const>

[defuse]

It's this resize() that's crashing:

template<typename T>
std::istream& operator>>(std::istream& in, sparse_vector<T> &v)
{
    in >> v.domain_size_;
    consume_newline(in);

    size_t s;
    in >> s;
    consume_newline(in);
    v.indices.resize(s);

(gdb) bt
#0  0x00007ffff57e95f8 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff57eaa7a in abort () from /usr/lib/libc.so.6
#2  0x00007ffff60fdb3d in __gnu_cxx::__verbose_terminate_handler () at /build/gcc-multilib/src/gcc-5.2.0/libstdc++-v3/libsupc++/vterminate.cc:95
#3  0x00007ffff60fb996 in __cxxabiv1::__terminate (handler=<optimized out>) at /build/gcc-multilib/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:47
#4  0x00007ffff60fb9e1 in std::terminate () at /build/gcc-multilib/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:57
#5  0x00007ffff60fbbf8 in __cxxabiv1::__cxa_throw (obj=obj@entry=0x1cf38200, tinfo=0x7ffff63e2ac0 <typeinfo for std::length_error>, dest=0x7ffff6111180 <std::length_error::~length_error()>)
    at /build/gcc-multilib/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_throw.cc:87
#6  0x00007ffff6124a3f in std::__throw_length_error (__s=0x4d049f "vector::_M_default_append") at /build/gcc-multilib/src/gcc-5.2.0/libstdc++-v3/src/c++11/functexcept.cc:86
#7  0x000000000048b6ce in std::vector<unsigned long, std::allocator<unsigned long> >::_M_check_len (this=0x7fffffffcab8, __n=18446744073709551615, __s=0x4d049f "vector::_M_default_append")
    at /usr/include/c++/5.2.0/bits/stl_vector.h:1425
#8  0x00000000004a0c93 in std::vector<unsigned long, std::allocator<unsigned long> >::_M_default_append (this=0x7fffffffcab8, __n=18446744073709551615) at /usr/include/c++/5.2.0/bits/vector.tcc:555
#9  0x000000000049e7cf in std::vector<unsigned long, std::allocator<unsigned long> >::resize (this=0x7fffffffcab8, __new_size=18446744073709551615) at /usr/include/c++/5.2.0/bits/stl_vector.h:676
#10 0x000000000049a97b in libsnark::operator>><libsnark::knowledge_commitment<libsnark::alt_bn128_G2, libsnark::alt_bn128_G1> > (in=..., v=...)
    at depinst/include/libsnark/common/data_structures/sparse_vector.tcc:289
#11 0x0000000000497e55 in libsnark::operator>><libsnark::alt_bn128_pp> (in=..., pk=...) at depinst/include/libsnark/zk_proof_systems/ppzksnark/r1cs_ppzksnark/r1cs_ppzksnark.tcc:59
#12 0x000000000049630a in libzerocash::ZerocashParams::LoadProvingKeyFromFile (path="./zerocashTest-proving-key", tree_depth=4) at libzerocash/ZerocashParams.cpp:76
#13 0x000000000040fab5 in SaveAndLoadKeysFromFiles (tree_depth=4) at tests/zerocashTest.cpp:114
#14 0x00000000004143cd in main (argc=1, argv=0x7fffffffde48) at tests/zerocashTest.cpp:558
(gdb) select-frame 10
(gdb) info locals
s = 18446744073709551615

(that value is -1)

[defuse]

It's an I/O error. I added:

    in >> s;
    std::cout << "The good is: " << in.good();
    consume_newline(in);
    std::cout << "The size s is : " << s << std::endl;
    std::cout << "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" << std::endl;

Gives:

The good is: 1The size s is : 2
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
The good is: 0The size s is : 18446744073709551615
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

So an I/O error is happening somewhere.

[defuse]

Thankfully, it is that read that's failing, not an unreported failure somewhere else in the code. Evidence:

template<typename T>
std::istream& operator>>(std::istream& in, sparse_vector<T> &v)
{
    std::cout << "0The good is: " << in.good() << std::endl;
    std::cout << "0The eof is: " << in.eof() << std::endl;
    std::cout << "0The fail is: " << in.fail() << std::endl;
    in >> v.domain_size_;
    consume_newline(in);

    size_t s;
    std::cout << "AThe good is: " << in.good() << std::endl;
    std::cout << "AThe eof is: " << in.eof() << std::endl;
    std::cout << "AThe fail is: " << in.fail() << std::endl;
    in >> s;
    std::cout << "The good is: " << in.good() << std::endl;
    std::cout << "The eof is: " << in.eof() << std::endl;
    std::cout << "The fail is: " << in.fail() << std::endl;
    consume_newline(in);
    std::cout << "The size s is : " << s << std::endl;
    std::cout << "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" << std::endl;

0The good is: 1
0The eof is: 0
0The fail is: 0
AThe good is: 1
AThe eof is: 0
AThe fail is: 0
The good is: 0
The eof is: 0
The fail is: 1

[defuse]

Now it failed differently:

The position is: 116674745
X [ 
32303635373133353235 ] <-- this is hex encoded
The good is: 1
The eof is: 0
The fail is: 0
The size s is : 3735363834383036
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
terminate called after throwing an instance of 'std::length_error'
  what():  vector::_M_default_append

No error, but it actually read huge valid integer. (Actually something is wrong here because the code to print X is in an if that checks s is -1, but down below it isn't, and inside that loop is a clear() so don't trust those flags. Errr. its because s is hex encoded because std::hex is sticky, oops.).

Here's a better one (but still hex encoded for some reason):

The position is: 116673930
X [ 
31313738353535383034 ]
The good is: 1
The eof is: 0
The fail is: 0
The size s is : ffffffffffffffff
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

[defuse]

For some reason it's proceeding on to the setgenerate command even while it's still loading the proving key (which takes forever...) and that's causing a timeout or something.

[defuse]

It's not a timeout, I switched to the faster-loading non-point-compressed parameters and it's still dying the same way.

[defuse]

Oh, what's happening is that the wait-for-wallet-to-load code has an upper limit on the wait of 30 seconds, and if it takes longer than that, it just tries to run the test anyway. That should be fixed to be a failure itself.

[defuse]

Fixed, so assuming no other problems come up, the next step is #434.

[nathan-at-least]

I'm reviewing these changes (and I just committed the supporting change in bbotzc then deployed it; it was already deployed by manual editing of the ci.leastauthority.com files locally).

[nathan-at-least]

Focusing on #438 for the time being.