New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Daemon crashes when sending from z address to multiple z addresses #1779

Closed
etherchain-org opened this Issue Nov 4, 2016 · 12 comments

Comments

Projects
None yet
3 participants
@etherchain-org

etherchain-org commented Nov 4, 2016

I tried to send a quite large transaction from a z address to multiple z addresses. After that the daemon crashed with the following exception:

zcashd: zcash/JoinSplit.cpp:307: libzcash::ZCProof libzcash::JoinSplitCircuit<NumInputs, NumOutputs>::prove(const boost::array<libzcash::JSInput, NumInputs>&, const boost::array<libzcash::JSOutput, NumOutputs>&, boost::array<libzcash::Note, NumOutputs>&, boost::array<boost::array<unsigned char, 601ul>, NumOutputs>&, uint256&, const uint256&, uint256&, boost::array<uint256, NumInputs>&, boost::array<uint256, NumInputs>&, boost::array<uint256, NumOutputs>&, uint64_t, uint64_t, const uint256&, bool) [with long unsigned int NumInputs = 2ul; long unsigned int NumOutputs = 2ul; uint64_t = long unsigned int]: Assertion `pb.is_satisfied()' failed.

The transaction was quite large with more than 150 recipients.

Edit: I am getting the same error when sending to just 10 recipients. Sending just to a single recipient seems to work.

@bitcartel

This comment has been minimized.

Show comment
Hide comment
@bitcartel

bitcartel Nov 4, 2016

Contributor

@etherchain-org Please try the transaction with 1.0.1 released today; it has better error reporting around JoinSplits.

@ebfull Ping.

Contributor

bitcartel commented Nov 4, 2016

@etherchain-org Please try the transaction with 1.0.1 released today; it has better error reporting around JoinSplits.

@ebfull Ping.

@bitcartel bitcartel added the wallet label Nov 4, 2016

@etherchain-org

This comment has been minimized.

Show comment
Hide comment
@etherchain-org

etherchain-org Nov 4, 2016

All the tx were issued with version 1.0.1. Unfortunately there was no error in the application log except the crash message.

All the tx were issued with version 1.0.1. Unfortunately there was no error in the application log except the crash message.

@bitcartel

This comment has been minimized.

Show comment
Hide comment
@bitcartel

bitcartel Nov 4, 2016

Contributor

Is this a reproducible problem? If you try to send to those 10 recipients, will it fail every time? If you change the amounts involved does it crash? If possible, please copy and paste the z_sendmany command you are trying to execute from the terminal (mask the zaddrs and amounts for privacy) Thanks.

Contributor

bitcartel commented Nov 4, 2016

Is this a reproducible problem? If you try to send to those 10 recipients, will it fail every time? If you change the amounts involved does it crash? If possible, please copy and paste the z_sendmany command you are trying to execute from the terminal (mask the zaddrs and amounts for privacy) Thanks.

@bitcartel

This comment has been minimized.

Show comment
Hide comment
@bitcartel

bitcartel Nov 4, 2016

Contributor

Also try running with -debug=zrpc which prints out specific information related to z_sendmany to debug.log.

Contributor

bitcartel commented Nov 4, 2016

Also try running with -debug=zrpc which prints out specific information related to z_sendmany to debug.log.

@etherchain-org

This comment has been minimized.

Show comment
Hide comment
@etherchain-org

etherchain-org Nov 4, 2016

I tried firt to send to 150+ recipients which crashed the client. After that I tried with just 10 recipients and it also crashed. After that i tried it with just one recipient and it worked. Unfortunately testing is very time consuming for me as a crash deletes all information in regards to the ongoing operations and I need to recover from them manually. Ideally the status of ongoing transactions should be saved across restarts so that we can properly recover from such crashes.

I tried firt to send to 150+ recipients which crashed the client. After that I tried with just 10 recipients and it also crashed. After that i tried it with just one recipient and it worked. Unfortunately testing is very time consuming for me as a crash deletes all information in regards to the ongoing operations and I need to recover from them manually. Ideally the status of ongoing transactions should be saved across restarts so that we can properly recover from such crashes.

@bitcartel

This comment has been minimized.

Show comment
Hide comment
@bitcartel

bitcartel Nov 4, 2016

Contributor

Thanks. Have you tried making a new 150+ recipient send? Or are you now only sending with <10 recipients?

Contributor

bitcartel commented Nov 4, 2016

Thanks. Have you tried making a new 150+ recipient send? Or are you now only sending with <10 recipients?

@etherchain-org

This comment has been minimized.

Show comment
Hide comment
@etherchain-org

etherchain-org Nov 4, 2016

Right now I am not doing any sends at all and try to migrate user balances over to regular t addresses. It is simply to risky to run payouts that might result in a client crash as it is impossible to recover from those crashes properly (as all the information on ongoing opid's is lost.

It would be good to have an automated test case where some blocks are mined, the mined coins are protected and after that sent to a large number of randomly generated t and z addresses.

Right now I am not doing any sends at all and try to migrate user balances over to regular t addresses. It is simply to risky to run payouts that might result in a client crash as it is impossible to recover from those crashes properly (as all the information on ongoing opid's is lost.

It would be good to have an automated test case where some blocks are mined, the mined coins are protected and after that sent to a large number of randomly generated t and z addresses.

@bitcartel

This comment has been minimized.

Show comment
Hide comment
@bitcartel

bitcartel Nov 4, 2016

Contributor

In the z-address you were sending from, roughly how many unspent notes were at that address and how many do you think should have been used to complete the send to the 10 recipients? e.g. do you have 1 big note e.g. 10.0 which is sufficient to send 0.5 x 10, or do you have a bunch of smaller notes?

Contributor

bitcartel commented Nov 4, 2016

In the z-address you were sending from, roughly how many unspent notes were at that address and how many do you think should have been used to complete the send to the 10 recipients? e.g. do you have 1 big note e.g. 10.0 which is sufficient to send 0.5 x 10, or do you have a bunch of smaller notes?

@etherchain-org

This comment has been minimized.

Show comment
Hide comment
@etherchain-org

etherchain-org Nov 4, 2016

Could you point me in the right direction in order to find out how many unspent notes are available for a given address?

Could you point me in the right direction in order to find out how many unspent notes are available for a given address?

@bitcartel

This comment has been minimized.

Show comment
Hide comment
@bitcartel

bitcartel Nov 4, 2016

Contributor

No rpc call exists for that right now. This would help debugging if you have time:

  1. On a new clean system, install zcashd, copy wallet.dat (or export/import the zaddr key)
  2. Launch and wait for node to sync. Balance for the zaddr should match that of your real node.
  3. Shut down node and disconnect system from the internet
  4. Launch node with debug=zrpc
  5. Execute the z_sendmany
  6. After crash, examine output in debug.log (this will show the unspent notes when z_sendmany invoked, and the creation of joinsplits). If you can sanitize the output and share here or via email, that would be great.
Contributor

bitcartel commented Nov 4, 2016

No rpc call exists for that right now. This would help debugging if you have time:

  1. On a new clean system, install zcashd, copy wallet.dat (or export/import the zaddr key)
  2. Launch and wait for node to sync. Balance for the zaddr should match that of your real node.
  3. Shut down node and disconnect system from the internet
  4. Launch node with debug=zrpc
  5. Execute the z_sendmany
  6. After crash, examine output in debug.log (this will show the unspent notes when z_sendmany invoked, and the creation of joinsplits). If you can sanitize the output and share here or via email, that would be great.
@bitcartel

This comment has been minimized.

Show comment
Hide comment
@bitcartel

bitcartel Nov 5, 2016

Contributor

@etherchain-org I have replicated the issue. Did you find this problem existed with 1.0.0 release or only after you started using 1.0.1 ?

Contributor

bitcartel commented Nov 5, 2016

@etherchain-org I have replicated the issue. Did you find this problem existed with 1.0.0 release or only after you started using 1.0.1 ?

bitcartel added a commit to bitcartel/zcash that referenced this issue Nov 5, 2016

Fixes #1779 so that sending to multiple zaddrs no longer fails.
Commit 2eeb6b randomized the order of input and output notes,
but this is now known to prevent the chaining of multiple joinsplits
in a single transaction.  The root cause has yet to be determined.

This patch is a temporary fix and disables the shuffling of input
and output notes.  It also adds a chained joinsplit test to the
python qa test suite.
@etherchain-org

This comment has been minimized.

Show comment
Hide comment
@etherchain-org

etherchain-org Nov 5, 2016

Thanks for your efforts. I did not experience the issue with 1.0.0 only with 1.0.1.

Thanks for your efforts. I did not experience the issue with 1.0.0 only with 1.0.1.

bitcartel added a commit to bitcartel/zcash that referenced this issue Nov 5, 2016

bitcartel added a commit to bitcartel/zcash that referenced this issue Nov 5, 2016

bitcartel added a commit to bitcartel/zcash that referenced this issue Nov 5, 2016

Revert "Remove stale comment"
This reverts commit 328d39d.
Part of fix for #1779

bitcartel added a commit to bitcartel/zcash that referenced this issue Nov 5, 2016

Revert "Randomize JoinSplits in z_sendmany"
This reverts commit 2eeb6be.
Part of fix for #1779.

bitcartel added a commit to bitcartel/zcash that referenced this issue Nov 5, 2016

Add GenIdentity, an identity function for MappedShuffle.
We use this function in z_sendmany as part of the fix for #1779.

@str4d str4d modified the milestones: 1.0.2, 1.0.3 Nov 5, 2016

zkbot pushed a commit that referenced this issue Nov 6, 2016

zkbot
Auto merge of #1790 - bitcartel:1779_send_multiple_zaddrs_logic_error…
…, r=bitcartel

Fixes #1779 so that sending to multiple zaddrs no longer fails.

Closes #1779

Commit 2eeb6b randomized the order of input and output notes,
but this is now known to prevent the chaining of multiple joinsplits
in a single transaction.  The root cause has yet to be determined.

This patch is a temporary fix and disables the shuffling of input
and output notes.  It also adds a chained joinsplit test to the
python qa test suite.

@zkbot zkbot closed this in #1790 Nov 6, 2016

zkbot pushed a commit that referenced this issue Nov 16, 2016

zkbot
Auto merge of #1797 - ebfull:improve-joinsplit-diagnostics, r=bitcartel
Improve joinsplit diagnostics

I don't advocate merging this for the hotfix release (to fix #1779) but this PR can be used to diagnose the real issue and should be merged ASAP afterward.

~I still need to add tests for `last()` and `element()` though.~ Done.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment