z_sendmany error: Witness for spendable note does not have same anchor as change input

[etherchain-org]

I receive the following error when trying to send from a z address to 10 other z addresses.

I am using the standard z_sendmany API. How are we supposed to transact with z addresses in an efficient way?!

[bitcartel]

@etherchain-org Which version of zcashd are you using? v1.0.1 or v1.0.2? Do you use debug=zrpc or debug=all when launching zcashd? Can you share some of the debug output related to the z_sendmany operation?

[etherchain-org]

I am using v1.0.2. Unfortunately I am not having any debug logging flags enabled and I would prefer not to restart the node as that will take it offline for several hours. Is there any way to change the logging level on the fly?

[bitcartel]

@etherchain-org I don't believe there is a way to change the logging level on the fly. We have not added one. Have you updated to v1.0.3 on one of your nodes and does this problem persist?

[etherchain-org]

@bitcartel I have updated my node to v1.0.3 and the issue persists.

[bitcartel]

Another user reports:

z to z transactions are still failing: async rpc opid-f3f8a153-5e7e-4b2d-b0e2-5791d9aee7bb finished (status=failed, error=Witness for spendable note does not have same anchor as change input)
was sending from one z address to 10 other z addresses

[bitcartel]

@etherchain-org As a data point, when you updated your node to v1.0.3, did you do a -reindex?

[peterbitfly]

@bitcartel I did not do a -reindex as it takes quite a lot of time with the z addresses in my wallet

[str4d]

I think that the bug is caused by a difference in anchors between the change and non-change input notes.

Let's assume that we want to send 1.5 ZEC each to four people, and we only have 1 ZEC notes. The resulting multi-JoinSplit will look like this:

• JS1
  • In: 1, 1
  • Out: 1.5, 0.5 (a)
• JS2
  • In: 0.5 (a), 1
  • Out: 1.5, 0 (b)
• JS3
  • In: 1, 1
  • Out: 1.5, 0.5 (c)
• JS4
  • In: 0.5 (c), 1
  • Out: 1.5, 0 (d)

For JS1, the anchor is the one corresponding to the fetched witnesses for the input notes. For JS2, it is the anchor corresponding to the tree state after appending the JS1 commitments to the JS1 anchor's tree state (a block tree state). Because the JS2 change is 0, the JS3 anchor is taken from a block state again.

Here's where I think the bug comes in:

• For JS4's change input note, its witness is calculated by taking the tree state for JS3's anchor, and appending the JS3 commitments.
• For JS4's non-change input notes, their witnesses are calculated by fetching witnesses for a block tree state, and appending the JS3 commitments.

But we don't lock on cs_main between these steps, or between JoinSplits (obviously because proof generation takes a while). So, what I think is happening is that JS3 is created with a particular block anchor, and then another block arrives while its proof is being calculated, and then the non-change input notes for JS4 use the anchor from the next block. This causes different commitment trees to be created for the change and non-change input notes, resulting in the given error. The bug primarily affects many-z-addr transactions, because the the transaction creation time approaches or exceeds the block interval time.

[str4d]

Thinking more about this, you could remove JS1 and JS2, and the same argument would apply. I included JS1 and JS2 because originally I thought this bug was caused by the intermediate tree state being cleared, which is not well-specified in the protocol spec.

[bitcartel]

I'll try to make a test for this. For each joinsplit in a chain, if there non-change notes to spend, we lock on cs_main to call pwalletMain->GetNoteWitnesses. So it may be that instead we need to retrieve all input note witnesses before we start processing the chain.

[str4d]

For each joinsplit in a chain, if there non-change notes to spend, we lock on cs_main to call pwalletMain->GetNoteWitnesses. So it may be that instead we need to retrieve all input note witnesses before we start processing the chain.

That would make the most sense to me, and would probably be the simplest to implement (via a zInputWitnessesDeque parallel to zInputsDequeue).

[bitcartel]

Quick update: I managed to recreate this problem manually. Developing test and fix right now.

[daira]

I remain convinced that choosing the most recent anchor for a JoinSplit is not a good idea.

[bitcartel]

That would have to be a separate PR. Currently the anchor is obtained by calling GetNoteWitnesses() on the wallet, but there is no way to specify what type of anchor to retrieve e.g. oldest, at height x, or n confirmations ago.