implement merkle map

[es92]

Can be used for nullifiers, membership, and key -> value maps

[mitschabaude]

The implementation looks good to me with minor comments!

I have a more high-level question though: To provide a usable tool for nullifiers, don't we need something which computes the key from the leaf, and uses it for getting / setting, inside the circuit? Because we need to connect the leaf to the key inside the circuit?

Currently it seems like only MerkleMapWitness is suitable for in-circuit use, but not MerkleMap, similar to MerkleWitness vs MerkleTree.

[Comdex]

Occasionally found this pr, I also share my previous research on the use of nullifier(That needs to be handled in the circuit), it seems that we can only use the sparse merkle tree (SMT) with a tree height of 256(If I'm not mistaken, since the result of the poseidon hash in snarkyjs actually only affects 254 bits, we only need trees of height 254?) to deal with the nullifier in snarkyjs now, because SMT can generate non-membership proof and we can verify the proof in the circuit, but the disadvantage is that the verification of this proof requires more hash times, the efficiency is relatively low and it is easy to reach the upper limit of the snarkyjs circuit constraint size.

[es92]

The implementation looks good to me with minor comments!

I have a more high-level question though: To provide a usable tool for nullifiers, don't we need something which computes the key from the leaf, and uses it for getting / setting, inside the circuit? Because we need to connect the leaf to the key inside the circuit?

Currently it seems like only MerkleMapWitness is suitable for in-circuit use, but not MerkleMap, similar to MerkleWitness vs MerkleTree.

Thanks! I think this should be sufficient to implement nullifiers still, with just MerkleMapWitness actually. Following is the flow I was thinking of for membership (which nullifier should follow from, just with keys derived from preimages of commitments). Particularly in the following example, showing in circuit that the current merkle map hasn't had a public key as a member, but then in circuit, updating the merkle map to mark the public key as a member:

• Outside of the circuit, create a witness for the key you're intending to update (say its your public key)
• Inside of the circuit, use the witness to prove the MerkleMapWitness has the following properties:
  • witness actually matches the public key (this is the check that links "computes the key from the leaf")
  • value at that witness for the prior root was Field(0)
  • update the value at that witness to Field(1), compute the new root, and set the merkle map root for the contract to that

I think that's sufficient for accomplishing what we need inside the circuit? Lmk if I missed something or can explain something better too

[mitschabaude]

which nullifier should follow from, just with keys derived from preimages of commitments

This is the thing that I thought was missing. It's possible though: The MerkleMapWitness has the key baked into it implicitly, and re-computes it in the circuit when doing witness.computeRootAndKey. At that point the user can take the key and check that it matches whatever is the scheme for deriving the key from the leaf / value.

I think this is fine, but IMO it would add value to have such a key-deriving-and-checking scheme baked in already, so there's less load on users to come up with their own scheme and implement the checks correctly.

In general, the whole MerkleTree implementation imposes a 'reverse thinking' to the user, where you compute something directly outside the circuit and then verify that it matches inside the circuit. IMO, where it makes sense this should be abstracted from the user such that there are simple direct functions, like "check non-inclusion of this value" or "add this value to the tree" available for use in the circuit. (We can do the witness-computation-outside-the-circuit within snarkyjs, in a Circuit.witness block, hidden from the user) Do you agree @es92?

However, let's address this later and move forward with this PR as-is

[es92]

which nullifier should follow from, just with keys derived from preimages of commitments

This is the thing that I thought was missing. It's possible though: The MerkleMapWitness has the key baked into it implicitly, and re-computes it in the circuit when doing witness.computeRootAndKey. At that point the user can take the key and check that it matches whatever is the scheme for deriving the key from the leaf / value.

I think this is fine, but IMO it would add value to have such a key-deriving-and-checking scheme baked in already, so there's less load on users to come up with their own scheme and implement the checks correctly.

In general, the whole MerkleTree implementation imposes a 'reverse thinking' to the user, where you compute something directly outside the circuit and then verify that it matches inside the circuit. IMO, where it makes sense this should be abstracted from the user such that there are simple direct functions, like "check non-inclusion of this value" or "add this value to the tree" available for use in the circuit. (We can do the witness-computation-outside-the-circuit within snarkyjs, in a Circuit.witness block, hidden from the user) Do you agree @es92?

However, let's address this later and move forward with this PR as-is

This makes sense to me - I think I wasn't aware of some of those features too (Circuit.witness sounds very cool).

And yep the idea was to just copy the structure of Merkle Tree Witness. Would be supportive of other implementations down the road (but also like moving forward with this as is to get the functionality out at the least)