Expose side loaded verification key to snarkyjs

[mitschabaude]

"Side-loaded" verification keys are used by Pickles to recursively verify proofs which aren't baked into the verifying circuit at compile time. This is for example necessary for zkApp proofs, which are verified in the transaction snark but can't be baked in at compile time. This seems to be an alternative way of using proof recursion, which is more flexible w.r.t. what proofs can be verified. A user reports needing this feature:

A few weeks back we talked about verifying Proofs inside ZkPrograms / Contracts with a dynamically passed verification-keys (not baked in ones). This is a feature I need for a project I want to develop

This would also be needed to develop ZkPrograms which are mutually recursive (they can verify the proofs of each other, not just in one direction). Right now, in ZkProgram there is a strict order in which the verification keys have to be compiled: If Program A wants to verify proofs of Program B, then compiling Program A requires the verification key / tag of Program B to be already available. So, there is no way that Program B could depend on Program A.

The goal is to expose the more flexible recursion style to snarkyjs.

[rpanic]

This would enable more general-style rollups which could have a "network"-state on their own, making the development of more efficient and distributed rollups possible versus the current trustless but centralized per-contract idea.

Also, more contract patterns could emerge, like upgradable proxy contracts, etc.

[maht0rz]

Would an API like this possibly made sense? What are the technical limitations and requirements of side loading?

class ZkApp extends SmartContract {
  @method
  verify(proof: AnyProof) {
      // bring in a verification key as private input
      const vk = Circuit.witness(VerificationKey, () => new VerificationKey())
      // verify the VK is one of the 'right ones'
      const isValidVk = vk.equals(vk1).or(vk.equals(vk2));
      isValidVk.assertTrue();
      // verify the proof
      vk.verify(proof);
  }
}

[mitschabaude]

That would be a nice API @maht0rz! I imagine something like this should be possible, I think the OCaml code that verifies zkApp proofs in a circuit looks similar

[maht0rz]

@mitschabaude i'd love to contribute by implementing this but i am currently bottlenecked by my knowledge of Kimchi, and how it could be exposed into snarkyjs. I personally don't know why exactly this doesnt work already, so that makes it difficult to try and figure out what needs to change in the first place.

[rpanic]

I'd be very interested in the reasons why that is not exposed by default. Is it just a matter of API or is there a different mechanism for the side-loaded keys under the hood? How does the side-loading mechanism actually work on a abstract level?

[mitschabaude]

@maht0rz @rpanic it's not exposed already / by default because to expose something from ocaml generally some work needs to happen, and back when I exposed pickles to make proving work for smart contracts I didn't have this on my radar (#115)

Work usually needs to happen on the ocaml side in this file: https://github.com/MinaProtocol/mina/blob/dfc3a21ce43904bd9d23a3d3ce2c3a2c570071f2/src/lib/snarky_js_bindings/lib/snarky_js_bindings_lib.ml#L2231

[mitschabaude]

Is it just a matter of API or is there a different mechanism for the side-loaded keys under the hood?

There's a different mechanism! Pickles supports two kinds of verification keys: "compiled" (these are what snarkyjs currently uses) and "side-loaded":
https://github.com/MinaProtocol/mina/blob/dfc3a21ce43904bd9d23a3d3ce2c3a2c570071f2/src/lib/pickles/tag.ml#L5

[maht0rz]

I'd love to see this feature hit o1js as soon as possible, this will make circuit design & design of 3rd party libraries much easier.
We want to allow users not to recompile their Protokit application chains, when they want to change what kind of off-chain proofs their Runtime accepts. Thats just one of many features that would be possible with sideloading. Is there any way to prio the implementation of the API?