You might have come across terms like "prover", "keys", and phrases like "regenerating the verification key" or "why is the key 8GB in size?" or even "the proof failed because the verification key hash was different." It can all seem a bit complex, but don't worry, we're here to make it simple.
In this article, we’re going to break down the different types of keys, explain their purposes, and show you how they are created.
Our main focus will be on the boojum, a new proof system. But if you're familiar with the old proof system, the principles we'll discuss apply there as well.
We offer 13 distinct types of 'base' circuits, including Vm, Decommitter, and others, which you can view in the full list here. In addition, there are 15 'recursive' circuits. Out of these, 13 are 'leaves,' each corresponding to a basic type, while one is a 'node,' and another is a 'scheduler' that oversees all others. You can find more details in the full list here.
In our new proof system, there's also a final element known as the compressor, or snark wrapper, representing an additional type of circuit.
It's essential to note that each circuit type requires its unique set of keys.
Also, the base circuits, leaves, node and scheduler are STARK based with FRI commitments, while the snark wrapper is SNARK based with KZG commitment. This results in slightly different contents of the keys, but their role stays the same.
In the following CPU and GPU links, you'll find GCS buckets containing the latest keys.
The primary key for a given circuit is called setup key. These keys can be substantial in size - approximately 14GB
for our circuits. Due to their size, we don't store them directly on GitHub; instead, they need to be generated.
If you’re wondering what these setup keys contain, think of them as the 'source code of the circuit.'
This implies that any modifications to a circuit necessitate the regeneration of the setup keys to align with the changes made.
To generate the proof, we need the setup key. However, to verify the proof, a much smaller key, known as the
verification key, is required.
These verification keys are available on GitHub, and you can view them here. Each verification
key is stored in a separate file. They are named in the format verification_X_Y_key.json, for example,
verification_basic_4_key.json.
Comparing these files with the list of circuits mentioned earlier, you'll notice there are 13 files named
verification_basic_Y, 15 files for leaf, one each for node and scheduler, and an additional one for wrapper.
In simpler terms, each verification key contains multiple 'hashes' or commitments, derived from different sections of the setup key. These hashes enable the proof verification process.
The hash of the verification key serves a quick reference to ensure that both parties involved are using the same keys. For instance:
- Our state keeper uses this hash to confirm that the L1 contract possesses the correct key.
- The witness generator refers to this hash to determine which jobs it should take on.
Typically, we embed these hashes directly into an environment variable for easy access. You can find an example of this here for SNARK_WRAPPER_VK_HASH.
These keys, also referred to as Common Reference Strings (CRS), are essential for KZG commitments and were a crucial part of our old proving system.
With the introduction of the new prover, CRS is only utilized in the final step, specifically during the snark_wrapper phase. However, since the computational requirements in this stage are significantly reduced compared to the past, we can rely on a smaller CRS file, namely the setup_2^24.key.
Setup keys house the ProverSetupData object, which in turn contains the full Merkle tree. This is part of the reason why setup keys can be quite large in size.
To put it in simpler terms, if we consider the circuits as a massive collection of linear equations, the setup key essentially contains all the parameters for these equations. Every detail that defines and regulates these equations is stored within the setup key, making it a comprehensive and crucial component in the proving process.
Verification keys are stored in a more accessible format, as JSON files, making it relatively easy to explore their contents.
Inside, you’ll find numerous configuration fields related to the circuit. These include the size of the circuit, the number of columns, the locations of constants, where to insert the public input, and the size of the public input, among other details.
Additionally, at the end of the file, there’s a Merkle tree hash. In our case, there are actually 16 hashes because our proving system utilizes a 'Cap' Merkle tree. Imagine a Merkle tree with 16 roots instead of just one; this design ensures that each Merkle path is slightly shorter, improving efficiency.
As previously stated, the verification key hash is derived from hash function applied to the data contained in the verification key. You can view the exact process of how the keccak hash is computed in the Verifier.sol file.
For SNARK circuits (like snark_wrapper), we use keccak as hash function. For START based circuits, we use more circuit friendly hash function (currently Poseidon2).