From f366378c66793bc59216060147a272b77bbe364a Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Fri, 24 Nov 2023 18:36:52 +0600 Subject: [PATCH 01/16] slow update tree explanation --- .../public_private_calls/main.md | 107 ++++++++++++++++++ .../public_private_calls/slow_updates_tree.md | 95 ++++++++++++++++ docs/sidebars.js | 12 +- 3 files changed, 213 insertions(+), 1 deletion(-) create mode 100644 docs/docs/concepts/foundation/communication/public_private_calls/main.md create mode 100644 docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/main.md b/docs/docs/concepts/foundation/communication/public_private_calls/main.md new file mode 100644 index 00000000000..683126a14a3 --- /dev/null +++ b/docs/docs/concepts/foundation/communication/public_private_calls/main.md @@ -0,0 +1,107 @@ +--- +title: Private <--> Public execution +--- + +import Image from "@theme/IdealImage"; + +import Disclaimer from "../../../misc/common/\_disclaimer.mdx"; + + + +Aztec operates on a model of private and public functions that are able to work together. Private functions work by providing evidence of correct execution generated locally through kernel proofs. Public functions, on the other hand, are able to utilize the latest state to manage updates and perform alterations. + +On this page, you’ll learn: + +- How private and public functions work +- The role of public functions in managing state alterations and updates +- Communication and interactions between private and public functions +- How the sequencer manages the order of operations of private functions + +### Objectives + +The goal for L2 communication is to setup the most simple mechanism that will support + +- _private_ and _public_ functions +- _private_ functions that can call _private_ or _public_ functions +- _public_ functions that can call _private_ or _public_ functions + +Before diving into the communication abstracts for Aztec, we need to understand some of our limitations. One being that public functions (as known from Ethereum) must operate on the current state to provide meaningful utility, e.g., at the tip. +This works fine when there is only one builder (sequencer) executing it first, and then others verifying as the builder always knows the tip. On the left in the diagram below, we see a block where the transactions are applied one after another each building on the state before it. For example, if Tx 1 update storage `a = 5`, then in Tx 2 reading `a` will return `5`. + +This works perfectly well when everything is public and a single builder is aware of all changes. However, in a private setting, we require the user to present evidence of correct execution as part of their transaction in the form of a kernel proof (generated locally on user device ahead of time). This way, the builder doesn't need to have knowledge of everything happening in the transaction, only the results. If we were to build this proof on the latest state, we would encounter problems. How can two different users build proofs at the same time, given that they will be executed one after the other by the sequencer? The simple answer is that they cannot, as race conditions would arise where one of the proofs would be invalidated by the other due to a change in the state root (which would nullify Merkle paths). + +To avoid this issue, we permit the use of historical data as long as the data has not been nullified previously. Note, that because this must include nullifiers that were inserted after the proof generation, but before execution we need to nullify (and insert the data again) to prove that it was not nullified. Without emitting the nullifier we would need our proof to point to the current head of the nullifier tree to have the same effect, e.g., back to the race conditions we were trying to avoid. + +In this model, instead of informing the builder of our intentions, we construct the proof $\pi$ and then provide them with the transaction results (new commitments and nullifiers, contract deployments and cross-chain messages) in addition to $\pi$. The builder will then be responsible for inserting these new commitments and nullifiers into the state. They will be aware of the intermediates and can discard transactions that try to produce existing nullifiers (double spend), as doing so would invalidate the rollup proof. + +On the left-hand side of the diagram below, we see the fully public world where storage is shared, while on the right-hand side, we see the private world where all reads are historic. + + + +Given that Aztec will comprise both private and public functions, it is imperative that we determine the optimal ordering for these functions. From a logical standpoint, it is reasonable to execute the private functions first as they are executed on a state $S_i$, where $i \le n$, with $S_n$ representing the current state where the public functions always operate on the current state $S_n$. Prioritizing the private functions would also afford us the added convenience of enabling them to invoke the public functions, which is particularly advantageous when implementing a peer-to-pool architecture such as that employed by Uniswap. + +Transactions that involve both private and public functions will follow a specific order of execution, wherein the private functions will be executed first, followed by the public functions, and then moving on to the next transaction. + +It is important to note that the execution of private functions is prioritized before executing any public functions. This means that private functions cannot "wait" on the results of any of their calls to public functions. Stated differently, any calls made across domains are unilateral in nature, akin to shouting into the void with the hope that something will occur at a later time. The figure below illustrates the order of function calls on the left-hand side, while the right-hand side shows how the functions will be executed. Notably, the second private function call is independent of the output of the public function and merely occurs after its execution. + + + +Multiple of these transactions are then ordered into a L2 block by the sequencer, who will also be executing the public functions (as they require the current head). Example seen below. + + + +:::info +Be mindful that if part of a transaction is reverting, say the public part of a call, it will revert the entire transaction. Similarly to Ethereum, it might be possible for the block builder to create a block such that your valid transaction reverts because of altered state, e.g., trade incurring too much slippage or the like. +::: + +To summarize: + +- _Private_ function calls are fully "prepared" and proven by the user, which provides the kernel proof along with new commitments and nullifiers to the sequencer. +- _Public_ functions altering public state (updatable storage) must be executed at the current "head" of the chain, which only the sequencer can ensure, so these must be executed separately to the _private_ functions. +- _Private_ and _public_ functions within an Aztec transaction are therefore ordered such that first _private_ functions are executed, and then _public_. + +A more comprehensive overview of the interplay between private and public functions and their ability to manipulate data is presented below. It is worth noting that all data reads performed by private functions are historical in nature, and that private functions are not capable of modifying public storage. Conversely, public functions have the capacity to manipulate private storage (e.g., inserting new commitments, potentially as part of transferring funds from the public domain to the secret domain). + + + +:::info +You can think of private and public functions as being executed by two actors that can only communicate to each other by mailbox. +::: + +So, with private functions being able to call public functions (unilaterally) we had a way to go from private to public, what about the other way? Recall that public functions CANNOT call private functions directly. Instead, you can use the append-only merkle tree to save messages from a public function call, that can later be executed by a private function. Note, only a transaction coming after the one including the message from a public function can consume it. In practice this means that unless you are the sequencer it will not be within the same rollup. + +Given that private functions have the capability of calling public functions unilaterally, it is feasible to transition from a private to public function within the same transaction. However, the converse is not possible. To achieve this, the append-only merkle tree can be employed to save messages from a public function call, which can then be executed by a private function at a later point in time. It is crucial to reiterate that this can only occur at a later stage and cannot take place within the same rollup because the proof cannot be generated by the user. + +:::info +Theoretically the builder has all the state trees after the public function has inserted a message in the public tree, and is able to create a proof consuming those messages in the same block. But it requires pending UTXO's on a block-level. +::: + +From the above, we should have a decent idea about what private and public functions can do inside the L2, and how they might interact. + +## A note on L2 access control + +Many applications rely on some form of access control to function well. USDC have a blacklist, where only parties not on the list should be able to transfer. And other systems such as Aave have limits such that only the pool contract is able to mint debt tokens and transfers held funds. + +Access control like this cannot easily be enforced in the private domain, as reading is also nullifying(to ensure data is up to date). However, as it is possible to read historic public state, one can combine private and public functions to get the desired effect. + +Say the public state holds a `mapping(address user => bool blacklisted)` and a value with the block number of the last update `last_updated`. The private functions can then use this public blacklist IF it also performs a public function call that reverts if the block number of the historic state is older than the `last_updated`. This means that updating the blacklist would make pending transactions fail, but allow a public blacklist to be used. Similar would work for the Aave example, where it is just a public value with the allowed caller contracts. Example of how this would be written is seen below. Note that because the `onlyFresh` is done in public, the user might not know when he is generating his proof whether it will be valid or not. + +```solidity +function transfer( + secret address to, + secret uint256 amount, + secret HistoricState state + ) secret returns(bool) { + if (blacklisted[msg.sender] || blacklisted[to]) revert("Blacklisted"); + onlyFresh(state.blockNumber); + // logic +} + +function onlyFresh(pub uint256 blockNumber) public { + if (blockNumber < last_updated) revert("Stale state"); +} +``` + +:::info +This is not a perfect solution, as any functions using access control might end up doing a lot of public calls it could put a significant burden on sequencers and greatly increase the cost of the transaction for the user. We are investigating ways to improve. +::: diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md new file mode 100644 index 00000000000..ad1de113f41 --- /dev/null +++ b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md @@ -0,0 +1,95 @@ +--- +title: Privately access Public State +--- + +n Aztec, private and public execution environments are completely separate and operate with distinct state management. In the previous section we learned that functions can call private functions, and public functions can save messages to a merkle tree to then be read by a private function. + +It is also possible to access persistent data across both private and public state using a **slow updates tree**, which we will talk about on this page. Please note that we are still experimenting with this feature and it is not something that has ever been achieved before. + +On this page you will learn: + +1. Why a slow updates tree exists & use cases +2. How it works +3. How it can be used to access data in public & private domains +4. Limitations + +## The need for a slow updates tree + +This structure was created specifically to privately & publicly access data that is not updated often. It should be used to store data that is non-sensitive yet crucial for private smart contracts to access. Using a slow updates tree, it is possible to: + +- Update public data from public and private functions +- Access public data from a public function +- Access public data from a private function _without revealing which contract we're executing_ + +This data structure is ideal for these use cases: + +- Address Registry: Enabling contracts to interact with other contracts more easily requires address storage accessible in both public and private executions. This can be particularly helpful in things such as proxy contracts. +- Access Control: Managing privileges in contracts, such as a token contract owner’s ability to mint new tokens, is streamlined when control information is shared between public and private executions. This might include things like blacklists and whitelists. + +## How it works + +We developed the Slow Update Tree in response to help balance public and private execution in a blockchain context. + +Earlier systems typically used either fully public or entirely private state trees, which led to privacy leaks if reading public data from private state. + +Using a shared state tree, it is possible to privately access public data by providing a membership proof to show that a value is indeed part of a commitment, and then check that the commitment matches the one stored in the state, as shown below. + +```mermaid +graph TD; + Cm[Commitment1] --> Vm1[Value1] + Cm --> Vm2[Value2] + Cm --> Vmn[Value3] + Vm1 --> Mp1[Membership Proof for Value1] + Mp1 --> St[Commitment in State] + St --> Cm +``` + +However this means that any changes to the commitment will invalidate our membership proof, and make our read fail. + +To solve this, the Slow Update Tree is a dual-tree structure - using a current and pending tree - that updates at the end of an epoch. This way, we can ensure that the values are stable throughout the epoch, and that the membership proofs are not invalidated by changes in other contracts more than once every epoch. + +```mermaid +graph TD; + Change{Epoch Over} -->|True| Current{Current} + Change -->|False| Pending{Pending} + Current --> Current1[Current Commitment 1] + Current --> CurrentM[Current Commitment 2] + CurrentM --> Value1[Current Value 1] + CurrentM --> Value2[Current Value 2] + CurrentM --> ValueN[Current Value n] + Pending --> PendingM[Pending Commitment 1] + PendingM --> PValue1[Pending Value 1] + PendingM --> PValue2[Pending Value 2] + PendingM --> PValueN[Pending Value n] +``` + +### Dual Tree Structure + +The Slow Update Tree operates on a dual-tree mechanism - a _current_ tree and a _pending_ tree. These trees manage the state data that needs to be shared between public and private executions. + +This _slow update_ mechanism minimizes the invalidation of reads and writes due to concurrent updates. + +### Epoch-Based Updates + +The current tree is replaced with the pending tree at the end of each epoch. Then a new pending tree is created, the epoch restarts, and the cycle begins again. + +### Accessing Data + +*From public state* Directly from the state +*From private state:* Performs a membership proof for the values in the tree, ensuring that they are part of the commitment. + +### Updating Values + +Updates are made to the pending tree. Then at the end of each epoch, the updates in the pending tree are committed and it becomes the current tree. + +## Limitations + +### Delayed State Finality + +Updates in the Slow Update Tree are only finalized at the end of an epoch. + +This could potentially lead to delays in state changes - for example an address may be added to a blacklist but this will not be reflected in the state until the end of the current epoch. + +### Complexity in State Management + +Developers are used to instant state updates, so the Slow Update Tree might take some getting used to. This should not be too much of an issue. \ No newline at end of file diff --git a/docs/sidebars.js b/docs/sidebars.js index 9616058d7d6..8852aa8925d 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -105,7 +105,17 @@ const sidebars = { id: "concepts/foundation/communication/main", }, items: [ - "concepts/foundation/communication/public_private_calls", + { + label: "Public <> Private Communication", + type: "category", + link: { + type: "doc", + id: "concepts/foundation/communication/public_private_calls/main", + }, + items: [ + "concepts/foundation/communication/public_private_calls/slow_updates_tree" + ] + }, "concepts/foundation/communication/cross_chain_calls", ], }, From 412b29841fe001441304604473fc15701ed86f0f Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Fri, 24 Nov 2023 21:50:16 +0600 Subject: [PATCH 02/16] sidebar --- .../communication/public_private_calls.md | 107 ------------------ .../contracts/syntax/slow_update_tree.md | 8 ++ docs/sidebars.js | 1 + 3 files changed, 9 insertions(+), 107 deletions(-) delete mode 100644 docs/docs/concepts/foundation/communication/public_private_calls.md create mode 100644 docs/docs/dev_docs/contracts/syntax/slow_update_tree.md diff --git a/docs/docs/concepts/foundation/communication/public_private_calls.md b/docs/docs/concepts/foundation/communication/public_private_calls.md deleted file mode 100644 index 683126a14a3..00000000000 --- a/docs/docs/concepts/foundation/communication/public_private_calls.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -title: Private <--> Public execution ---- - -import Image from "@theme/IdealImage"; - -import Disclaimer from "../../../misc/common/\_disclaimer.mdx"; - - - -Aztec operates on a model of private and public functions that are able to work together. Private functions work by providing evidence of correct execution generated locally through kernel proofs. Public functions, on the other hand, are able to utilize the latest state to manage updates and perform alterations. - -On this page, you’ll learn: - -- How private and public functions work -- The role of public functions in managing state alterations and updates -- Communication and interactions between private and public functions -- How the sequencer manages the order of operations of private functions - -### Objectives - -The goal for L2 communication is to setup the most simple mechanism that will support - -- _private_ and _public_ functions -- _private_ functions that can call _private_ or _public_ functions -- _public_ functions that can call _private_ or _public_ functions - -Before diving into the communication abstracts for Aztec, we need to understand some of our limitations. One being that public functions (as known from Ethereum) must operate on the current state to provide meaningful utility, e.g., at the tip. -This works fine when there is only one builder (sequencer) executing it first, and then others verifying as the builder always knows the tip. On the left in the diagram below, we see a block where the transactions are applied one after another each building on the state before it. For example, if Tx 1 update storage `a = 5`, then in Tx 2 reading `a` will return `5`. - -This works perfectly well when everything is public and a single builder is aware of all changes. However, in a private setting, we require the user to present evidence of correct execution as part of their transaction in the form of a kernel proof (generated locally on user device ahead of time). This way, the builder doesn't need to have knowledge of everything happening in the transaction, only the results. If we were to build this proof on the latest state, we would encounter problems. How can two different users build proofs at the same time, given that they will be executed one after the other by the sequencer? The simple answer is that they cannot, as race conditions would arise where one of the proofs would be invalidated by the other due to a change in the state root (which would nullify Merkle paths). - -To avoid this issue, we permit the use of historical data as long as the data has not been nullified previously. Note, that because this must include nullifiers that were inserted after the proof generation, but before execution we need to nullify (and insert the data again) to prove that it was not nullified. Without emitting the nullifier we would need our proof to point to the current head of the nullifier tree to have the same effect, e.g., back to the race conditions we were trying to avoid. - -In this model, instead of informing the builder of our intentions, we construct the proof $\pi$ and then provide them with the transaction results (new commitments and nullifiers, contract deployments and cross-chain messages) in addition to $\pi$. The builder will then be responsible for inserting these new commitments and nullifiers into the state. They will be aware of the intermediates and can discard transactions that try to produce existing nullifiers (double spend), as doing so would invalidate the rollup proof. - -On the left-hand side of the diagram below, we see the fully public world where storage is shared, while on the right-hand side, we see the private world where all reads are historic. - - - -Given that Aztec will comprise both private and public functions, it is imperative that we determine the optimal ordering for these functions. From a logical standpoint, it is reasonable to execute the private functions first as they are executed on a state $S_i$, where $i \le n$, with $S_n$ representing the current state where the public functions always operate on the current state $S_n$. Prioritizing the private functions would also afford us the added convenience of enabling them to invoke the public functions, which is particularly advantageous when implementing a peer-to-pool architecture such as that employed by Uniswap. - -Transactions that involve both private and public functions will follow a specific order of execution, wherein the private functions will be executed first, followed by the public functions, and then moving on to the next transaction. - -It is important to note that the execution of private functions is prioritized before executing any public functions. This means that private functions cannot "wait" on the results of any of their calls to public functions. Stated differently, any calls made across domains are unilateral in nature, akin to shouting into the void with the hope that something will occur at a later time. The figure below illustrates the order of function calls on the left-hand side, while the right-hand side shows how the functions will be executed. Notably, the second private function call is independent of the output of the public function and merely occurs after its execution. - - - -Multiple of these transactions are then ordered into a L2 block by the sequencer, who will also be executing the public functions (as they require the current head). Example seen below. - - - -:::info -Be mindful that if part of a transaction is reverting, say the public part of a call, it will revert the entire transaction. Similarly to Ethereum, it might be possible for the block builder to create a block such that your valid transaction reverts because of altered state, e.g., trade incurring too much slippage or the like. -::: - -To summarize: - -- _Private_ function calls are fully "prepared" and proven by the user, which provides the kernel proof along with new commitments and nullifiers to the sequencer. -- _Public_ functions altering public state (updatable storage) must be executed at the current "head" of the chain, which only the sequencer can ensure, so these must be executed separately to the _private_ functions. -- _Private_ and _public_ functions within an Aztec transaction are therefore ordered such that first _private_ functions are executed, and then _public_. - -A more comprehensive overview of the interplay between private and public functions and their ability to manipulate data is presented below. It is worth noting that all data reads performed by private functions are historical in nature, and that private functions are not capable of modifying public storage. Conversely, public functions have the capacity to manipulate private storage (e.g., inserting new commitments, potentially as part of transferring funds from the public domain to the secret domain). - - - -:::info -You can think of private and public functions as being executed by two actors that can only communicate to each other by mailbox. -::: - -So, with private functions being able to call public functions (unilaterally) we had a way to go from private to public, what about the other way? Recall that public functions CANNOT call private functions directly. Instead, you can use the append-only merkle tree to save messages from a public function call, that can later be executed by a private function. Note, only a transaction coming after the one including the message from a public function can consume it. In practice this means that unless you are the sequencer it will not be within the same rollup. - -Given that private functions have the capability of calling public functions unilaterally, it is feasible to transition from a private to public function within the same transaction. However, the converse is not possible. To achieve this, the append-only merkle tree can be employed to save messages from a public function call, which can then be executed by a private function at a later point in time. It is crucial to reiterate that this can only occur at a later stage and cannot take place within the same rollup because the proof cannot be generated by the user. - -:::info -Theoretically the builder has all the state trees after the public function has inserted a message in the public tree, and is able to create a proof consuming those messages in the same block. But it requires pending UTXO's on a block-level. -::: - -From the above, we should have a decent idea about what private and public functions can do inside the L2, and how they might interact. - -## A note on L2 access control - -Many applications rely on some form of access control to function well. USDC have a blacklist, where only parties not on the list should be able to transfer. And other systems such as Aave have limits such that only the pool contract is able to mint debt tokens and transfers held funds. - -Access control like this cannot easily be enforced in the private domain, as reading is also nullifying(to ensure data is up to date). However, as it is possible to read historic public state, one can combine private and public functions to get the desired effect. - -Say the public state holds a `mapping(address user => bool blacklisted)` and a value with the block number of the last update `last_updated`. The private functions can then use this public blacklist IF it also performs a public function call that reverts if the block number of the historic state is older than the `last_updated`. This means that updating the blacklist would make pending transactions fail, but allow a public blacklist to be used. Similar would work for the Aave example, where it is just a public value with the allowed caller contracts. Example of how this would be written is seen below. Note that because the `onlyFresh` is done in public, the user might not know when he is generating his proof whether it will be valid or not. - -```solidity -function transfer( - secret address to, - secret uint256 amount, - secret HistoricState state - ) secret returns(bool) { - if (blacklisted[msg.sender] || blacklisted[to]) revert("Blacklisted"); - onlyFresh(state.blockNumber); - // logic -} - -function onlyFresh(pub uint256 blockNumber) public { - if (blockNumber < last_updated) revert("Stale state"); -} -``` - -:::info -This is not a perfect solution, as any functions using access control might end up doing a lot of public calls it could put a significant burden on sequencers and greatly increase the cost of the transaction for the user. We are investigating ways to improve. -::: diff --git a/docs/docs/dev_docs/contracts/syntax/slow_update_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_update_tree.md new file mode 100644 index 00000000000..30f9792c0f7 --- /dev/null +++ b/docs/docs/dev_docs/contracts/syntax/slow_update_tree.md @@ -0,0 +1,8 @@ +--- +title: Slow Updates Tree +--- + +Slow Updates Tree is a data structure that allows for public data to be accessed in both private and public domains. Read more about it in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). + +The Slow Updates Tree works by having a current tree and a pending tree, and replacing the current tree with the pending tree after an epoch has passed. Public functions can read directly from the current tree, and private functions can perform a membership proof that values are part of a commitment. + diff --git a/docs/sidebars.js b/docs/sidebars.js index 8852aa8925d..8dc715a893a 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -327,6 +327,7 @@ const sidebars = { }, "dev_docs/contracts/syntax/events", "dev_docs/contracts/syntax/functions", + "dev_docs/contracts/syntax/slow_update_tree", "dev_docs/contracts/syntax/context", "dev_docs/contracts/syntax/globals", ], From b7f91fbd23dbecb25386ade983226d7ce8d2ca4e Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Sat, 25 Nov 2023 00:05:43 +0600 Subject: [PATCH 03/16] how-to without include_code --- .../contracts/syntax/slow_updates_tree.md | 318 ++++++++++++++++++ 1 file changed, 318 insertions(+) create mode 100644 docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md diff --git a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md new file mode 100644 index 00000000000..2211ef2b52a --- /dev/null +++ b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md @@ -0,0 +1,318 @@ +--- +title: Slow Updates Tree +--- + +Slow Updates Tree is a data structure that allows for public data to be accessed in both private and public domains. Read more about it in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). + +The Slow Updates Tree works by having a current tree and a pending tree, and replacing the current tree with the pending tree after an epoch has passed. Public functions can read directly from the current tree, and private functions can perform a membership proof that values are part of a commitment. + +On this page you will learn: + +1. The SlowMap data type +2. A SlowTree.nr smart contract +3. The components involved in using the Slow Updates Tree +4. How you can integrate it into your own smart contract + + +## Exploring an example integration through a **`TokenBlacklist`** Smart Contract + +The `TokenBlacklist` contract is a token contract that does not allow blacklisted accounts to perform mints or transfers. To achieve this, it interacts with a Slow Updates Tree primarily through the `SlowMap` interface. There are four main components involved in this smart contract: + +1. **TokenBlacklist.nr Contract:** This is the primary smart contract that utilizes the Slow Updates Tree for managing blacklisted addresses +2. **SlowMap Interface**: This interface is used within the `TokenBlacklist` contract to interact with the Slow Updates Tree. It provides methods for reading and updating values in the tree in both public and private contexts. +3. **SlowTree.nr Contract**: This is the smart contract we talked about previously. +4. **SlowMap type**: This is a type in the Azetc library that is utilized by the SlowTree contract. + +Let’s see how the TokenBlacklist Contract utilizes the SlowMap interface to interact with the SlowTree contract we talked about previously. + +## TokenBlacklist Contract + +You can find the full code for the TokenBlacklist smart contract [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract). + +### The SlowMap Interface and Its Integration + +The contract first imports the **`SlowMap`** interface: + +```rust +use crate::interfaces::SlowMap; +``` + +The **`SlowMap`** interface allows the contract to interact with its attached SlowTree. It abstracts these functions so they do not have to be implemented in the TokenBlacklist contract. + +### Constructor and Initialization of the Slow Updates Tree + +The contract's constructor takes the address of the slow updates contract: + +```rust +fn constructor(admin: AztecAddress, slow_updates_contract: AztecAddress) { + let mut slow_note = FieldNote::new(slow_updates_contract.address); + storage.slow_update.initialize(&mut slow_note, Option::none(), false); +} + +``` + +This initialization sets up the connection between the **`TokenBlacklist`** contract and a previously deployed SlowTree, allowing it to use the SlowMap interface to directly interact with the SlowTree. + +### Private Transfer Function Utilizing the Slow Updates Tree + +In the private transfer function, the contract uses the **`SlowMap`** interface to check if a user is blacklisted: + +```rust + +let slow = SlowMap::at(AztecAddress::new(storage.slow_update.get_note().value)); +let from_roles = UserFlags::new(slow.read_at(&mut context, from.address) as u120); +assert(!from_roles.is_blacklisted, "Blacklisted: Sender"); +let to_roles = UserFlags::new(slow.read_at(&mut context, to.address) as u120); +assert(!to_roles.is_blacklisted, "Blacklisted: Recipient"); + +``` + +Here, the contract reads the roles of the sender and recipient from the SlowTree using the **`read_at`** function in the **`SlowMap`**interface. It checks if either party is blacklisted, and if so, the transaction does not go ahead. + +## The SlowMap interface + +This interface used by the TokenBlackList contract contains multiple functions for interacting with its associated SlowTree. You can find the interface [here](https://github.com/AztecProtocol/aztec-packages/blob/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/interfaces.nr). + +For example, the `read_at()` function in the interface calls the `read_at()` private function in the SlowTree contract. + +## A SlowTree.nr smart contract + +The TokenBlacklist contract interacts with this SlowTree contract through the SlowMap interface. Here you can find all the functions included in this SlowTree contract and how it works. + +**Global Constants and Storage Structure:** + +At the start of the contract, we define global constants like so: + +```rust +global TREE_HEIGHT: Field = 254; +global MEMBERSHIP_SIZE: Field = 256; // TREE_HEIGHT + 2 +global UPDATE_SIZE: Field = 512; // TREE_HEIGHT * 2 + 4 +global EMPTY_ROOT: Field = 5785871043333994658400733180052743689641713274194136017445890613179954325976; + +struct Storage { + trees: Map>, +} +``` + +- `TREE_HEIGHT`, `MEMBERSHIP_SIZE`, and `UPDATE_SIZE` are constants that define the dimensions of the tree and the proof sizes. +- `EMPTY_ROOT` represents the initial state of the tree root. +- The `Storage` struct contains a map of `SlowMap`s that we will talk about more in the next section. + +**Constructor and Initialization** + +```rust + +#[aztec(private)] +fn constructor() {} + +#[aztec(public)] +fn initialize() { + storage.trees.at(context.msg_sender()).initialize(EMPTY_ROOT); +} +``` + +`The constructor calls the initialize` function that initializes the Slow Update Tree for `msg_sender` with an empty root, by calling the `initialize` method in the `SlowMap` library. + +**Reading and Updating in Public Context** + +```rust + +#[aztec(public)] +fn read_at_pub(key: Field) -> Field { + storage.trees.at(context.msg_sender()).read_at(key) +} + +#[aztec(public)] +fn update_at_public(p: SlowUpdateProof) { + storage.trees.at(context.msg_sender()).update_at(p); +} + +``` + +- **`read_at_pub`** allows reading a value from the tree in a public context. It works by getting the value at the key that correlates to `msg_sender()` by caling `read_at()` from the `SlowMap` library. +- **`update_at_public`** enables the public update of the tree by calling `update_at()` from the `SlowMap`using a `SlowUpdateProof` This is what a SlowUpdateProof looks like: + +```rust +// The slow update proof. Containing two merkle paths +// One for the before and one for the after trees. +// M = 2 * N + 4 +struct SlowUpdateProof { + index: Field, + new_value: Field, + before: SlowUpdateInner, + after: SlowUpdateInner, +} +``` + +**Reading in Private Context** + +```rust + +#[aztec(private)] + fn read_at(index: Field) -> Field { + let fields = pop_capsule(); + let p: MembershipProof = deserialize_membership_proof(fields); + assert(index == p.index, "Index does not match expected"); + + let expected_root = compute_merkle_root(p.value, p.index, p.sibling_path); + let selector = compute_selector("_assert_current_root(Field,Field)"); + context.call_public_function(context.this_address(), + selector, + [context.msg_sender(), expected_root]); + + p.value + } +``` + +- **`read_at`** is a private function that retrieves a value at a given index. It utilizes a **`MembershipProof`** to ensure the legitimacy of the read operation. This is what a MembershipProof looks like: + +```rust +struct MembershipProof { + index: Field, + value: Field, + sibling_path: [Field; N], +} +``` + +**Updating from private context** + +```rust + +#[aztec(private)] +fn update_at_private(index: Field, new_value: Field) { + // ... + context.call_public_function(context.this_address(), + selector, + [ + context.msg_sender(), + p.index, + p.new_value, + before_root, + after_root, + new_after_root + ]); +} + +``` + +- **`update_at_private`** updates a value in the tree privately. It uses **`SlowUpdateProof`** to validate the update before committing it. + +**Ensuring Tree Consistency** + +```rust +#[aztec(public)] +internal fn _assert_current_root(caller: Field, expected: Field) { + // ... +} + +#[aztec(public)] +internal fn _update(caller: Field, index: Field, new_value: Field, before: Field, after: Field, new_root: Field) { + // ... +} + +``` + +- **`_assert_current_root`** and **`_update`** are internal functions ensuring the consistency and integrity of the tree, and are called when reading and updating the tree from private state respectively. They are used to verify that the state of the tree before and after an update is as expected. + +## The SlowMap data type + +Under the hood, Aztec provides a library for the implementation of a slow_updates_tree. Here we will talk about some of the functions that are utilized in the TokenBlacklist contract and associated SlowTree, but you can find the full library [here](https://github.com/AztecProtocol/aztec-nr/tree/master/slow-updates-tree). + +### **`read_at`** function + +This function is called when the token wants to check that the account has not been blacklisted. + +```rust + +pub fn read_at(self: Self, key: Field) -> Field { + let time = self.context.public.unwrap().timestamp() as u120; + let leaf = self.read_leaf_at(key); + if time <= leaf.next_change as u120 { + leaf.before + } else { + leaf.after + } +} + +``` + +This function reads the current value of a specific key in the tree. It does this through: + +1. **Getting the current timestamp**: It first retrieves the current timestamp from the `context`. It then checks whether the current time is before or after the **`next_change`** timestamp of the leaf. +2. **Reading the leaf**: The function calls `read_leaf_at` to get the leaf object associated with the given key. The leaf object contains `before`, `after`, and `next_change` fields. +3. **Determining the current value**: Depending on whether the current time is before or after **`next_change`**, the function returns either the **`before`** or **`after`** value of the leaf. This mechanism allows for the slow update feature of the tree. + +### **`read_leaf_at`** function + +`read_at()` calls the `read_leaf_at()` function which reads a specific leaf from the tree using a key. + +```rust + +pub fn read_leaf_at(self: Self, key: Field) -> Leaf { + let derived_storage_slot = pedersen_hash([self.storage_slot, key]); + storage_read(derived_storage_slot, deserialize_leaf) +} + +``` + +1. **Deriving the storage slot**: It first derives a storage slot specific to the key using a Pedersen hash. This derivation ensures that each key has a unique and consistent location in storage. +2. Leaf Retrieval: It then reads the leaf data from the storage slot. The leaf data includes `before`, `after`, and `next_change` timestamps, which are used in the `read_at()` function. + +## How to integrate a slow updates tree + +You can utilize this example to implement a slow updates tree in your own smart contract. + +1. Copy the SlowTree.nr example, replace the constants with whatever you like, and deploy it to your sandbox +2. Copy the SlowMap interface for easy interaction with your deployed SlowTree +3. Import the SlowMap interface into your contract + +```rust +use crate::interfaces::SlowMap; +``` + +4. Take the slow_updates_tree address into the constructor (or hardcode it for test purposes) + +```rust +fn constructor(slow_updates_contract: AztecAddress) {} +``` + +5. Store a slow updates tree in both public and private storage + +```rust +slow_update: ImmutableSingleton::new(context, 7, FieldNoteMethods), + public_slow_update: PublicState::new( + context, + 8, + AztecAddressSerializationMethods, + ), +``` + +6. Store the SlowTree address in private storage as a FieldNote + +```rust +let mut slow_note = FieldNote::new(slow_updates_contract.address); + storage.slow_update.initialize(&mut slow_note, Option::none(), false); +``` + +7. Store the SlowTree address in public storage and initialize an instance of SlowMap using this address + +```rust +storage.public_slow_update.write(slow_updates_contract); + SlowMap::at(slow_updates_contract).initialize(context); +``` + +8. Now you can read from private functions: + +```rust +let slow = SlowMap::at(AztecAddress::new(storage.slow_update.get_note().value)); +``` + +9. Or from public functions: + +```rust +let slow = SlowMap::at(storage.public_slow_update.read()); +``` + +You can also update by using the `update_at_private()` function in the interface. + +Learn more by reading [how it works in more detail](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md) and checking out the [slow_updates_tree library](https://github.com/AztecProtocol/aztec-nr/blob/master/slow-updates-tree/src/slow_map.nr). \ No newline at end of file From a477e0a904d36b111cbd1e65ab51f1f011163543 Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Sat, 25 Nov 2023 00:06:18 +0600 Subject: [PATCH 04/16] delete file --- docs/docs/dev_docs/contracts/syntax/slow_update_tree.md | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 docs/docs/dev_docs/contracts/syntax/slow_update_tree.md diff --git a/docs/docs/dev_docs/contracts/syntax/slow_update_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_update_tree.md deleted file mode 100644 index 30f9792c0f7..00000000000 --- a/docs/docs/dev_docs/contracts/syntax/slow_update_tree.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: Slow Updates Tree ---- - -Slow Updates Tree is a data structure that allows for public data to be accessed in both private and public domains. Read more about it in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). - -The Slow Updates Tree works by having a current tree and a pending tree, and replacing the current tree with the pending tree after an epoch has passed. Public functions can read directly from the current tree, and private functions can perform a membership proof that values are part of a commitment. - From 777dc5428056ffea442a2a12d860d61b33f6ffc7 Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Sat, 25 Nov 2023 02:36:54 +0600 Subject: [PATCH 05/16] sidebar --- .../foundation/communication/public_private_calls/main.md | 2 +- docs/sidebars.js | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/main.md b/docs/docs/concepts/foundation/communication/public_private_calls/main.md index 683126a14a3..2feada08278 100644 --- a/docs/docs/concepts/foundation/communication/public_private_calls/main.md +++ b/docs/docs/concepts/foundation/communication/public_private_calls/main.md @@ -4,7 +4,7 @@ title: Private <--> Public execution import Image from "@theme/IdealImage"; -import Disclaimer from "../../../misc/common/\_disclaimer.mdx"; +import Disclaimer from "../../../../misc/common/\_disclaimer.mdx"; diff --git a/docs/sidebars.js b/docs/sidebars.js index 8dc715a893a..b4e46f53c63 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -327,7 +327,7 @@ const sidebars = { }, "dev_docs/contracts/syntax/events", "dev_docs/contracts/syntax/functions", - "dev_docs/contracts/syntax/slow_update_tree", + "dev_docs/contracts/syntax/slow_updates_tree", "dev_docs/contracts/syntax/context", "dev_docs/contracts/syntax/globals", ], From e1fef5a3520cc820eabfb9223ba1dc5e64a61c44 Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Sat, 25 Nov 2023 14:48:00 +0600 Subject: [PATCH 06/16] docs build errors --- .../concepts/foundation/communication/main.md | 2 +- docs/docs/concepts/foundation/main.md | 2 +- .../dev_docs/contracts/syntax/functions.md | 2 +- .../contracts/syntax/slow_updates_tree.md | 124 +++--------------- .../dev_docs/contracts/syntax/storage/main.md | 2 +- .../docs/dev_docs/debugging/sandbox-errors.md | 2 +- .../contracts/slow_tree_contract/src/main.nr | 24 ++-- .../token_blacklist_contract/src/main.nr | 8 +- 8 files changed, 46 insertions(+), 120 deletions(-) diff --git a/docs/docs/concepts/foundation/communication/main.md b/docs/docs/concepts/foundation/communication/main.md index 3dcd7fb7f5b..1f85585dacc 100644 --- a/docs/docs/concepts/foundation/communication/main.md +++ b/docs/docs/concepts/foundation/communication/main.md @@ -4,6 +4,6 @@ title: Contract Communication This section will walk over communication types that behaves differently than normal function calls from. -Namely, if functions are in different domains, private vs. public, their execution behaves a little differently to what you might expect! See [Private <--> Public execution](./public_private_calls.md). +Namely, if functions are in different domains, private vs. public, their execution behaves a little differently to what you might expect! See [Private <--> Public execution](./public_private_calls/main.md). Likewise, executing a function on a different domain than its origin needs a bit extra thought. See [L1 <--> L2 communication](./cross_chain_calls.md). \ No newline at end of file diff --git a/docs/docs/concepts/foundation/main.md b/docs/docs/concepts/foundation/main.md index 0791cd279d3..5d7e9579882 100644 --- a/docs/docs/concepts/foundation/main.md +++ b/docs/docs/concepts/foundation/main.md @@ -24,7 +24,7 @@ A user of the Aztec network will interact with the network through Aztec.js. Azt ### Private Execution Environment -The PXE provides a secure environment for the execution of sensitive operations, ensuring private information and decrypted data are not accessible to unauthorized applications. It hides the details of the [state model](./state_model/main.md) from end users, but the state model is important for Aztec developers to understand as it has implications for [private/public execution](./communication/public_private_calls.md) and [L1/L2 communication](./communication/cross_chain_calls.md). The PXE also includes the [ACIR Simulator](../advanced/acir_simulator.md) for private executions and the KeyStore for secure key management. +The PXE provides a secure environment for the execution of sensitive operations, ensuring private information and decrypted data are not accessible to unauthorized applications. It hides the details of the [state model](./state_model/main.md) from end users, but the state model is important for Aztec developers to understand as it has implications for [private/public execution](./communication/public_private_calls/main.md) and [L1/L2 communication](./communication/cross_chain_calls.md). The PXE also includes the [ACIR Simulator](../advanced/acir_simulator.md) for private executions and the KeyStore for secure key management. Procedurally, the PXE sends results of private function execution and requests for public function executions to the [sequencer](./nodes_clients/sequencer.md), which will update the state of the rollup. diff --git a/docs/docs/dev_docs/contracts/syntax/functions.md b/docs/docs/dev_docs/contracts/syntax/functions.md index 996e369dff1..fad4a510d60 100644 --- a/docs/docs/dev_docs/contracts/syntax/functions.md +++ b/docs/docs/dev_docs/contracts/syntax/functions.md @@ -20,7 +20,7 @@ In Aztec there are multiple different types of visibility that can be applied to ### Data Visibility -Data visibility is used to describe whether the data (or state) used in a function is generally accessible (public) or on a need to know basis (private). Functions with public data visibility are executed by the sequencer, and functions with private data visibility are executed by the user. For more information on why this is the case, see [communication](../../../concepts/foundation/communication/public_private_calls.md). +Data visibility is used to describe whether the data (or state) used in a function is generally accessible (public) or on a need to know basis (private). Functions with public data visibility are executed by the sequencer, and functions with private data visibility are executed by the user. For more information on why this is the case, see [communication](../../../concepts/foundation/communication/public_private_calls/main.md). In the following sections, we are going to see how these two "types" co-exists and interact. diff --git a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md index 2211ef2b52a..6dd0f15be5f 100644 --- a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md +++ b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md @@ -33,9 +33,7 @@ You can find the full code for the TokenBlacklist smart contract [here](https:// The contract first imports the **`SlowMap`** interface: -```rust -use crate::interfaces::SlowMap; -``` +#include_code interface yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr The **`SlowMap`** interface allows the contract to interact with its attached SlowTree. It abstracts these functions so they do not have to be implemented in the TokenBlacklist contract. @@ -43,13 +41,7 @@ The **`SlowMap`** interface allows the contract to interact with its attached Sl The contract's constructor takes the address of the slow updates contract: -```rust -fn constructor(admin: AztecAddress, slow_updates_contract: AztecAddress) { - let mut slow_note = FieldNote::new(slow_updates_contract.address); - storage.slow_update.initialize(&mut slow_note, Option::none(), false); -} - -``` +#include_code constructor yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr This initialization sets up the connection between the **`TokenBlacklist`** contract and a previously deployed SlowTree, allowing it to use the SlowMap interface to directly interact with the SlowTree. @@ -57,15 +49,7 @@ This initialization sets up the connection between the **`TokenBlacklist`** cont In the private transfer function, the contract uses the **`SlowMap`** interface to check if a user is blacklisted: -```rust - -let slow = SlowMap::at(AztecAddress::new(storage.slow_update.get_note().value)); -let from_roles = UserFlags::new(slow.read_at(&mut context, from.address) as u120); -assert(!from_roles.is_blacklisted, "Blacklisted: Sender"); -let to_roles = UserFlags::new(slow.read_at(&mut context, to.address) as u120); -assert(!to_roles.is_blacklisted, "Blacklisted: Recipient"); - -``` +#include_code transfer_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr Here, the contract reads the roles of the sender and recipient from the SlowTree using the **`read_at`** function in the **`SlowMap`**interface. It checks if either party is blacklisted, and if so, the transaction does not go ahead. @@ -81,53 +65,25 @@ The TokenBlacklist contract interacts with this SlowTree contract through the Sl **Global Constants and Storage Structure:** -At the start of the contract, we define global constants like so: - -```rust -global TREE_HEIGHT: Field = 254; -global MEMBERSHIP_SIZE: Field = 256; // TREE_HEIGHT + 2 -global UPDATE_SIZE: Field = 512; // TREE_HEIGHT * 2 + 4 -global EMPTY_ROOT: Field = 5785871043333994658400733180052743689641713274194136017445890613179954325976; +At the start of the contract, we define global constants and storage struct: -struct Storage { - trees: Map>, -} -``` +#include_code constants_and_storage yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr - `TREE_HEIGHT`, `MEMBERSHIP_SIZE`, and `UPDATE_SIZE` are constants that define the dimensions of the tree and the proof sizes. - `EMPTY_ROOT` represents the initial state of the tree root. - The `Storage` struct contains a map of `SlowMap`s that we will talk about more in the next section. -**Constructor and Initialization** +**Initialization** -```rust +#include_code initialize yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr -#[aztec(private)] -fn constructor() {} - -#[aztec(public)] -fn initialize() { - storage.trees.at(context.msg_sender()).initialize(EMPTY_ROOT); -} -``` - -`The constructor calls the initialize` function that initializes the Slow Update Tree for `msg_sender` with an empty root, by calling the `initialize` method in the `SlowMap` library. +An empty Slow Tree is initialized for `msg_sender` by calling the `initialize()` method in the `SlowMap` library. **Reading and Updating in Public Context** -```rust +#include_code read_at_pub yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr -#[aztec(public)] -fn read_at_pub(key: Field) -> Field { - storage.trees.at(context.msg_sender()).read_at(key) -} - -#[aztec(public)] -fn update_at_public(p: SlowUpdateProof) { - storage.trees.at(context.msg_sender()).update_at(p); -} - -``` +#include_code update_at_pub yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr - **`read_at_pub`** allows reading a value from the tree in a public context. It works by getting the value at the key that correlates to `msg_sender()` by caling `read_at()` from the `SlowMap` library. - **`update_at_public`** enables the public update of the tree by calling `update_at()` from the `SlowMap`using a `SlowUpdateProof` This is what a SlowUpdateProof looks like: @@ -146,25 +102,9 @@ struct SlowUpdateProof { **Reading in Private Context** -```rust - -#[aztec(private)] - fn read_at(index: Field) -> Field { - let fields = pop_capsule(); - let p: MembershipProof = deserialize_membership_proof(fields); - assert(index == p.index, "Index does not match expected"); - - let expected_root = compute_merkle_root(p.value, p.index, p.sibling_path); - let selector = compute_selector("_assert_current_root(Field,Field)"); - context.call_public_function(context.this_address(), - selector, - [context.msg_sender(), expected_root]); - - p.value - } -``` +#include_code read_at_private yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr -- **`read_at`** is a private function that retrieves a value at a given index. It utilizes a **`MembershipProof`** to ensure the legitimacy of the read operation. This is what a MembershipProof looks like: +- **`read_at`** is a private function that retrieves a value at a given index. It utilizes a **`MembershipProof`** to ensure indexes are the same. This is what a MembershipProof looks like: ```rust struct MembershipProof { @@ -176,43 +116,17 @@ struct MembershipProof { **Updating from private context** -```rust - -#[aztec(private)] -fn update_at_private(index: Field, new_value: Field) { - // ... - context.call_public_function(context.this_address(), - selector, - [ - context.msg_sender(), - p.index, - p.new_value, - before_root, - after_root, - new_after_root - ]); -} - -``` +#include_code update_at_private yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr -- **`update_at_private`** updates a value in the tree privately. It uses **`SlowUpdateProof`** to validate the update before committing it. +- **`update_at_private()`** updates a value in the tree privately. It uses **`SlowUpdateProof`** to validate the update before committing it. -**Ensuring Tree Consistency** +**Ensuring Tree is Valid** -```rust -#[aztec(public)] -internal fn _assert_current_root(caller: Field, expected: Field) { - // ... -} +#include_code assert_current_root yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr -#[aztec(public)] -internal fn _update(caller: Field, index: Field, new_value: Field, before: Field, after: Field, new_root: Field) { - // ... -} - -``` +#include_code _update yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr -- **`_assert_current_root`** and **`_update`** are internal functions ensuring the consistency and integrity of the tree, and are called when reading and updating the tree from private state respectively. They are used to verify that the state of the tree before and after an update is as expected. +`_assert_current_root` and `_update` are internal functions that ensure the tree is what is expected, and is called when reading & updating the tree from private state respectively. They are used to verify that the state of the tree before and after an update is as expected. ## The SlowMap data type @@ -315,4 +229,4 @@ let slow = SlowMap::at(storage.public_slow_update.read()); You can also update by using the `update_at_private()` function in the interface. -Learn more by reading [how it works in more detail](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md) and checking out the [slow_updates_tree library](https://github.com/AztecProtocol/aztec-nr/blob/master/slow-updates-tree/src/slow_map.nr). \ No newline at end of file +Learn more by checking out the [slow_updates_tree library](https://github.com/AztecProtocol/aztec-nr/blob/master/slow-updates-tree/src/slow_map.nr). \ No newline at end of file diff --git a/docs/docs/dev_docs/contracts/syntax/storage/main.md b/docs/docs/dev_docs/contracts/syntax/storage/main.md index 0a33ce92a6a..e05d7af6120 100644 --- a/docs/docs/dev_docs/contracts/syntax/storage/main.md +++ b/docs/docs/dev_docs/contracts/syntax/storage/main.md @@ -23,7 +23,7 @@ On this page, you’ll learn: Public state variables can be read by anyone, while private state variables can only be read by their owner (or people whom the owner has shared the decrypted data or note viewing key with). -Public state follows the Ethereum style account model, where each contract has its own key-value datastore. Private state follows a UTXO model, where note contents (pre-images) are only known by the sender and those able to decrypt them - see ([state model](../../../../concepts/foundation/state_model/main.md) and [private/public execution](../../../../concepts/foundation/communication/public_private_calls.md)) for more background. +Public state follows the Ethereum style account model, where each contract has its own key-value datastore. Private state follows a UTXO model, where note contents (pre-images) are only known by the sender and those able to decrypt them - see ([state model](../../../../concepts/foundation/state_model/main.md) and [private/public execution](../../../../concepts/foundation/communication/public_private_calls/main.md)) for more background. ## Storage struct diff --git a/docs/docs/dev_docs/debugging/sandbox-errors.md b/docs/docs/dev_docs/debugging/sandbox-errors.md index 3cff6905f27..8a9e45d5d86 100644 --- a/docs/docs/dev_docs/debugging/sandbox-errors.md +++ b/docs/docs/dev_docs/debugging/sandbox-errors.md @@ -86,7 +86,7 @@ Calling a private Aztec.nr function in a public kernel is not allowed. #### 3005 - PUBLIC_KERNEL\_\_NON_EMPTY_PRIVATE_CALL_STACK -Public functions are executed after all the private functions are (see [private-public execution](../../concepts/foundation/communication/public_private_calls.md)). As such, private call stack must be empty when executing in the public kernel. +Public functions are executed after all the private functions are (see [private-public execution](../../concepts/foundation/communication/public_private_calls/main.md)). As such, private call stack must be empty when executing in the public kernel. #### 3011 - PUBLIC_KERNEL\_\_CALCULATED_PRIVATE_CALL_HASH_AND_PROVIDED_PRIVATE_CALL_HASH_MISMATCH diff --git a/yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr b/yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr index 0af6f5565a2..f745e9acac0 100644 --- a/yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr +++ b/yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr @@ -31,6 +31,7 @@ contract SlowTree { use crate::capsule::pop_capsule; use crate::types::{MembershipProof, deserialize_membership_proof}; + // docs:start:constants_and_storage global TREE_HEIGHT: Field = 254; global MEMBERSHIP_SIZE: Field = 256; // TREE_HEIGHT + 2 global UPDATE_SIZE: Field = 512; // TREE_HEIGHT * 2 + 4 @@ -40,6 +41,7 @@ contract SlowTree { struct Storage { trees: Map>, } + // docs:end:constants_and_storage impl Storage { fn init(context: Context) -> pub Self { @@ -60,22 +62,23 @@ contract SlowTree { #[aztec(private)] fn constructor() {} - + // docs:start:initialize #[aztec(public)] fn initialize() { storage.trees.at(context.msg_sender()).initialize(EMPTY_ROOT); } - + // docs:end:initialize + // docs:start:read_at_pub #[aztec(public)] fn read_at_pub(key: Field) -> Field { storage.trees.at(context.msg_sender()).read_at(key) } - + // docs:end:read_at_pub #[aztec(public)] fn read_leaf_at_pub(key: Field) -> Leaf { storage.trees.at(context.msg_sender()).read_leaf_at(key) } - + // docs:start:read_at_private #[aztec(private)] fn read_at(index: Field) -> Field { let fields = pop_capsule(); @@ -90,18 +93,22 @@ contract SlowTree { p.value } - + // docs:end:read_at_private + // docs:start:assert_current_root #[aztec(public)] internal fn _assert_current_root(caller: Field, expected: Field) { let root = storage.trees.at(caller).current_root(); assert(root == expected, "Root does not match expected"); } + // docs:end:assert_current_root + // docs:start:update_at_pub #[aztec(public)] fn update_at_public(p: SlowUpdateProof) { storage.trees.at(context.msg_sender()).update_at(p); } - + // docs:end:update_at_pub + // docs:start:update_at_private #[aztec(private)] fn update_at_private(index: Field, new_value: Field) { let fields = pop_capsule(); @@ -126,7 +133,8 @@ contract SlowTree { new_after_root ]); } - + // docs:end:update_at_private + // docs:start:_update #[aztec(public)] internal fn _update(caller: Field, index: Field, new_value: Field, before: Field, after: Field, new_root: Field) { let current_root = storage.trees.at(caller).current_root(); @@ -137,7 +145,7 @@ contract SlowTree { storage.trees.at(caller).update_unsafe_at(index, new_value, new_root); } - + // docs:end:_update unconstrained fn un_read_leaf_at(address: Field, key: Field) -> Leaf { storage.trees.at(address).read_leaf_at(key) } diff --git a/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr b/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr index 8e51623b854..a0d806f3f9c 100644 --- a/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr +++ b/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr @@ -52,7 +52,9 @@ contract TokenBlacklist { safe_u120_serialization::{SafeU120SerializationMethods, SAFE_U120_SERIALIZED_LEN}, roles::UserFlags, }; + // docs:start:interface use crate::interfaces::SlowMap; + // docs:end:interface struct Storage { admin: PublicState, @@ -100,12 +102,12 @@ contract TokenBlacklist { } } } - + // docs:start:constructor #[aztec(private)] fn constructor(admin: AztecAddress, slow_updates_contract: AztecAddress) { let mut slow_note = FieldNote::new(slow_updates_contract.address); storage.slow_update.initialize(&mut slow_note, Option::none(), false); - + // docs:end:constructor let selector = compute_selector("_initialize((Field),(Field))"); context.call_public_function(context.this_address(), selector, @@ -291,6 +293,7 @@ contract TokenBlacklist { 1 } + // docs:start:transfer_private #[aztec(private)] fn transfer(from: AztecAddress, to: AztecAddress, amount: Field, nonce: Field) -> Field { let slow = SlowMap::at(AztecAddress::new(storage.slow_update.get_note().value)); @@ -298,6 +301,7 @@ contract TokenBlacklist { assert(!from_roles.is_blacklisted, "Blacklisted: Sender"); let to_roles = UserFlags::new(slow.read_at(&mut context, to.address) as u120); assert(!to_roles.is_blacklisted, "Blacklisted: Recipient"); + // docs:end:transfer_private if (from.address != context.msg_sender()) { assert_current_call_valid_authwit(&mut context, from); From 6dec84f99caa5b6884076e7bbdc648e42b6f904e Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Sun, 26 Nov 2023 23:03:41 +0600 Subject: [PATCH 07/16] include_code --- .../public_private_calls/slow_updates_tree.md | 2 +- .../contracts/syntax/slow_updates_tree.md | 112 ++++-------------- .../slow-updates-tree/src/slow_map.nr | 6 + .../contracts/slow_tree_contract/src/types.nr | 2 + .../token_blacklist_contract/src/main.nr | 9 ++ 5 files changed, 43 insertions(+), 88 deletions(-) diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md index ad1de113f41..bbba75ac5bc 100644 --- a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md +++ b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md @@ -15,7 +15,7 @@ On this page you will learn: ## The need for a slow updates tree -This structure was created specifically to privately & publicly access data that is not updated often. It should be used to store data that is non-sensitive yet crucial for private smart contracts to access. Using a slow updates tree, it is possible to: +This structure was created specifically to privately & publicly access data that is not updated often. It should be used to store data that is not sensitive but should be accessible from private smart contracts. Using a slow updates tree, it is possible to: - Update public data from public and private functions - Access public data from a public function diff --git a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md index 6dd0f15be5f..abe1a90a959 100644 --- a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md +++ b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md @@ -20,10 +20,10 @@ The `TokenBlacklist` contract is a token contract that does not allow blackliste 1. **TokenBlacklist.nr Contract:** This is the primary smart contract that utilizes the Slow Updates Tree for managing blacklisted addresses 2. **SlowMap Interface**: This interface is used within the `TokenBlacklist` contract to interact with the Slow Updates Tree. It provides methods for reading and updating values in the tree in both public and private contexts. -3. **SlowTree.nr Contract**: This is the smart contract we talked about previously. +3. **SlowTree.nr Contract**: This is a smart contract that instantiates a slow updates tree and allows us to access and manipulate its contents. 4. **SlowMap type**: This is a type in the Azetc library that is utilized by the SlowTree contract. -Let’s see how the TokenBlacklist Contract utilizes the SlowMap interface to interact with the SlowTree contract we talked about previously. +Let’s see how these components work together to allow private and public functions to read a tree of blacklisted accounts: ## TokenBlacklist Contract @@ -33,7 +33,7 @@ You can find the full code for the TokenBlacklist smart contract [here](https:// The contract first imports the **`SlowMap`** interface: -#include_code interface yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr +#include_code interface yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust The **`SlowMap`** interface allows the contract to interact with its attached SlowTree. It abstracts these functions so they do not have to be implemented in the TokenBlacklist contract. @@ -41,7 +41,7 @@ The **`SlowMap`** interface allows the contract to interact with its attached Sl The contract's constructor takes the address of the slow updates contract: -#include_code constructor yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr +#include_code constructor yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust This initialization sets up the connection between the **`TokenBlacklist`** contract and a previously deployed SlowTree, allowing it to use the SlowMap interface to directly interact with the SlowTree. @@ -49,7 +49,7 @@ This initialization sets up the connection between the **`TokenBlacklist`** cont In the private transfer function, the contract uses the **`SlowMap`** interface to check if a user is blacklisted: -#include_code transfer_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr +#include_code transfer_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust Here, the contract reads the roles of the sender and recipient from the SlowTree using the **`read_at`** function in the **`SlowMap`**interface. It checks if either party is blacklisted, and if so, the transaction does not go ahead. @@ -61,13 +61,13 @@ For example, the `read_at()` function in the interface calls the `read_at()` pri ## A SlowTree.nr smart contract -The TokenBlacklist contract interacts with this SlowTree contract through the SlowMap interface. Here you can find all the functions included in this SlowTree contract and how it works. +The TokenBlacklist contract interacts with this SlowTree contract through the SlowMap interface. The SlowTree is a contract that utilizes the slow update tree library and instansiates and manipulates a tree. Here you can find some of the functions included in this SlowTree contract and how it works. **Global Constants and Storage Structure:** At the start of the contract, we define global constants and storage struct: -#include_code constants_and_storage yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr +#include_code constants_and_storage yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust - `TREE_HEIGHT`, `MEMBERSHIP_SIZE`, and `UPDATE_SIZE` are constants that define the dimensions of the tree and the proof sizes. - `EMPTY_ROOT` represents the initial state of the tree root. @@ -75,56 +75,40 @@ At the start of the contract, we define global constants and storage struct: **Initialization** -#include_code initialize yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr +#include_code initialize yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust An empty Slow Tree is initialized for `msg_sender` by calling the `initialize()` method in the `SlowMap` library. **Reading and Updating in Public Context** -#include_code read_at_pub yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr +#include_code read_at_pub yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust -#include_code update_at_pub yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr +#include_code update_at_pub yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust - **`read_at_pub`** allows reading a value from the tree in a public context. It works by getting the value at the key that correlates to `msg_sender()` by caling `read_at()` from the `SlowMap` library. - **`update_at_public`** enables the public update of the tree by calling `update_at()` from the `SlowMap`using a `SlowUpdateProof` This is what a SlowUpdateProof looks like: -```rust -// The slow update proof. Containing two merkle paths -// One for the before and one for the after trees. -// M = 2 * N + 4 -struct SlowUpdateProof { - index: Field, - new_value: Field, - before: SlowUpdateInner, - after: SlowUpdateInner, -} -``` +#include_code slow_update_proof yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr rust **Reading in Private Context** -#include_code read_at_private yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr +#include_code read_at_private yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust - **`read_at`** is a private function that retrieves a value at a given index. It utilizes a **`MembershipProof`** to ensure indexes are the same. This is what a MembershipProof looks like: -```rust -struct MembershipProof { - index: Field, - value: Field, - sibling_path: [Field; N], -} -``` +#include_code membership_proof yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/types.nr rust **Updating from private context** -#include_code update_at_private yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr +#include_code update_at_private yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust - **`update_at_private()`** updates a value in the tree privately. It uses **`SlowUpdateProof`** to validate the update before committing it. **Ensuring Tree is Valid** -#include_code assert_current_root yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr +#include_code assert_current_root yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust -#include_code _update yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr +#include_code _update yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust `_assert_current_root` and `_update` are internal functions that ensure the tree is what is expected, and is called when reading & updating the tree from private state respectively. They are used to verify that the state of the tree before and after an update is as expected. @@ -136,19 +120,7 @@ Under the hood, Aztec provides a library for the implementation of a slow_update This function is called when the token wants to check that the account has not been blacklisted. -```rust - -pub fn read_at(self: Self, key: Field) -> Field { - let time = self.context.public.unwrap().timestamp() as u120; - let leaf = self.read_leaf_at(key); - if time <= leaf.next_change as u120 { - leaf.before - } else { - leaf.after - } -} - -``` +#include_code read_at yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr rust This function reads the current value of a specific key in the tree. It does this through: @@ -160,14 +132,7 @@ This function reads the current value of a specific key in the tree. It does thi `read_at()` calls the `read_leaf_at()` function which reads a specific leaf from the tree using a key. -```rust - -pub fn read_leaf_at(self: Self, key: Field) -> Leaf { - let derived_storage_slot = pedersen_hash([self.storage_slot, key]); - storage_read(derived_storage_slot, deserialize_leaf) -} - -``` +#include_code read_leaf_at yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr rust 1. **Deriving the storage slot**: It first derives a storage slot specific to the key using a Pedersen hash. This derivation ensures that each key has a unique and consistent location in storage. 2. Leaf Retrieval: It then reads the leaf data from the storage slot. The leaf data includes `before`, `after`, and `next_change` timestamps, which are used in the `read_at()` function. @@ -180,53 +145,26 @@ You can utilize this example to implement a slow updates tree in your own smart 2. Copy the SlowMap interface for easy interaction with your deployed SlowTree 3. Import the SlowMap interface into your contract -```rust -use crate::interfaces::SlowMap; -``` - -4. Take the slow_updates_tree address into the constructor (or hardcode it for test purposes) - -```rust -fn constructor(slow_updates_contract: AztecAddress) {} -``` +#include_code interface yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust 5. Store a slow updates tree in both public and private storage -```rust -slow_update: ImmutableSingleton::new(context, 7, FieldNoteMethods), - public_slow_update: PublicState::new( - context, - 8, - AztecAddressSerializationMethods, - ), -``` +#include_code slow_updates_storage yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust 6. Store the SlowTree address in private storage as a FieldNote -```rust -let mut slow_note = FieldNote::new(slow_updates_contract.address); - storage.slow_update.initialize(&mut slow_note, Option::none(), false); -``` +#include_code constructor yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust 7. Store the SlowTree address in public storage and initialize an instance of SlowMap using this address -```rust -storage.public_slow_update.write(slow_updates_contract); - SlowMap::at(slow_updates_contract).initialize(context); -``` +#include_code write_slow_update_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -8. Now you can read from private functions: +8. Now you can read and update from private functions: -```rust -let slow = SlowMap::at(AztecAddress::new(storage.slow_update.get_note().value)); -``` +#include_code get_and_update_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust 9. Or from public functions: -```rust -let slow = SlowMap::at(storage.public_slow_update.read()); -``` - -You can also update by using the `update_at_private()` function in the interface. +#include_code get_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust Learn more by checking out the [slow_updates_tree library](https://github.com/AztecProtocol/aztec-nr/blob/master/slow-updates-tree/src/slow_map.nr). \ No newline at end of file diff --git a/yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr b/yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr index f36d9f79593..d2dc5c67830 100644 --- a/yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr +++ b/yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr @@ -43,6 +43,7 @@ struct SlowUpdateInner { sibling_path: [Field; N], } +// docs:start:slow_update_proof // The slow update proof. Containing two merkle paths // One for the before and one for the after trees. // M = 2 * N + 4 @@ -52,6 +53,7 @@ struct SlowUpdateProof { before: SlowUpdateInner, after: SlowUpdateInner, } +// docs:end:slow_update_proof pub fn deserialize_slow_update_proof(serialized: [Field; M]) -> SlowUpdateProof { SlowUpdateProof::deserialize(serialized) @@ -142,11 +144,14 @@ impl SlowMap { } } + // docs:start:read_leaf_at pub fn read_leaf_at(self: Self, key: Field) -> Leaf { let derived_storage_slot = pedersen_hash([self.storage_slot, key]); storage_read(derived_storage_slot, deserialize_leaf) } + // docs:end:read_leaf_at + // docs:start:read_at // Reads the "CURRENT" value of the leaf pub fn read_at(self: Self, key: Field) -> Field { let time = self.context.public.unwrap().timestamp() as u120; @@ -157,6 +162,7 @@ impl SlowMap { leaf.after } } + // docs:end:read_at // Will update values in the "AFTER" tree // - updates the leaf and root to follow current values, moving from after to before if diff --git a/yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/types.nr b/yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/types.nr index 0f4f5eb4119..9245cb9aa13 100644 --- a/yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/types.nr +++ b/yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/types.nr @@ -1,3 +1,4 @@ +// docs:start:membership_proof // A single inclusion proof. // M = N + 2 struct MembershipProof { @@ -5,6 +6,7 @@ struct MembershipProof { value: Field, sibling_path: [Field; N], } +// docs:end:membership_proof fn deserialize_membership_proof(serialized: [Field; M]) -> MembershipProof { let mut sibling_path = [0; N]; diff --git a/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr b/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr index a0d806f3f9c..ad5203ea59c 100644 --- a/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr +++ b/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr @@ -92,6 +92,7 @@ contract TokenBlacklist { ) }, ), + // docs:start:slow_updates_storage // Below is an abomination to have same value in private and public (immutable in solidity). slow_update: ImmutableSingleton::new(context, 7, FieldNoteMethods), public_slow_update: PublicState::new( @@ -99,6 +100,8 @@ contract TokenBlacklist { 8, AztecAddressSerializationMethods, ), + // docs:end:slow_updates_storage + } } } @@ -123,8 +126,10 @@ contract TokenBlacklist { #[aztec(private)] fn init_slow_tree(user: AztecAddress) { let roles = UserFlags { is_admin: true, is_minter: false, is_blacklisted: false }.get_value() as Field; + // docs:start:get_and_update_private let slow = SlowMap::at(AztecAddress::new(storage.slow_update.get_note().value)); slow.update_at_private(&mut context, user.address, roles); + // docs:end:get_and_update_private context.call_public_function(context.this_address(), compute_selector("_init_slow_tree((Field))"), [context.msg_sender()]); @@ -148,7 +153,9 @@ contract TokenBlacklist { #[aztec(public)] fn mint_public(to: AztecAddress, amount: Field) -> Field { + // docs:start:get_public let slow = SlowMap::at(storage.public_slow_update.read()); + // docs:end:get_public let to_roles = UserFlags::new(slow.read_at_pub(context, to.address) as u120); assert(!to_roles.is_blacklisted, "Blacklisted: Recipient"); @@ -340,8 +347,10 @@ contract TokenBlacklist { internal fn _initialize(new_admin: AztecAddress, slow_updates_contract: AztecAddress) { assert(new_admin.address != 0, "invalid admin"); storage.admin.write(new_admin); + // docs:start:write_slow_update_public storage.public_slow_update.write(slow_updates_contract); SlowMap::at(slow_updates_contract).initialize(context); + // docs:end:write_slow_update_public } /// Internal /// From fd14bf290cebec2142933c4f231ee3eaf97ce2db Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Sun, 26 Nov 2023 17:14:11 +0000 Subject: [PATCH 08/16] fix typo --- .../communication/public_private_calls/slow_updates_tree.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md index bbba75ac5bc..5aadd175d73 100644 --- a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md +++ b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md @@ -2,7 +2,7 @@ title: Privately access Public State --- -n Aztec, private and public execution environments are completely separate and operate with distinct state management. In the previous section we learned that functions can call private functions, and public functions can save messages to a merkle tree to then be read by a private function. +In Aztec, private and public execution environments are completely separate and operate with distinct state management. In the previous section we learned that functions can call private functions, and public functions can save messages to a merkle tree to then be read by a private function. It is also possible to access persistent data across both private and public state using a **slow updates tree**, which we will talk about on this page. Please note that we are still experimenting with this feature and it is not something that has ever been achieved before. @@ -92,4 +92,4 @@ This could potentially lead to delays in state changes - for example an address ### Complexity in State Management -Developers are used to instant state updates, so the Slow Update Tree might take some getting used to. This should not be too much of an issue. \ No newline at end of file +Developers are used to instant state updates, so the Slow Update Tree might take some getting used to. This should not be too much of an issue. From 43d687c064d4690055ef7578c5bd7120f99cc445 Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Sun, 26 Nov 2023 23:28:06 +0600 Subject: [PATCH 09/16] fix docs build errors --- docs/docs/dev_docs/contracts/syntax/functions.md | 2 +- docs/docs/dev_docs/contracts/syntax/storage/main.md | 2 +- docs/docs/dev_docs/debugging/sandbox-errors.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/docs/dev_docs/contracts/syntax/functions.md b/docs/docs/dev_docs/contracts/syntax/functions.md index fad4a510d60..334aaa1401b 100644 --- a/docs/docs/dev_docs/contracts/syntax/functions.md +++ b/docs/docs/dev_docs/contracts/syntax/functions.md @@ -213,7 +213,7 @@ Calling a public function from another public function is quite similar to what ### Private -> Public -As discussed above, private function execution and calls take place on the user's device, while public function execution and calls take place on a sequencer, in two different places at two different times, it is natural to question how we can achieve composability between the two. The solution is asynchronicity. Further reading can be found in the foundational concepts [here](../../../concepts/foundation/communication/public_private_calls.md). +As discussed above, private function execution and calls take place on the user's device, while public function execution and calls take place on a sequencer, in two different places at two different times, it is natural to question how we can achieve composability between the two. The solution is asynchronicity. Further reading can be found in the foundational concepts [here](../../../concepts/foundation/communication/public_private_calls/main.md). Private function execution takes place on the users device, where it keeps track of any public function calls that have been made. Whenever private execution completes, and a kernel proof is produced, the transaction sent to the network will include all of the public calls that were dispatched. When the sequencer receives the messages, it will take over and execute the public parts of the transaction. diff --git a/docs/docs/dev_docs/contracts/syntax/storage/main.md b/docs/docs/dev_docs/contracts/syntax/storage/main.md index e05d7af6120..b647cc6ff50 100644 --- a/docs/docs/dev_docs/contracts/syntax/storage/main.md +++ b/docs/docs/dev_docs/contracts/syntax/storage/main.md @@ -172,7 +172,7 @@ We know its verbose, and are working on making it less so. #### Mapping example -Say we want to have a group of `minters` that are able to mint assets in our contract, and we want them in public storage, because [access control in private is quite cumbersome](../../../../concepts/foundation/communication/public_private_calls.md#a-note-on-l2-access-control). In the `Storage` struct we can add it as follows: +Say we want to have a group of `minters` that are able to mint assets in our contract, and we want them in public storage, because [access control in private is quite cumbersome](../../../../concepts/foundation/communication/public_private_calls/main.md#a-note-on-l2-access-control). In the `Storage` struct we can add it as follows: #include_code storage_minters /yarn-project/noir-contracts/src/contracts/token_contract/src/main.nr rust diff --git a/docs/docs/dev_docs/debugging/sandbox-errors.md b/docs/docs/dev_docs/debugging/sandbox-errors.md index 8a9e45d5d86..2469d704cab 100644 --- a/docs/docs/dev_docs/debugging/sandbox-errors.md +++ b/docs/docs/dev_docs/debugging/sandbox-errors.md @@ -191,7 +191,7 @@ Users may create a proof against a historic state in Aztec. The rollup circuits - "${treeName} tree next available leaf index mismatch" - validating a tree's root is not enough. It also checks that the `next_available_leaf_index` is as expected. This is the next index we can insert new values into. Note that for the public data tree, this test is skipped since as it is a sparse tree unlike the others. -- "Public call stack size exceeded" - In Aztec, the sequencer executes all enqueued public functions in a transaction (to prevent race conditions - see [private-public execution](../../concepts/foundation/communication/public_private_calls.md)). This error says there are too many public functions requested. +- "Public call stack size exceeded" - In Aztec, the sequencer executes all enqueued public functions in a transaction (to prevent race conditions - see [private-public execution](../../concepts/foundation/communication/public_private_calls/main.md)). This error says there are too many public functions requested. - "Array size exceeds target length" - happens if you add more items than allowed by the constants set due to our circuit limitations (eg sending too many L2 to L1 messages or creating a function that exceeds the call stack length or returns more values than what Aztec.nr functions allow) From f9d37e3753e390e52f1e7977b835eb7b1a109a0c Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Tue, 28 Nov 2023 08:04:40 +0000 Subject: [PATCH 10/16] Apply suggestions from code review Co-authored-by: Rahul Kothari Co-authored-by: josh crites --- .../public_private_calls/slow_updates_tree.md | 10 +++++----- .../dev_docs/contracts/syntax/slow_updates_tree.md | 6 +++--- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md index 5aadd175d73..744121b453e 100644 --- a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md +++ b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md @@ -2,15 +2,15 @@ title: Privately access Public State --- -In Aztec, private and public execution environments are completely separate and operate with distinct state management. In the previous section we learned that functions can call private functions, and public functions can save messages to a merkle tree to then be read by a private function. +In Aztec, private and public execution environments are completely separate and operate with distinct state management. It is not possible for private functions to reliably access the most recent public data public state (only sequencers can do that!) -It is also possible to access persistent data across both private and public state using a **slow updates tree**, which we will talk about on this page. Please note that we are still experimenting with this feature and it is not something that has ever been achieved before. +But, what about historical public data (or public data that changes infrequently)? Through a **slow updates tree**, you can have private functions access historical public state. Please note that we are still experimenting with this feature. On this page you will learn: 1. Why a slow updates tree exists & use cases 2. How it works -3. How it can be used to access data in public & private domains +3. How it can be used to access historical public data 4. Limitations ## The need for a slow updates tree @@ -75,8 +75,8 @@ The current tree is replaced with the pending tree at the end of each epoch. The ### Accessing Data -*From public state* Directly from the state -*From private state:* Performs a membership proof for the values in the tree, ensuring that they are part of the commitment. +**From public state:** Accessed directly from the state +**From private state:** Performs a membership proof for the values in the tree, ensuring that they are part of the commitment. ### Updating Values diff --git a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md index abe1a90a959..1276c72e085 100644 --- a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md +++ b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md @@ -2,14 +2,14 @@ title: Slow Updates Tree --- -Slow Updates Tree is a data structure that allows for public data to be accessed in both private and public domains. Read more about it in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). +Slow Updates Tree is a data structure that allows for public data to be accessed in both private and public domains. Read the high level overview in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). -The Slow Updates Tree works by having a current tree and a pending tree, and replacing the current tree with the pending tree after an epoch has passed. Public functions can read directly from the current tree, and private functions can perform a membership proof that values are part of a commitment. +The Slow Updates Tree works by having a current tree and a pending tree, and replacing the current tree with the pending tree after an epoch has passed. Public functions can read directly from the current tree, and private functions can perform a membership proof that values are part of a commitment to the current state of the tree. On this page you will learn: 1. The SlowMap data type -2. A SlowTree.nr smart contract +2. About the SlowTree.nr smart contract 3. The components involved in using the Slow Updates Tree 4. How you can integrate it into your own smart contract From 7e2560fe0c5ba4c5e1805458ee69e229016b38fc Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Tue, 28 Nov 2023 16:55:39 +0600 Subject: [PATCH 11/16] address changes, added reference --- .../public_private_calls/slow_updates_tree.md | 24 +-- .../contracts/syntax/slow_updates_tree.md | 183 +++++++++++------- .../token_blacklist_contract/src/main.nr | 17 +- 3 files changed, 142 insertions(+), 82 deletions(-) diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md index 744121b453e..e1eb6420848 100644 --- a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md +++ b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md @@ -1,10 +1,10 @@ --- -title: Privately access Public State +title: Privately access Historic Public Data --- -In Aztec, private and public execution environments are completely separate and operate with distinct state management. It is not possible for private functions to reliably access the most recent public data public state (only sequencers can do that!) +In Aztec, private and public execution environments are completely separate and operate with distinct state management. It is not possible for private functions to reliably access the most recent public data public state - only sequencers can do that. -But, what about historical public data (or public data that changes infrequently)? Through a **slow updates tree**, you can have private functions access historical public state. Please note that we are still experimenting with this feature. +But, what about historical public data (or public data that changes infrequently)? Through a **slow updates tree**, private functions can access historical public state. Please note that we are still experimenting with this feature. On this page you will learn: @@ -15,11 +15,11 @@ On this page you will learn: ## The need for a slow updates tree -This structure was created specifically to privately & publicly access data that is not updated often. It should be used to store data that is not sensitive but should be accessible from private smart contracts. Using a slow updates tree, it is possible to: +This structure was created specifically to privately & publicly access historical public data that is not updated often. It should be used to store data that is not sensitive but needs to be accessible from private smart contracts. Using a slow updates tree, it is possible to: -- Update public data from public and private functions -- Access public data from a public function -- Access public data from a private function _without revealing which contract we're executing_ +- Access historic public data from a private function +- Access historic public data from a public function +- Update public data (that does not need updated often) from public and private functions This data structure is ideal for these use cases: @@ -28,15 +28,15 @@ This data structure is ideal for these use cases: ## How it works -We developed the Slow Update Tree in response to help balance public and private execution in a blockchain context. +We developed the Slow Update Tree to help balance public and private execution in a blockchain context. Earlier systems typically used either fully public or entirely private state trees, which led to privacy leaks if reading public data from private state. -Using a shared state tree, it is possible to privately access public data by providing a membership proof to show that a value is indeed part of a commitment, and then check that the commitment matches the one stored in the state, as shown below. +Using a shared state tree, it is possible to privately access historic public data by providing a membership proof to show that a value is indeed part of a commitment, and then check that the commitment matches the one stored in the state, as shown below. ```mermaid graph TD; - Cm[Commitment1] --> Vm1[Value1] + Cm[CommitmentToKnowingAValue] --> Vm1[Value1] Cm --> Vm2[Value2] Cm --> Vmn[Value3] Vm1 --> Mp1[Membership Proof for Value1] @@ -92,4 +92,6 @@ This could potentially lead to delays in state changes - for example an address ### Complexity in State Management -Developers are used to instant state updates, so the Slow Update Tree might take some getting used to. This should not be too much of an issue. +Developers are used to instant state updates, so the Slow Update Tree might take some getting used to. But we believe this won't take long! + +For a code walkthrough of how a token blacklist contract can use a slow updates tree, read [this](../../../../dev_docs/contracts/syntax/slow_updates_tree.md). \ No newline at end of file diff --git a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md index 1276c72e085..a6eb7e44c73 100644 --- a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md +++ b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md @@ -8,13 +8,13 @@ The Slow Updates Tree works by having a current tree and a pending tree, and rep On this page you will learn: -1. The SlowMap data type -2. About the SlowTree.nr smart contract -3. The components involved in using the Slow Updates Tree -4. How you can integrate it into your own smart contract +1. About the SlowTree.nr smart contract +2. The components involved in using the Slow Updates Tree +3. How you can integrate it into your own smart contract +Scroll down to view [integration guide](./slow_updates_tree.md#how-to-integrate-a-slow-updates-tree) and [reference](./slow_updates_tree.md#reference). -## Exploring an example integration through a **`TokenBlacklist`** Smart Contract +# Exploring an example integration through a **`TokenBlacklist`** Smart Contract The `TokenBlacklist` contract is a token contract that does not allow blacklisted accounts to perform mints or transfers. To achieve this, it interacts with a Slow Updates Tree primarily through the `SlowMap` interface. There are four main components involved in this smart contract: @@ -23,13 +23,11 @@ The `TokenBlacklist` contract is a token contract that does not allow blackliste 3. **SlowTree.nr Contract**: This is a smart contract that instantiates a slow updates tree and allows us to access and manipulate its contents. 4. **SlowMap type**: This is a type in the Azetc library that is utilized by the SlowTree contract. -Let’s see how these components work together to allow private and public functions to read a tree of blacklisted accounts: - ## TokenBlacklist Contract You can find the full code for the TokenBlacklist smart contract [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract). -### The SlowMap Interface and Its Integration +### Importing SlowMap The contract first imports the **`SlowMap`** interface: @@ -53,118 +51,169 @@ In the private transfer function, the contract uses the **`SlowMap`** interface Here, the contract reads the roles of the sender and recipient from the SlowTree using the **`read_at`** function in the **`SlowMap`**interface. It checks if either party is blacklisted, and if so, the transaction does not go ahead. +## SlowTree.nr smart contract + +Under the hood, this TokenBlacklist contract uses the SlowMap interface to interact with a SlowTree contract. SlowTree.nr is a contract that utilizes the slow update tree library and instansiates and manipulates a tree. + +If you are using the SlowTree contract with the SlowMap interface, the exact implementation details of this smart contract are not required to know. You can read the code [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/slow_tree_contract). + ## The SlowMap interface This interface used by the TokenBlackList contract contains multiple functions for interacting with its associated SlowTree. You can find the interface [here](https://github.com/AztecProtocol/aztec-packages/blob/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/interfaces.nr). -For example, the `read_at()` function in the interface calls the `read_at()` private function in the SlowTree contract. +## The SlowMap type -## A SlowTree.nr smart contract +This is a library provided by Aztec for low-level interacts with a slow updates tree. You can see the code for the library [here](https://github.com/AztecProtocol/aztec-nr/blob/master/slow-updates-tree/src/slow_map.nr). -The TokenBlacklist contract interacts with this SlowTree contract through the SlowMap interface. The SlowTree is a contract that utilizes the slow update tree library and instansiates and manipulates a tree. Here you can find some of the functions included in this SlowTree contract and how it works. -**Global Constants and Storage Structure:** +# How to integrate a slow updates tree -At the start of the contract, we define global constants and storage struct: +You can use this example to implement a slow updates tree in your own smart contract. -#include_code constants_and_storage yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust +1. Copy the SlowTree.nr example and its dependencies, found [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/slow_tree_contract). Replace the constants with whatever you like and deploy it to your sandbox +2. Copy the SlowMap interface for easy interaction with your deployed SlowTree. Find it [here](https://github.com/AztecProtocol/aztec-packages/blob/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/interfaces.nr) +3. Import the SlowMap interface into your contract -- `TREE_HEIGHT`, `MEMBERSHIP_SIZE`, and `UPDATE_SIZE` are constants that define the dimensions of the tree and the proof sizes. -- `EMPTY_ROOT` represents the initial state of the tree root. -- The `Storage` struct contains a map of `SlowMap`s that we will talk about more in the next section. +#include_code interface yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -**Initialization** +5. Store a slow updates tree in both public and private storage -#include_code initialize yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust +#include_code slow_updates_storage yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -An empty Slow Tree is initialized for `msg_sender` by calling the `initialize()` method in the `SlowMap` library. +6. Store the SlowTree address in private storage as a FieldNote -**Reading and Updating in Public Context** +#include_code constructor yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -#include_code read_at_pub yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust +7. Store the SlowTree address in public storage and initialize an instance of SlowMap using this address -#include_code update_at_pub yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust +#include_code write_slow_update_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -- **`read_at_pub`** allows reading a value from the tree in a public context. It works by getting the value at the key that correlates to `msg_sender()` by caling `read_at()` from the `SlowMap` library. -- **`update_at_public`** enables the public update of the tree by calling `update_at()` from the `SlowMap`using a `SlowUpdateProof` This is what a SlowUpdateProof looks like: +8. Now you can read and update from private functions: -#include_code slow_update_proof yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr rust +#include_code get_and_update_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -**Reading in Private Context** +9. Or from public functions: -#include_code read_at_private yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust +#include_code get_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -- **`read_at`** is a private function that retrieves a value at a given index. It utilizes a **`MembershipProof`** to ensure indexes are the same. This is what a MembershipProof looks like: +# Reference -#include_code membership_proof yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/types.nr rust +## Struct `SlowMap` -**Updating from private context** +### Overview +The `SlowMap` struct is used to interact with a slow updates tree deployed via the SlowTree smart contract. -#include_code update_at_private yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust +### Fields -- **`update_at_private()`** updates a value in the tree privately. It uses **`SlowUpdateProof`** to validate the update before committing it. +| Name | Type | Description | +|---------|-----------|---------------------------------| +| address | `Field` | The address of the SlowTree contract | -**Ensuring Tree is Valid** +## Functions -#include_code assert_current_root yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust +### at -#include_code _update yarn-project/noir-contracts/src/contracts/slow_tree_contract/src/main.nr rust +Returns an instance of `SlowMap` at the specified address. -`_assert_current_root` and `_update` are internal functions that ensure the tree is what is expected, and is called when reading & updating the tree from private state respectively. They are used to verify that the state of the tree before and after an update is as expected. +**Parameters** -## The SlowMap data type +| Name | Type | Description | +|----------|----------------|----------------------------| +| `address`| `AztecAddress` | The address of the SlowTree | -Under the hood, Aztec provides a library for the implementation of a slow_updates_tree. Here we will talk about some of the functions that are utilized in the TokenBlacklist contract and associated SlowTree, but you can find the full library [here](https://github.com/AztecProtocol/aztec-nr/tree/master/slow-updates-tree). +**Return** -### **`read_at`** function +| Name | Type | Description | +|-------|-----------|------------------------------| +| - | `SlowMap` | The `SlowMap` instance | -This function is called when the token wants to check that the account has not been blacklisted. +**Example** -#include_code read_at yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr rust +#include_code slowmap_at yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -This function reads the current value of a specific key in the tree. It does this through: +### initialize -1. **Getting the current timestamp**: It first retrieves the current timestamp from the `context`. It then checks whether the current time is before or after the **`next_change`** timestamp of the leaf. -2. **Reading the leaf**: The function calls `read_leaf_at` to get the leaf object associated with the given key. The leaf object contains `before`, `after`, and `next_change` fields. -3. **Determining the current value**: Depending on whether the current time is before or after **`next_change`**, the function returns either the **`before`** or **`after`** value of the leaf. This mechanism allows for the slow update feature of the tree. +Initializes the `SlowMap`. -### **`read_leaf_at`** function +**Parameters** -`read_at()` calls the `read_leaf_at()` function which reads a specific leaf from the tree using a key. +| Name | Type | Description | +|-----------|-----------------|----------------------| +| `context` | `PublicContext` | The execution context | -#include_code read_leaf_at yarn-project/aztec-nr/slow-updates-tree/src/slow_map.nr rust +**Return** -1. **Deriving the storage slot**: It first derives a storage slot specific to the key using a Pedersen hash. This derivation ensures that each key has a unique and consistent location in storage. -2. Leaf Retrieval: It then reads the leaf data from the storage slot. The leaf data includes `before`, `after`, and `next_change` timestamps, which are used in the `read_at()` function. +| Name | Type | Description | +|------|------|-------------| +| - | - | - | -## How to integrate a slow updates tree +**Example** -You can utilize this example to implement a slow updates tree in your own smart contract. +#include_code slowmap_initialize yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -1. Copy the SlowTree.nr example, replace the constants with whatever you like, and deploy it to your sandbox -2. Copy the SlowMap interface for easy interaction with your deployed SlowTree -3. Import the SlowMap interface into your contract +### read_at_pub -#include_code interface yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust +Reads a value at a specified index from a public function. -5. Store a slow updates tree in both public and private storage +**Parameters** -#include_code slow_updates_storage yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust +| Name | Type | Description | +|-----------|-----------------|-----------------------| +| `context` | `PublicContext` | The execution context | +| `index` | `Field` | The index to read at | -6. Store the SlowTree address in private storage as a FieldNote +**Return** -#include_code constructor yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust +| Name | Type | Description | +|----------|--------|-----------------------| +| `result` | `Field`| The value at `index` | -7. Store the SlowTree address in public storage and initialize an instance of SlowMap using this address +**Example** -#include_code write_slow_update_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust +#include_code read_at_pub yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -8. Now you can read and update from private functions: +### read_at + +Reads a value at a specified index from a private function. + +**Parameters** + +| Name | Type | Description | +|-----------|--------------------|------------------------| +| `context` | `PrivateContext` | The execution context | +| `index` | `Field` | The index to read at | + +**Return** + +| Name | Type | Description | +|----------|--------|-----------------------| +| `result` | `Field`| The value at `index` | + +**Example** + +#include_code slowmap_read_at yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust + +### update_at_private + +Updates a value at a specified index from a private function. Does not return anything. + +**Parameters** + +| Name | Type | Description | +|-------------|--------------------|------------------------| +| `context` | `PrivateContext` | The execution context | +| `index` | `Field` | The index to update | +| `new_value` | `Field` | The new value | + +**Example** #include_code get_and_update_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -9. Or from public functions: +## Updating from public -#include_code get_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust +This is not a method in the interface as it can be done using regular Aztec.nr public storage update syntax. + +**Example** + +#include_code write_slow_update_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -Learn more by checking out the [slow_updates_tree library](https://github.com/AztecProtocol/aztec-nr/blob/master/slow-updates-tree/src/slow_map.nr). \ No newline at end of file diff --git a/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr b/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr index df3b843acc6..73b14c9cf43 100644 --- a/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr +++ b/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr @@ -92,8 +92,8 @@ contract TokenBlacklist { ) }, ), - // docs:start:slow_updates_storage // Below is an abomination to have same value in private and public (immutable in solidity). + // docs:start:slow_updates_storage slow_update: ImmutableSingleton::new(context, 7, FieldNoteMethods), public_slow_update: PublicState::new( context, @@ -147,13 +147,18 @@ contract TokenBlacklist { storage.admin.write(new_admin); // docs:start:write_slow_update_public storage.public_slow_update.write(slow_updates_contract); - SlowMap::at(slow_updates_contract).initialize(context); // docs:end:write_slow_update_public + // docs:start:slowmap_initialize + SlowMap::at(slow_updates_contract).initialize(context); + // docs:end:slowmap_initialize + } #[aztec(private)] fn update_roles(user: AztecAddress, roles: Field) { + // docs:start:slowmap_at let slow = SlowMap::at(AztecAddress::new(storage.slow_update.get_note().value)); + // docs:end:slowmap_at let caller_roles = UserFlags::new(slow.read_at(&mut context, context.msg_sender()) as u120); assert(caller_roles.is_admin, "caller is not admin"); @@ -161,11 +166,13 @@ contract TokenBlacklist { } #[aztec(public)] - // docs:start:get_public fn mint_public(to: AztecAddress, amount: Field) { + // docs:start:get_public let slow = SlowMap::at(storage.public_slow_update.read()); - // docs:end:get_public + // docs:end:get_public + // docs:start:read_at_pub let to_roles = UserFlags::new(slow.read_at_pub(context, to.address) as u120); + // docs:end:read_at_pub assert(!to_roles.is_blacklisted, "Blacklisted: Recipient"); let caller_roles = UserFlags::new(slow.read_at_pub(context, context.msg_sender()) as u120); @@ -261,7 +268,9 @@ contract TokenBlacklist { #[aztec(private)] fn redeem_shield(to: AztecAddress, amount: Field, secret: Field) { let slow = SlowMap::at(AztecAddress::new(storage.slow_update.get_note().value)); + // docs:start:slowmap_read_at let to_roles = UserFlags::new(slow.read_at(&mut context, to.address) as u120); + // docs:end:slowmap_read_at assert(!to_roles.is_blacklisted, "Blacklisted: Recipient"); let pending_shields = storage.pending_shields; From 46cbad0fd013fde8fcebed873e5c71dd8b6e853e Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Wed, 29 Nov 2023 15:57:43 +0600 Subject: [PATCH 12/16] more information on each component, restructured, diagram --- .../public_private_calls/slow_updates_tree.md | 55 +++--- .../contracts/syntax/slow_updates_tree.md | 159 ++++++++++++------ 2 files changed, 137 insertions(+), 77 deletions(-) diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md index e1eb6420848..32266cb1532 100644 --- a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md +++ b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md @@ -15,7 +15,7 @@ On this page you will learn: ## The need for a slow updates tree -This structure was created specifically to privately & publicly access historical public data that is not updated often. It should be used to store data that is not sensitive but needs to be accessible from private smart contracts. Using a slow updates tree, it is possible to: +This structure was created specifically to privately & publicly access historical public data. It should be used to store public data that doesn't change often. - Access historic public data from a private function - Access historic public data from a public function @@ -28,15 +28,15 @@ This data structure is ideal for these use cases: ## How it works -We developed the Slow Update Tree to help balance public and private execution in a blockchain context. +We developed the Slow Updates Tree to help balance public and private execution in a blockchain context. Earlier systems typically used either fully public or entirely private state trees. -Earlier systems typically used either fully public or entirely private state trees, which led to privacy leaks if reading public data from private state. +**First iteration: Shared State Tree** Using a shared state tree, it is possible to privately access historic public data by providing a membership proof to show that a value is indeed part of a commitment, and then check that the commitment matches the one stored in the state, as shown below. ```mermaid graph TD; - Cm[CommitmentToKnowingAValue] --> Vm1[Value1] + Cm[Commmitment1] --> Vm1[Value1] Cm --> Vm2[Value2] Cm --> Vmn[Value3] Vm1 --> Mp1[Membership Proof for Value1] @@ -44,9 +44,28 @@ graph TD; St --> Cm ``` -However this means that any changes to the commitment will invalidate our membership proof, and make our read fail. +Note: a *commitment* refers to a cryptographic assurance that a specific set of data is included in the tree. In the current design, we are using Merkle trees to commit to the values. -To solve this, the Slow Update Tree is a dual-tree structure - using a current and pending tree - that updates at the end of an epoch. This way, we can ensure that the values are stable throughout the epoch, and that the membership proofs are not invalidated by changes in other contracts more than once every epoch. +However, these would be contract-specific trees, so we will be leaking the contract address of the contract doing a lookup. + +**Second iteration: Multi-level Shared State Tree** + +This privacy issue can be solved by organizing data into layers, with each contract having its own commitment within a larger tree. Only the top-level commitment is revealed. + +```mermaid +graph TD; + C[Top Commitment] -->|Contains| C1[Commitment1] + C -->|Contains| Cm[Commitment2] + Cm -->|Contains| Vm1[Value1] + Cm -->|Contains| Vm2[Value2] + Cm -->|Contains| Vmn[Valuen] +``` + +However, this means that any canges in any contract's data will change the top-level commitment and invalidate the proofs for all other contracts. + +**Third iteration: Slow Updates Tree** + +To solve this, the Slow Updates Tree is a dual-tree structure - a current tree and a pending tree. Any updates are added to the pending tree, which then becomes the current tree at the end of an epoch. The pending tree is replicated from the current tree, and the cycle begins again. ```mermaid graph TD; @@ -63,22 +82,16 @@ graph TD; PendingM --> PValueN[Pending Value n] ``` -### Dual Tree Structure - -The Slow Update Tree operates on a dual-tree mechanism - a _current_ tree and a _pending_ tree. These trees manage the state data that needs to be shared between public and private executions. +This way, we can ensure that the values are stable throughout the epoch, and that the membership proofs are not invalidated by changes in other contracts more than once every epoch. -This _slow update_ mechanism minimizes the invalidation of reads and writes due to concurrent updates. - -### Epoch-Based Updates - -The current tree is replaced with the pending tree at the end of each epoch. Then a new pending tree is created, the epoch restarts, and the cycle begins again. +## Reads and Writes ### Accessing Data -**From public state:** Accessed directly from the state -**From private state:** Performs a membership proof for the values in the tree, ensuring that they are part of the commitment. +*From public state:* Accessed directly from the state +*From private state:* Performs a membership proof for the values in the tree, ensuring that they are part of the commitment. -### Updating Values +### Updating Data Updates are made to the pending tree. Then at the end of each epoch, the updates in the pending tree are committed and it becomes the current tree. @@ -86,12 +99,10 @@ Updates are made to the pending tree. Then at the end of each epoch, the updates ### Delayed State Finality -Updates in the Slow Update Tree are only finalized at the end of an epoch. - -This could potentially lead to delays in state changes - for example an address may be added to a blacklist but this will not be reflected in the state until the end of the current epoch. +Updates in the Slow Updates Tree are only finalized at the end of an epoch. -### Complexity in State Management +Developers are used to instant state updates, so the Slow Updates Tree might take some getting used to. But we believe this won't take long! -Developers are used to instant state updates, so the Slow Update Tree might take some getting used to. But we believe this won't take long! +## Dive into the code For a code walkthrough of how a token blacklist contract can use a slow updates tree, read [this](../../../../dev_docs/contracts/syntax/slow_updates_tree.md). \ No newline at end of file diff --git a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md index a6eb7e44c73..d68614f36db 100644 --- a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md +++ b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md @@ -2,99 +2,148 @@ title: Slow Updates Tree --- -Slow Updates Tree is a data structure that allows for public data to be accessed in both private and public domains. Read the high level overview in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). +Slow Updates Tree is a data structure that allows for historic public data to be accessed in both private and public domains. Read the high level overview in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). The Slow Updates Tree works by having a current tree and a pending tree, and replacing the current tree with the pending tree after an epoch has passed. Public functions can read directly from the current tree, and private functions can perform a membership proof that values are part of a commitment to the current state of the tree. On this page you will learn: -1. About the SlowTree.nr smart contract -2. The components involved in using the Slow Updates Tree -3. How you can integrate it into your own smart contract +1. [The components involved in using the Slow Updates Tree](slow_updates_tree.md#components-involved-in-implementing-a-slow-updates-tree) +2. [How you can integrate it into your own smart contract](slow_updates_tree.md#how-to-integrate-a-slow-updates-tree) +3. [An example of a token blacklisting contract that uses the slow updates tree](slow_updates_tree.md#exploring-an-example-integration-through-a-tokenblacklist-smart-contract) +4. [Interface Reference](slow_updates_tree.md#reference) + +# Components involved in implementing a Slow Updates Tree + +There are generally 4 main components involved to make it easier to use a Slow Updates Tree, with 3 already implemented by Aztec. This makes it easier to interact with a slow updates tree through a simple interface. These four components are: + +## Main Smart Contract + +This is the primary smart contract that will use the Slow Updates Tree. In the example we use a [token with blacklisting features](slow_updates_tree.md#exploring-an-example-integration-through-a-tokenblacklist-smart-contract). + +## Interface + +This interface can be used by your contract to interact with the Slow Updates Tree. It provides methods for reading and updating values in the tree in both public and private contexts. You can find it [here](https://github.com/AztecProtocol/aztec-packages/blob/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/interfaces.nr). + +## SlowTree.nr Contract + +This is a smart contract developed by Aztec that establishes and manages a slow updates tree structure. It allows developers to access and interact with the tree, such as reading and updating data. + +You can find it [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/slow_tree_contract). + +## SlowMap type + +This is a type in the Aztec.nr library that is utilized by the SlowTree contract. It defines the underlying data structure for the slow updates tree, and handles storing both the current and pending values for each data entry. + +You can find it [here](https://github.com/AztecProtocol/aztec-nr/blob/master/slow-updates-tree/src/slow_map.nr). + +The diagram below describes how these components work together. It does not contain all the functionality. + +```mermaid +graph TD + MSC[Main Smart Contract] --> INT[Interface] + STC --> SMT + + INT_RAP[read_at_pub] <--> STC_RAP[read_at_public] + INT_RA[read_at] <--> STC_RA[read_at] + INT_UAP[update_at_public] <--> STC_UAP[update_at_public] + INT_UA[update_at_private] <--> STC_UA[update_at_private] + + STC_RA <--> VMP[verify_membership_proof] + STC_UA <--> CR[compute_roots] + + subgraph INT[Interface] + INT_RAP + INT_UAP + INT_RA + INT_UA + end + + subgraph STC[SlowTree.nr] + STC_RAP + STC_UAP + STC_RA + STC_UA + end + + subgraph SMT[SlowMap Type] + Change{Epoch Over} -->|True| Current{Current} + Change -->|False| Pending{Pending} + Current --> Current1[Current Commitment 1] + Current --> CurrentM[Current Commitment M] + CurrentM --> Value1[Current Value 1] + CurrentM --> Value2[Current Value 2] + CurrentM --> ValueN[Current Value N] + Pending --> PendingM[Pending Commitment 1] + PendingM --> PValue1[Pending Value 1] + PendingM --> PValue2[Pending Value 2] + PendingM --> PValueN[Pending Value N] + end + + style INT fill:#fff,stroke:#333,stroke-width:1px + style STC fill:#fff,stroke:#333,stroke-width:1px + style SMT fill:#fff,stroke:#333,stroke-width:1px +``` -Scroll down to view [integration guide](./slow_updates_tree.md#how-to-integrate-a-slow-updates-tree) and [reference](./slow_updates_tree.md#reference). - -# Exploring an example integration through a **`TokenBlacklist`** Smart Contract - -The `TokenBlacklist` contract is a token contract that does not allow blacklisted accounts to perform mints or transfers. To achieve this, it interacts with a Slow Updates Tree primarily through the `SlowMap` interface. There are four main components involved in this smart contract: - -1. **TokenBlacklist.nr Contract:** This is the primary smart contract that utilizes the Slow Updates Tree for managing blacklisted addresses -2. **SlowMap Interface**: This interface is used within the `TokenBlacklist` contract to interact with the Slow Updates Tree. It provides methods for reading and updating values in the tree in both public and private contexts. -3. **SlowTree.nr Contract**: This is a smart contract that instantiates a slow updates tree and allows us to access and manipulate its contents. -4. **SlowMap type**: This is a type in the Azetc library that is utilized by the SlowTree contract. - -## TokenBlacklist Contract - -You can find the full code for the TokenBlacklist smart contract [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract). +# How to integrate a slow updates tree -### Importing SlowMap - -The contract first imports the **`SlowMap`** interface: +1. Copy the *SlowTree.nr* example and its dependencies, found [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/slow_tree_contract). Replace the constants with whatever you like and deploy it to your sandbox +2. Copy the *SlowMap interface* for easy interaction with your deployed SlowTree. Find it [here](https://github.com/AztecProtocol/aztec-packages/blob/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/interfaces.nr) +3. Import this interface into your contract #include_code interface yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -The **`SlowMap`** interface allows the contract to interact with its attached SlowTree. It abstracts these functions so they do not have to be implemented in the TokenBlacklist contract. +5. Store a slow updates tree in both public and private storage -### Constructor and Initialization of the Slow Updates Tree +#include_code slow_updates_storage yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -The contract's constructor takes the address of the slow updates contract: +6. Store the SlowTree address in private storage as a FieldNote #include_code constructor yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -This initialization sets up the connection between the **`TokenBlacklist`** contract and a previously deployed SlowTree, allowing it to use the SlowMap interface to directly interact with the SlowTree. - -### Private Transfer Function Utilizing the Slow Updates Tree - -In the private transfer function, the contract uses the **`SlowMap`** interface to check if a user is blacklisted: - -#include_code transfer_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust - -Here, the contract reads the roles of the sender and recipient from the SlowTree using the **`read_at`** function in the **`SlowMap`**interface. It checks if either party is blacklisted, and if so, the transaction does not go ahead. +7. Store the SlowTree address in public storage and initialize an instance of SlowMap using this address -## SlowTree.nr smart contract +#include_code write_slow_update_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -Under the hood, this TokenBlacklist contract uses the SlowMap interface to interact with a SlowTree contract. SlowTree.nr is a contract that utilizes the slow update tree library and instansiates and manipulates a tree. +8. Now you can read and update from private functions: -If you are using the SlowTree contract with the SlowMap interface, the exact implementation details of this smart contract are not required to know. You can read the code [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/slow_tree_contract). +#include_code get_and_update_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -## The SlowMap interface +9. Or from public functions: -This interface used by the TokenBlackList contract contains multiple functions for interacting with its associated SlowTree. You can find the interface [here](https://github.com/AztecProtocol/aztec-packages/blob/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/interfaces.nr). +#include_code get_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -## The SlowMap type +View the [reference](slow_updates_tree.md#reference) for more information. -This is a library provided by Aztec for low-level interacts with a slow updates tree. You can see the code for the library [here](https://github.com/AztecProtocol/aztec-nr/blob/master/slow-updates-tree/src/slow_map.nr). +## Exploring an example integration through a **`TokenBlacklist`** Smart Contract +The `TokenBlacklist` contract is a token contract that does not allow blacklisted accounts to perform mints or transfers. In this section we will go through how this is achieved through the slow updates tree. -# How to integrate a slow updates tree +You can find the full code for the TokenBlacklist smart contract [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract). -You can use this example to implement a slow updates tree in your own smart contract. +### Importing SlowMap -1. Copy the SlowTree.nr example and its dependencies, found [here](https://github.com/AztecProtocol/aztec-packages/tree/master/yarn-project/noir-contracts/src/contracts/slow_tree_contract). Replace the constants with whatever you like and deploy it to your sandbox -2. Copy the SlowMap interface for easy interaction with your deployed SlowTree. Find it [here](https://github.com/AztecProtocol/aztec-packages/blob/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/interfaces.nr) -3. Import the SlowMap interface into your contract +The contract first imports the **`SlowMap`** interface: #include_code interface yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -5. Store a slow updates tree in both public and private storage +This interface allows the contract to interact with its attached SlowTree. It abstracts these functions so they do not have to be implemented in the TokenBlacklist contract. -#include_code slow_updates_storage yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust +### Constructor and Initialization -6. Store the SlowTree address in private storage as a FieldNote +The contract's constructor takes the address of the slow updates contract: #include_code constructor yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -7. Store the SlowTree address in public storage and initialize an instance of SlowMap using this address +This initialization sets up the connection between the **`TokenBlacklist`** contract and a previously deployed SlowTree, allowing it to use the interface to directly interact with the SlowTree. -#include_code write_slow_update_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust - -8. Now you can read and update from private functions: +### Private Transfer Function Utilizing the Slow Updates Tree -#include_code get_and_update_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust +In the private transfer function, the contract uses the interface to check if a user is blacklisted: -9. Or from public functions: +#include_code transfer_private yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -#include_code get_public yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust +Here, the contract reads the roles of the sender and recipient from the SlowTree using the **`read_at`** function in the interface. It checks if either party is blacklisted, and if so, the transaction does not go ahead. # Reference From e6a95f538fa473c5a877a4a58117e0f36649128a Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Thu, 30 Nov 2023 15:48:25 +0600 Subject: [PATCH 13/16] historic -> historical --- .../public_private_calls/slow_updates_tree.md | 8 ++++---- docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md index 32266cb1532..be604933fab 100644 --- a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md +++ b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md @@ -1,5 +1,5 @@ --- -title: Privately access Historic Public Data +title: Privately access Historical Public Data --- In Aztec, private and public execution environments are completely separate and operate with distinct state management. It is not possible for private functions to reliably access the most recent public data public state - only sequencers can do that. @@ -17,8 +17,8 @@ On this page you will learn: This structure was created specifically to privately & publicly access historical public data. It should be used to store public data that doesn't change often. -- Access historic public data from a private function -- Access historic public data from a public function +- Access historical public data from a private function +- Access historical public data from a public function - Update public data (that does not need updated often) from public and private functions This data structure is ideal for these use cases: @@ -32,7 +32,7 @@ We developed the Slow Updates Tree to help balance public and private execution **First iteration: Shared State Tree** -Using a shared state tree, it is possible to privately access historic public data by providing a membership proof to show that a value is indeed part of a commitment, and then check that the commitment matches the one stored in the state, as shown below. +Using a shared state tree, it is possible to privately access historical public data by providing a membership proof to show that a value is indeed part of a commitment, and then check that the commitment matches the one stored in the state, as shown below. ```mermaid graph TD; diff --git a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md index d68614f36db..17aab72a0f0 100644 --- a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md +++ b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md @@ -2,7 +2,7 @@ title: Slow Updates Tree --- -Slow Updates Tree is a data structure that allows for historic public data to be accessed in both private and public domains. Read the high level overview in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). +Slow Updates Tree is a data structure that allows for historical public data to be accessed in both private and public domains. Read the high level overview in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). The Slow Updates Tree works by having a current tree and a pending tree, and replacing the current tree with the pending tree after an epoch has passed. Public functions can read directly from the current tree, and private functions can perform a membership proof that values are part of a commitment to the current state of the tree. From d3c98c125f67b8a484eef4881b6bd05077cc5d4d Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Fri, 1 Dec 2023 13:42:14 +0600 Subject: [PATCH 14/16] comments & grammar --- .../roadmap/engineering_roadmap.md | 4 +- .../docs/concepts/foundation/accounts/keys.md | 2 +- .../docs/concepts/foundation/accounts/main.md | 2 +- .../public_private_calls/main.md | 2 + .../public_private_calls/slow_updates_tree.md | 41 ++----------------- .../resources/common_patterns/main.md | 2 +- .../contracts/syntax/slow_updates_tree.md | 22 +++++----- .../dev_docs/getting_started/core-concepts.md | 2 +- 8 files changed, 23 insertions(+), 54 deletions(-) diff --git a/docs/docs/about_aztec/roadmap/engineering_roadmap.md b/docs/docs/about_aztec/roadmap/engineering_roadmap.md index a9f71aa40fa..890d0f1e14a 100644 --- a/docs/docs/about_aztec/roadmap/engineering_roadmap.md +++ b/docs/docs/about_aztec/roadmap/engineering_roadmap.md @@ -119,7 +119,9 @@ CI takes up a significant amount of time. It gets its own section here, so we re ## Slow Updates tree? -We _need_ a way to read mutable public data from a private function. +We _need_ a way to read mutable public data from a private function. + +Note: we just published the [Slow Updates Tree](../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). ## Contract classes and instances? diff --git a/docs/docs/concepts/foundation/accounts/keys.md b/docs/docs/concepts/foundation/accounts/keys.md index b3a49ffbe93..589667cbab3 100644 --- a/docs/docs/concepts/foundation/accounts/keys.md +++ b/docs/docs/concepts/foundation/accounts/keys.md @@ -28,7 +28,7 @@ Similar to using a private note, but using an immutable private note removes the ### Using the slow updates tree -A compromise between the two solutions above is to use the slow updates tree. This would not generate additional nullifiers and commitments for each transaction while allowing the user to rotate their key. However, this causes every transaction to now have a time-to-live determined by the frequency of the slow updates tree. +A compromise between the two solutions above is to use the [slow updates tree](../communication/public_private_calls/slow_updates_tree.md). This would not generate additional nullifiers and commitments for each transaction while allowing the user to rotate their key. However, this causes every transaction to now have a time-to-live determined by the frequency of the slow updates tree. ### Reusing the privacy master key diff --git a/docs/docs/concepts/foundation/accounts/main.md b/docs/docs/concepts/foundation/accounts/main.md index 23dcfa6c549..c96238632be 100644 --- a/docs/docs/concepts/foundation/accounts/main.md +++ b/docs/docs/concepts/foundation/accounts/main.md @@ -127,7 +127,7 @@ These two patterns combined allow an account contract to answer whether an actio Aztec requires users to define [encryption and nullifying keys](./keys.md) that are needed for receiving and spending private notes. Unlike transaction signing, encryption and nullifying is enshrined at the protocol. This means that there is a single scheme used for encryption and nullifying. These keys are derived from a master public key. This master public key, in turn, is used when deterministically deriving the account's address. -A side effect of committing to a master public key as part of the address is that _this key cannot be rotated_. While an account contract implementation could include methods for rotating the signing key, this is unfortunately not possible for encryption and nullifying keys (note that rotating nullifying keys also creates other challenges such as preventing double spends). We are exploring usage of the slow updates tree to enable rotating these keys. +A side effect of committing to a master public key as part of the address is that _this key cannot be rotated_. While an account contract implementation could include methods for rotating the signing key, this is unfortunately not possible for encryption and nullifying keys (note that rotating nullifying keys also creates other challenges such as preventing double spends). We are exploring usage of the [slow updates tree](../communication/public_private_calls/slow_updates_tree.md) to enable rotating these keys. NOTE: While we entertained the idea of abstracting note encryption, where account contracts would define an `encrypt` method that would use a user-defined scheme, there are two main reasons we decided against this. First is that this entailed that, in order to receive funds, a user had to first deploy their account contract, which is a major UX issue. Second, users could define malicious `encrypt` methods that failed in certain circumstances, breaking application flows that required them to receive a private note. While this issue already exists in Ethereum when transferring ETH (see the [king of the hill](https://coinsbench.com/27-king-ethernaut-da5021cd4aa6)), its impact is made worse in Aztec since any execution failure in a private function makes the entire transaction unprovable (ie it is not possible to catch errors in calls to other private functions), and furthermore because encryption is required for any private state (not just for transferring ETH). Nevertheless, both of these problems are solvable. Initialization can be worked around by embedding a commitment to the bytecode in the address and removing the need for actually deploying contracts before interacting with them, and the king of the hill issue can be mitigated by introducing a full private VM that allows catching reverts. As such, we may be able to abstract encryption in the future as well. diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/main.md b/docs/docs/concepts/foundation/communication/public_private_calls/main.md index 2feada08278..e406053e8eb 100644 --- a/docs/docs/concepts/foundation/communication/public_private_calls/main.md +++ b/docs/docs/concepts/foundation/communication/public_private_calls/main.md @@ -105,3 +105,5 @@ function onlyFresh(pub uint256 blockNumber) public { :::info This is not a perfect solution, as any functions using access control might end up doing a lot of public calls it could put a significant burden on sequencers and greatly increase the cost of the transaction for the user. We are investigating ways to improve. ::: + +Using a dual-tree structure with a pending and a current tree, it is possible to update public data from a private function. The update is fulfilled when the pending tree becomes the current after the end of a specified epoch. It is also possible to read historical public data directly from a private function. This works perfectly for public data that is not updated often, such as a blacklist. This structure is called a slow updates tree, and you can read about how it works [in the next section](./slow_updates_tree.md). diff --git a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md index be604933fab..c6ceb070e26 100644 --- a/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md +++ b/docs/docs/concepts/foundation/communication/public_private_calls/slow_updates_tree.md @@ -2,7 +2,7 @@ title: Privately access Historical Public Data --- -In Aztec, private and public execution environments are completely separate and operate with distinct state management. It is not possible for private functions to reliably access the most recent public data public state - only sequencers can do that. +In Aztec, private and public execution environments are completely separate and operate with distinct state management. It is not possible for private functions to reliably access the most recent public data public state - only sequencers can do that. You'll want to [read the previous section](./main.md) to understand this before reading this page. But, what about historical public data (or public data that changes infrequently)? Through a **slow updates tree**, private functions can access historical public state. Please note that we are still experimenting with this feature. @@ -28,44 +28,9 @@ This data structure is ideal for these use cases: ## How it works -We developed the Slow Updates Tree to help balance public and private execution in a blockchain context. Earlier systems typically used either fully public or entirely private state trees. +We developed the Slow Updates Tree to help balance public and private execution in a blockchain context. Earlier systems typically used either fully public or entirely private state trees. -**First iteration: Shared State Tree** - -Using a shared state tree, it is possible to privately access historical public data by providing a membership proof to show that a value is indeed part of a commitment, and then check that the commitment matches the one stored in the state, as shown below. - -```mermaid -graph TD; - Cm[Commmitment1] --> Vm1[Value1] - Cm --> Vm2[Value2] - Cm --> Vmn[Value3] - Vm1 --> Mp1[Membership Proof for Value1] - Mp1 --> St[Commitment in State] - St --> Cm -``` - -Note: a *commitment* refers to a cryptographic assurance that a specific set of data is included in the tree. In the current design, we are using Merkle trees to commit to the values. - -However, these would be contract-specific trees, so we will be leaking the contract address of the contract doing a lookup. - -**Second iteration: Multi-level Shared State Tree** - -This privacy issue can be solved by organizing data into layers, with each contract having its own commitment within a larger tree. Only the top-level commitment is revealed. - -```mermaid -graph TD; - C[Top Commitment] -->|Contains| C1[Commitment1] - C -->|Contains| Cm[Commitment2] - Cm -->|Contains| Vm1[Value1] - Cm -->|Contains| Vm2[Value2] - Cm -->|Contains| Vmn[Valuen] -``` - -However, this means that any canges in any contract's data will change the top-level commitment and invalidate the proofs for all other contracts. - -**Third iteration: Slow Updates Tree** - -To solve this, the Slow Updates Tree is a dual-tree structure - a current tree and a pending tree. Any updates are added to the pending tree, which then becomes the current tree at the end of an epoch. The pending tree is replicated from the current tree, and the cycle begins again. +The Slow Updates Tree is a dual-tree structure - a current tree and a pending tree. Any updates are added to the pending tree, which then becomes the current tree at the end of an epoch. The pending tree is replicated from the current tree, and the cycle begins again. ```mermaid graph TD; diff --git a/docs/docs/dev_docs/contracts/resources/common_patterns/main.md b/docs/docs/dev_docs/contracts/resources/common_patterns/main.md index 82afee5c96d..770e92f4e92 100644 --- a/docs/docs/dev_docs/contracts/resources/common_patterns/main.md +++ b/docs/docs/dev_docs/contracts/resources/common_patterns/main.md @@ -41,7 +41,7 @@ Note - you could also create a note and send it to the user. The problem is ther ### Reading public storage in private You can't read public storage in private domain. But nevertheless reading public storage is desirable. There are two ways: -1. For public storage that changes infrequently, use the slow updates tree! More details TBD +1. For public storage that changes infrequently, use the slow updates tree! Learn more about it [here](../../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). 2. You pass the data as a parameter to your private method and later assert in public that the data is correct. E.g.: ```rust diff --git a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md index 17aab72a0f0..e108e0cff23 100644 --- a/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md +++ b/docs/docs/dev_docs/contracts/syntax/slow_updates_tree.md @@ -4,28 +4,28 @@ title: Slow Updates Tree Slow Updates Tree is a data structure that allows for historical public data to be accessed in both private and public domains. Read the high level overview in the [concepts section](../../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md). -The Slow Updates Tree works by having a current tree and a pending tree, and replacing the current tree with the pending tree after an epoch has passed. Public functions can read directly from the current tree, and private functions can perform a membership proof that values are part of a commitment to the current state of the tree. +The slow updates tree works by having a current tree and a pending tree, and replacing the current tree with the pending tree after an epoch has passed. Public functions can read directly from the current tree, and private functions can perform a membership proof that values are part of a commitment to the current state of the tree. On this page you will learn: -1. [The components involved in using the Slow Updates Tree](slow_updates_tree.md#components-involved-in-implementing-a-slow-updates-tree) +1. [The components involved in using the slow updates tree](slow_updates_tree.md#components-involved-in-implementing-a-slow-updates-tree) 2. [How you can integrate it into your own smart contract](slow_updates_tree.md#how-to-integrate-a-slow-updates-tree) 3. [An example of a token blacklisting contract that uses the slow updates tree](slow_updates_tree.md#exploring-an-example-integration-through-a-tokenblacklist-smart-contract) 4. [Interface Reference](slow_updates_tree.md#reference) -# Components involved in implementing a Slow Updates Tree +# Components involved in implementing a slow updates tree -There are generally 4 main components involved to make it easier to use a Slow Updates Tree, with 3 already implemented by Aztec. This makes it easier to interact with a slow updates tree through a simple interface. These four components are: +There are generally 4 main components involved to make it easier to use a slow updates tree, with 3 already implemented by Aztec. This makes it easier to interact with a slow updates tree through a simple interface. These four components are: -## Main Smart Contract +## Main smart contract -This is the primary smart contract that will use the Slow Updates Tree. In the example we use a [token with blacklisting features](slow_updates_tree.md#exploring-an-example-integration-through-a-tokenblacklist-smart-contract). +This is the primary smart contract that will use the slow updates tree. In the example we use a [token with blacklisting features](slow_updates_tree.md#exploring-an-example-integration-through-a-tokenblacklist-smart-contract). ## Interface -This interface can be used by your contract to interact with the Slow Updates Tree. It provides methods for reading and updating values in the tree in both public and private contexts. You can find it [here](https://github.com/AztecProtocol/aztec-packages/blob/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/interfaces.nr). +This interface of the slow updates tree contract allows your contract to interact with the Slow Updates Tree contract. It provides methods for reading and updating values in the tree in both public and private contexts. You can find it [here](https://github.com/AztecProtocol/aztec-packages/blob/master/yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/interfaces.nr). -## SlowTree.nr Contract +## SlowTree.nr contract This is a smart contract developed by Aztec that establishes and manages a slow updates tree structure. It allows developers to access and interact with the tree, such as reading and updating data. @@ -93,7 +93,7 @@ graph TD #include_code interface yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust -5. Store a slow updates tree in both public and private storage +5. Create a storage init function for the same value in both public and private storage #include_code slow_updates_storage yarn-project/noir-contracts/src/contracts/token_blacklist_contract/src/main.nr rust @@ -129,7 +129,7 @@ The contract first imports the **`SlowMap`** interface: This interface allows the contract to interact with its attached SlowTree. It abstracts these functions so they do not have to be implemented in the TokenBlacklist contract. -### Constructor and Initialization +### Constructor and initialization The contract's constructor takes the address of the slow updates contract: @@ -137,7 +137,7 @@ The contract's constructor takes the address of the slow updates contract: This initialization sets up the connection between the **`TokenBlacklist`** contract and a previously deployed SlowTree, allowing it to use the interface to directly interact with the SlowTree. -### Private Transfer Function Utilizing the Slow Updates Tree +### Private transfer function utilizing the slow updates tree In the private transfer function, the contract uses the interface to check if a user is blacklisted: diff --git a/docs/docs/dev_docs/getting_started/core-concepts.md b/docs/docs/dev_docs/getting_started/core-concepts.md index e22e1d3a04d..fdc8a91ca5a 100644 --- a/docs/docs/dev_docs/getting_started/core-concepts.md +++ b/docs/docs/dev_docs/getting_started/core-concepts.md @@ -31,7 +31,7 @@ You can call a public function from a private function by using `context.call_pu #include_code call_public_function yarn-project/noir-contracts/src/contracts/card_game_contract/src/main.nr rust -You cannot call a private function from a public function, but you can use a slow updates tree to read historic public state and stage writes to public state from a private function. +You cannot call a private function from a public function, but you can use a [slow updates tree](../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md) to read historic public state and stage writes to public state from a private function. ### Data types From 2f3e7334814e5bb637e8ee98d4e193b4f06e79b4 Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Fri, 1 Dec 2023 13:44:18 +0600 Subject: [PATCH 15/16] broken link --- docs/docs/dev_docs/tutorials/writing_private_voting_contract.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/dev_docs/tutorials/writing_private_voting_contract.md b/docs/docs/dev_docs/tutorials/writing_private_voting_contract.md index ecfa4fb2b2d..a541e3abf5d 100644 --- a/docs/docs/dev_docs/tutorials/writing_private_voting_contract.md +++ b/docs/docs/dev_docs/tutorials/writing_private_voting_contract.md @@ -105,7 +105,7 @@ This `init` function will be called every time we access `storage` in our functi The next step is to initialize the contract with a constructor. The constructor will take an address as a parameter and set the admin. -All constructors must be private, and because the admin is in public storage, we cannot directly update it from the constructor. You can find more information about this [here](../../concepts/foundation/communication/public_private_calls.md). +All constructors must be private, and because the admin is in public storage, we cannot directly update it from the constructor. You can find more information about this [here](../../concepts/foundation/communication/public_private_calls/main.md). Therefore our constructor must call a public function by using `context.call_public_function()`. Paste this under the `impl` storage block: From 78cd7247a8371d4ad3e5cee0c53a559a477f078f Mon Sep 17 00:00:00 2001 From: Cat McGee Date: Fri, 1 Dec 2023 14:28:36 +0600 Subject: [PATCH 16/16] merge conflicts --- docs/docs/dev_docs/getting_started/core-concepts.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/dev_docs/getting_started/core-concepts.md b/docs/docs/dev_docs/getting_started/core-concepts.md index fdc8a91ca5a..e5087c624c1 100644 --- a/docs/docs/dev_docs/getting_started/core-concepts.md +++ b/docs/docs/dev_docs/getting_started/core-concepts.md @@ -31,7 +31,7 @@ You can call a public function from a private function by using `context.call_pu #include_code call_public_function yarn-project/noir-contracts/src/contracts/card_game_contract/src/main.nr rust -You cannot call a private function from a public function, but you can use a [slow updates tree](../../concepts/foundation/communication/public_private_calls/slow_updates_tree.md) to read historic public state and stage writes to public state from a private function. +You cannot call a private function from a public function, but you can use a slow updates tree to read historical public state and stage writes to public state from a private function. ### Data types