feat: Aztec prelude #4496
feat: Aztec prelude #4496
Conversation
Benchmark resultsMetrics with a significant change:
Detailed resultsAll benchmarks are run on txs on the This benchmark source data is available in JSON format on S3 here. Values are compared against data from master at commit L2 block published to L1Each column represents the number of txs on an L2 block published to L1.
L2 chain processingEach column represents the number of blocks on the L2 chain where each block has 16 txs.
Circuits statsStats on running time and I/O sizes collected for every circuit run across all benchmarks.
Tree insertion statsThe duration to insert a fixed batch of leaves into each tree type.
MiscellaneousTransaction sizes based on how many contracts are deployed in the tx.
Transaction processing duration by data writes.
|
noir/test_programs/execution_success/assert_statement/src/main.nr
Outdated
Show resolved
Hide resolved
yarn-project/noir-contracts/contracts/inclusion_proofs_contract/src/main.nr
Outdated
Show resolved
Hide resolved
yarn-project/noir-contracts/contracts/easy_private_voting_contract/src/main.nr
Outdated
Show resolved
Hide resolved
yarn-project/noir-contracts/contracts/inclusion_proofs_contract/src/main.nr
Outdated
Show resolved
Hide resolved
yarn-project/noir-contracts/contracts/test_contract/src/main.nr
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
This is great work!
Would it be at all possible to include everything with a aztec::? e.g. aztec::get_portal_address()? This way we do not pollute keywords that devs might not know about?
curious to hear @iAmMichaelConnor opinion as well
Closes #2918. This adds a new macro function that processes the unresolved trait impls and injects a new function before name resolution takes place. This lets us leverage the parser and write the function in Noir instead of having to deal with more complicated processed objects. In order to find all of the note types we look for impls of the `NoteInterface` trait. This is a bit more involved than it seems, since other crates may also have structs that impl this trait, and those will have already been processed. We rely on the fact that the contract crate is processed last, and search for external crate impls via the NodeInterner and internal ones in the unresolved functions. This method is robust as long as we do the contract crate after all of its imports, which holds because the imports are recursively collected from the root crate. I also added a small escape hatch mechanism by skipping any code generation if the function is already defined by the user. This could use some polishing since it is possible for the user to e.g. provide the function but get the arity or parameter types wrong, in which case they'd get a duplicate definition error. Diagnosis and error messages can be improved here (#4647), but I think a simple mechanism is sufficient for now. --- ### Minor issues - One of the function arguments is a fixed-size array, which should be as big as the largest note length. We are not yet pulling the note length (#4649), so I'm passing a hardcoded value for now. 12 Fields ought to be enough for anyone. - Doing this introduces an implicit dependency on `AztecAddress` on all contracts. This won't be an issue once #4496 is in, but I did have to manually add it to some of the account contracts for now. - I created a new macro function that is called on each crate after collection but before resolution. Due to some internal compilers shenanigans (mostly who holds mut references to what) I chose to specialize this function so that for now it only passes the collected unresolved functions, making it a bit niche for the current use case. @vezenovm and I are planning to generalize this down the road (#4653). - I'm now importing in the macro from some places that were not previously used, notably the HIR and Noir errors. I am not sure if we might want to pull those differently - I simply imported what I needed. - I also introduced some getters to provide mutable access to private fields of the HIR Context and CrateDefMap. This is because we need to modify the contract module in the def map by declaring the new function (which is how we get things like duplicate definition detection). We're already mutating the HIR Context in the macros, so also mutating its members doesn't feel like a stretch. - Finally, I don't know enough about how Noir errors work to know how to produce a useful `Location` value for the new function, if indeed that can be done. Providing an empty span seems to at least not cause weird errors on the LSP plugin, so that's how I left it for now.
|
Current dependencies on/for this PR:
This stack of pull requests is managed by Graphite. |
Docs PreviewHey there! 👋 You can check your preview at https://65e5e9bc5761583e3cf60081--aztec-docs-dev.netlify.app |
|
I'm concerned that this adds a little too much "magic" to how we use these types as well as adding a lot of potential for conflicts in future. If every macro processor can add their own preludes then it's almost guaranteed that they're going to clash, especially with types as general as In general, I'm not a fan of importing a large number of types/functions from a library just because we're depending on it. Instead this should be opt in, if we look at preludes from Rust libraries for instance we'd need an It's important to note that while they're core types to the aztec contract writing experience, from Noir's perspective aztec-nr is fundamentally just another library. |
|
I would prefer that for now we just keep the prelude file which would simplify users performing the imports themselves (as they don't need to remember exactly where in Alternatively, in contracts decorated with |
@TomAFrench I don't know what is the best solution implementation-wise here but not including the PR and putting the burden of importing these basic types on aztec contract devs doesn't sound good to me. But if you feel like at this point this would cause more issues then benefits I am ok with not merging it now. But I think once things are more stable we should have something like this (at least for types like AztecAddress).
@TomAFrench In Solidity you can just use types like address directly and I feel like solidity devs are the core audience here. Looking at a contract as a non-rust dev and seeing that I have to import some prelude thing I would have no idea what's going on. But if you feel that some of the types there are too generic I would not mind removing them.
@TomAFrench Why does this matter when the aztec macros are only used for aztec contracts? |
I don't think that Solidity is the right comparison here as the language and the smart contract development framework are one and the same - there's no non smart contract solidity development. A better comparison in my mind would be Arbitrum Stylus or Solana as they're both using a general purpose langauge with a smart contract framework built on top, in both of these cases it's necessary to import from a library to get smart contract types, etc.
Because the ultimate aim is to have the macro logic expressed in Noir source code such that the aztec macros can be included in the |
We can remove these general names if we would like or just use an alias such as
I agree! However, these statements can be applied to the entirety of I'm ok with making this opt-in but its feels like it would actually add confusion for users with some of the new macros the aztec team is working to add. For example here #4610 the
We can improve this UX though. We should maintain the boundary of library and language, but we don't yet have full macros on the Noir side that can enable this.
Yes absolutely. But we are not at this point yet, and we do not have any timeline for it right now. And with new macros potentially using reserved names, |
|
@TomAFrench Most of these comments feel like reasons as to why we should move to get rid of We should set aside a discussion next week on the future of |
* master: (39 commits) fix: Remove the `VerificationKey` from `ProverInstance` (#4908) chore(avm-transpiler): minor rust fixes (#4889) add some migration notes (#4913) fix: depreciated ci image (#4911) feat: Public initializer check (#4894) feat: Use yarns topological build to get rid of explicit sequential steps, and let it solve. (#4868) git subrepo push --branch=master barretenberg fix: Temporarily skip failing deployment test feat: Check initializer by default in private functions (#4832) feat: Allow nullifier proofs in public (#4892) refactor: Move remaining data out of Honk UltraComposer (#4848) chore: Do not download foundry during L1 contracts fast bootstrap (#4865) chore: Add custom inspect for base types (#4890) refactor: rename read request to note hash read request (#4888) feat(avm-simulator): implement NOTEHASHEXISTS (#4882) Fix noir mirror path. feat: public refunds via FPC (#4750) chore: purge SafeU120 (#4819) refactor: replacing use of `L2Tx` with `TxEffect` (#4876) fix: Fetch Headers and Bodies separately #4167 (#4632) ...
Closes #2918. This adds a new macro function that processes the unresolved trait impls and injects a new function before name resolution takes place. This lets us leverage the parser and write the function in Noir instead of having to deal with more complicated processed objects. In order to find all of the note types we look for impls of the `NoteInterface` trait. This is a bit more involved than it seems, since other crates may also have structs that impl this trait, and those will have already been processed. We rely on the fact that the contract crate is processed last, and search for external crate impls via the NodeInterner and internal ones in the unresolved functions. This method is robust as long as we do the contract crate after all of its imports, which holds because the imports are recursively collected from the root crate. I also added a small escape hatch mechanism by skipping any code generation if the function is already defined by the user. This could use some polishing since it is possible for the user to e.g. provide the function but get the arity or parameter types wrong, in which case they'd get a duplicate definition error. Diagnosis and error messages can be improved here (AztecProtocol/aztec-packages#4647), but I think a simple mechanism is sufficient for now. --- ### Minor issues - One of the function arguments is a fixed-size array, which should be as big as the largest note length. We are not yet pulling the note length (AztecProtocol/aztec-packages#4649), so I'm passing a hardcoded value for now. 12 Fields ought to be enough for anyone. - Doing this introduces an implicit dependency on `AztecAddress` on all contracts. This won't be an issue once AztecProtocol/aztec-packages#4496 is in, but I did have to manually add it to some of the account contracts for now. - I created a new macro function that is called on each crate after collection but before resolution. Due to some internal compilers shenanigans (mostly who holds mut references to what) I chose to specialize this function so that for now it only passes the collected unresolved functions, making it a bit niche for the current use case. @vezenovm and I are planning to generalize this down the road (AztecProtocol/aztec-packages#4653). - I'm now importing in the macro from some places that were not previously used, notably the HIR and Noir errors. I am not sure if we might want to pull those differently - I simply imported what I needed. - I also introduced some getters to provide mutable access to private fields of the HIR Context and CrateDefMap. This is because we need to modify the contract module in the def map by declaring the new function (which is how we get things like duplicate definition detection). We're already mutating the HIR Context in the macros, so also mutating its members doesn't feel like a stretch. - Finally, I don't know enough about how Noir errors work to know how to produce a useful `Location` value for the new function, if indeed that can be done. Providing an empty span seems to at least not cause weird errors on the LSP plugin, so that's how I left it for now.
|
This was succeeded by #4929 so i am closing. |
Summary
Resolves #2837
Adds an
aztecprelude similar to the stdlib prelude in Noir or Rust. Now when contracts have anaztecdependency developers do not need to directly import the types specified in the prelude, they are just automatically imported by any contract using aztec-nr. A developer can still import the types if they like but I think it is nice that a developer no longer needs to import types such asAztecAddress, state vars, etc., and can instead just start working with theaztecdep directly.Changes
The
MacroProcessorin Noir operates over an untyped AST before name resolution or def collection, and a typed AST that is processed after name resolution and before type checking.Attempting to make changes to the untyped AST before collection of definitions means we only have the AST of the root file in our crate and are missing the definitions in lower modules of the crate. This PR lets us add definitions to the definition collector for a crate as part of Noir compilation before name resolution.
This PR focuses only on adding an aztec prelude and is only adding to the collected imports. However, this process can be expanded to support injecting other objects into the definition collector. I wanted to start with a simple example and an aztec prelude seemed like a good starting point.EDIT: I decided to add a
prelude.nrintoaztec-nrso it is not hidden from those working on aztec-nr. The macro now simply checks whether we have a certain dependency and returns an extra preludes we want to inject using Noir's already preexisting process for adding preludes. Due to this I had to keep the old strategy for importeduse dep::aztec;into every contract for theaztecmacros. This could perhaps be improved to use a different name so that we can just useprelude.nrexclusively, but I felt this would be better left for a separate PR. cc'ing @Maddiaa0Another goal of this PR is to also provide a simple example of how to re-use Noir's parser to create the AST types we would like to insert instead of having a bunch of helpers such as
call(..)like we currently do inaztec_macros/lib.rs. Hopefully switching to this strategy will also speed up macro development. @nventuro is building on this with his work to removecompute_note_hash_and_nullifier.For now I just removed the imports from the
noir-contractsbut this could be extended to anywhere we seedep::aztec.Additional Context
It would be nice to automatically include the
aztecdependency for Aztec contracts, however, this would touch the noirc driver rather than simply the noirc frontend and would require a different process for enabling additional std dependencies.