Basic Yul compiler #16

g-r-a-n-t · 2020-02-17T09:26:30Z

Part of #10.

This includes just enough functionality to compile a contract like this:

contract Foo:
    pub def bar(x: u256) -> u256:
        return x

To keep it simple, types are basically ignored (everyting is u256) and all variables are stored on the stack. There are some Scope data structures in here as well, which aren't necessary for it to work, but would be worth reviewing as I intend to build more complexity on top of them.

The Yul code generated by the compiler would not actually be runnable, since it doesn't have the ABI builder from #15 to generate the selector switch from. In #17, I add that to this module and demonstrate that the contract can be deployed and ran in a test.

Selectors testing. Selectors testing.

ralexstokes

nice work! i like how you have broken a big chunk of work into digestible PRs... looking forward to seeing it all come together :)

i left some comments but overall seems like a good foundation for extending the compiler

ralexstokes · 2020-05-12T02:49:44Z

compiler/Cargo.toml

+[dependencies]
+vyper-parser = {path = "../parser", version = "0.1.0"}
+hex = "0.4"
+yultsur = { git = "https://github.com/g-r-a-n-t/yultsur" }


anything special in this fork of yultsur compared to the one maintained by the solidity devs?

I needed to add a few structs that exist in the Yul spec, but not in yultsur.

ralexstokes · 2020-05-12T02:50:37Z

compiler/src/yul/mod.rs

+    let tokens = parser::get_parse_tokens(src)?;
+    let vyp_module = parser::parsers::file_input(&tokens[..])?.1.node;
+
+    // TODO: Handle multiple contracts in one Vyper module cleanly.


note: resolved in future PRs

ralexstokes · 2020-05-12T02:52:56Z

compiler/src/yul/selectors.rs

+    yul::Expression::FunctionCall(yul::FunctionCall {
+        identifier: base::untyped_identifier("shr"),
+        arguments: vec![
+            base::untyped_literal_expr("224"),
+            yul::Expression::FunctionCall(yul::FunctionCall {
+                identifier: base::untyped_identifier("calldataload"),
+                arguments: vec![base::untyped_literal_expr("0")],
+            }),
+        ],
+    })


"leave the first 4 bytes (big-endian) of the calldata on the stack"

ralexstokes · 2020-05-12T02:54:54Z

compiler/src/yul/selectors.rs

+        keccak.finalize(&mut selector);
+    }
+
+    base::untyped_literal(&format!("0x{}", hex::encode(selector)))


i'd have to dig into yul further but i wonder if we can tighten the typing here? "untyped" literal may be the strictest thing we can specify given a hex string -- curious if you can add any color here...

This is what it looks like now:

literal! { (format!("0x{}", hex::encode(selector))) }

See the literal macro. (I should add better docs to that file, as the notation is somewhat confusing.)

Currently, solc's yul compiler doesn't even handle explicit types. And even if it did, it probably wouldn't be very useful, so I just removed the prefix altogether.

ralexstokes · 2020-05-12T03:05:01Z

compiler/src/yul/selectors.rs

+            block: yul::Block {
+                statements: vec![yul::Statement::Expression(yul::Expression::FunctionCall(
+                    yul::FunctionCall {
+                        identifier: base::untyped_identifier("foo"),


might not need to update here if this is temporary but i expect we can grab some identifier from the ethabi::Function

Oh, woops. This got fixed later on, probably after trying to run a function not named "foo".

ralexstokes · 2020-05-12T03:36:06Z

compiler/src/yul/ast_builder.rs

+    Ok(module
+        .body
+        .iter()
+        .map(|stmt| module_stmt(Rc::clone(&scope), &stmt.node))
+        .collect::<Result<Vec<Option<yul::Statement>>, CompileError>>()?
+        .into_iter()
+        .filter_map(|statement| {
+            if let Some(yul::Statement::Object(object)) = statement {
+                return Some(object);
+            }
+
+            None
+        })
+        .collect::<Vec<yul::Object>>())


i would expect this can be simplified like over here: #15 (comment)

Yeah, I'll remove this pattern wherever I see it.

ralexstokes · 2020-05-12T03:37:48Z

compiler/src/yul/ast_builder.rs

+pub fn module_stmt<'a>(
+    scope: Shared<ModuleScope<'a>>,
+    stmt: &'a vyp::ModuleStmt<'a>,
+) -> Result<Option<yul::Statement>, CompileError> {
+    match stmt {
+        vyp::ModuleStmt::TypeDef { .. } => {
+            type_def(scope, stmt)?;
+            Ok(None)
+        }
+        vyp::ModuleStmt::ContractDef { .. } => {
+            contract_def(scope, stmt).map(|object| Some(yul::Statement::Object(object)))
+        }
+        _ => Err(CompileError::static_str(
+            "Unable to translate module statement.",
+        )),
+    }


seems like the uses of Option here can just be cases of CompileError and then it simplifies much of the control flow...

ralexstokes · 2020-05-12T03:39:33Z

compiler/src/yul/ast_builder.rs

+            .collect::<Result<Vec<yul::Statement>, CompileError>>()?;
+
+        // TODO: Use functions from actual contract ABI once the builder is merged.
+        statements.push(yul::Statement::Switch(selectors::switch(vec![])?));


see comment under selectors module about this

Suggested change

statements.push(yul::Statement::Switch(selectors::switch(vec![])?));

statements.push(yul::Statement::Switch(selectors::dispatch_switch(vec![])?));

ralexstokes · 2020-05-12T03:41:13Z

compiler/src/yul/selectors.rs

@@ -0,0 +1,175 @@
+use crate::errors::CompileError;


something that came across my mind when reviewing this was the genericity of the function names switch, expression and case. you could make the argument that we rebind the names on import elsewhere and they make sense in the context of this module, but it is something to consider.

i left a comment at a usage of switch to point out what I mean

I agree. There must be applicable CS nomenclature for this.

i'd just go with what sounds natural and readable. and ideally unique across as many contexts as possible. of course, we quickly end up optimizing a many-dimensional space and can't satisfy all of our constraints at once :)

off the top of my head, i would call the thing expression as predicate as the selector::predicate, while still jargon-y is clearer to me which part of the expression it is referring to, rather than just selector::expression.

and i'd prolly add some more semantic content to the symbol for case -- something like switch_case or arm or something.... this being said if these functions don't leak outside of this module there is less need to do this given the context gained as a local reader of this module

ralexstokes · 2020-05-12T03:41:29Z

compiler/src/yul/selectors.rs

+}
+
+/// Builds an expression that loads the first 4 bytes of the calldata.
+pub fn expression() -> yul::Expression {


do we need anything pub in this module except switch?

Nope :)

https://github.com/g-r-a-n-t/rust-vyper/blob/types/compiler/src/yul/runtime/abi.rs

ralexstokes · 2020-05-13T03:45:57Z

thanks for the feedback! looks good to me :)

g-r-a-n-t mentioned this pull request Feb 17, 2020

Compile guest book contract to Yul #10

Closed

g-r-a-n-t force-pushed the yul-compiler branch from 1dfbf03 to 8039c29 Compare February 18, 2020 06:30

g-r-a-n-t added 4 commits February 18, 2020 18:35

Yul compiler from ethereum#11.

eac0176

Selectors testing.

604e0f7

Selectors testing. Selectors testing.

Selector cases with no return.

3b8c18a

Inline docs and refactoring.

663ac7f

g-r-a-n-t force-pushed the yul-compiler branch from cce5e1e to 663ac7f Compare February 18, 2020 10:36

ast_builder tests uncommented.

0efbd17

g-r-a-n-t requested review from davesque and pipermerriam February 20, 2020 07:56

g-r-a-n-t changed the title ~~WIP: Basic Yul compiler~~ Basic Yul compiler Feb 20, 2020

g-r-a-n-t mentioned this pull request Feb 20, 2020

Bytecode compiler and EVM tests #17

Merged

g-r-a-n-t mentioned this pull request Apr 2, 2020

Misc features for guestbook #18

Merged

ralexstokes approved these changes May 12, 2020

View reviewed changes

g-r-a-n-t merged commit e54a151 into ethereum:master Aug 25, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Basic Yul compiler #16

Basic Yul compiler #16

g-r-a-n-t commented Feb 17, 2020 •

edited

Loading

ralexstokes left a comment

ralexstokes May 12, 2020

g-r-a-n-t May 12, 2020

ralexstokes May 12, 2020

ralexstokes May 12, 2020

ralexstokes May 12, 2020

g-r-a-n-t May 12, 2020 •

edited

Loading

ralexstokes May 13, 2020

ralexstokes May 12, 2020

g-r-a-n-t May 12, 2020

ralexstokes May 12, 2020

g-r-a-n-t May 12, 2020

ralexstokes May 12, 2020

ralexstokes May 12, 2020

ralexstokes May 12, 2020

g-r-a-n-t May 12, 2020

ralexstokes May 13, 2020

ralexstokes May 12, 2020

g-r-a-n-t May 12, 2020

ralexstokes commented May 13, 2020

	statements.push(yul::Statement::Switch(selectors::switch(vec![])?));
	statements.push(yul::Statement::Switch(selectors::dispatch_switch(vec![])?));

Basic Yul compiler #16

Basic Yul compiler #16

Conversation

g-r-a-n-t commented Feb 17, 2020 • edited Loading

ralexstokes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

g-r-a-n-t May 12, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ralexstokes commented May 13, 2020

g-r-a-n-t commented Feb 17, 2020 •

edited

Loading

g-r-a-n-t May 12, 2020 •

edited

Loading