updates from shadow master (#398)

* disabling the pager in my pretty printing run script * change the elaboration of `new` and non-local Invocations to match the flat yul object style (#367) * making a change to double check work flow * adding a stub for computing the size of a contract; that'll be its own PR entirely one day fairly soon, but for now, we assume that we just need 32 bytes * using hex values for some boilerplate so that it matches the docs * reworking constructors to just allocate memory * adding some helper functions in util. one is a renaming for transactions from child contracts; the other is basically a type-checked comment pointing out uses of some pretty substantian technical debt * temporarily removing the code to translate child contracts. rewriting the translation of Invocations to remap everything to local invocations instead, into the names that will be created in the flat yul object * removing an assertion and adding translation of string literals * whitespace, scala style * utility function for undoing the transaction renaming * generate flat yul object from child contracts (#369) * rename, add boolean for main-ness in translate decl * changing signature of translateDeclaration to remove a return that doesn't do anything * whitespace, scala style * comments * scraps; adding an assert * changing how child contracts are processed * rewrite of translateTransaction * chainging signature of translateStatement * more threading through inMain * whitespace, scala style * adding some docs * removing the nicer output for typed names in favor of a bland one, per solc's current limitations. adding a comment about why. * adding a temporary fix to the top of the dispatch table pending talking with MC" * removing a stale comment * comments * adding missing declarations to invocations * cleaning up comments and some messy logic around temporary variables * removing comment * more hacks for specific tests in advance of type info * changing hardcoded value in the mustache file for this to be the current contract address * adding a couple of args to get this to typecheck * use type annotations in Yul codegen (#377) * removing a util function to rename functions in the dispatch table; that no longer makes sense with the solution for flattening that we've settled on for now * adding comments; expanding getcontractname * ripping out some code, adding comments * adding a comment that may help to locate a bug later * moving comment closer to the source and updating it * comments * scraps; this change seemed likely but breaks a bunch of tests * quick function to traverse over an expression and check a given predicate on all the annotations * fixing some import dependencies * traversal of a program rather than just an expression; i sure hope i don't have to come back and make this the fully polymorphic map reduce but i bet i will * adding updates to the expression returned in the checker from just the ReferenceIdentifier cases, after some debugging with @mcoblenz * adding additional ParserUtil.updateExprType uses as seems right. bunch of tests fail, some of which i expected some i didn't, and i don't know how to fix a lot of them * updating width calculation to actually inspect the type, now that we know it * removing an assert that caused a bunch of false negatives. * small changes per PR comments * copying a slightly cut down version of linked lists, but this needs to be edited down even more to be the right goal for a demo. * compute the sizes of contracts for allocation (#378) * copying a slightly cut down version of linked lists, but this needs to be edited down even more to be the right goal for a demo. * computing sizes of types recursively * adding some simple hard coded unit tests * copying over a helper from type checker tests to test on actual contracts in files * adding more general tests; it doesn't work but i think i know why * scraps and printlines from trying to debug sizes * changing the type of the size computation function so it has enough information, per MC. reworking tests * renaming size function; fixing the main test * adding some comments and a variable for pointer sizes * formatting, reworking base test * removing the two simpler unit tests. it is hard and somewhat pointless to be the parser and create a contract table by hand, the existing test is informative enough * fixing type checker errors (#380) * debugging scraps * copying a slightly cut down version of linked lists, but this needs to be edited down even more to be the right goal for a demo. * commenting out spammy prints * patch from MC * updating override syntax to not introduce a new field * updating expr type update to set the location and use the one field rather than build new ones * removing printlines * updating logged errors in a few places; this fixes some but not all of #379 * error in resourcesTest was in the test itself; it was incorrectly looking for m to be at int * formatting, whitespace * fixed permission passing test; also errors in the definition of the test, not the checker * fixing all permowned error * scala style, whitespace * scraps * adding a comment, removing a comment, removing an assert * compute offsets, store fields in memory (#384) * fixing a bug in computing sizes by translating from set to list; adding an assert; whitespace and style * adding helper to compute offsets * typoe and adding a test to demonstrate that the bug computing sizes is gone * adding a sandbox test case file * adding a traversal of the declarations sequence in the contract rather than relying on the toList of an unordered collection; using that helper function both in computing sizes and offsets * refactoring tests for util functions to reuse the code to produce a symbol table; adding tests for computing offests * style, whitespace * a couple more tests for offsets * a couple more tests for offsets * updating tests, adding comments for myself * helper to wrap call to field offsets, changing sload and sstore to m* * updating test to show that fields get overwritten; adding json so it gets run because it passes * messing with test files * changing the ganache test script to use the test skipping checking script that i think works across platforms, per #361 * adding sort, per #631 * scala style, fussy fixes to doc strings etc * scala style, fussy fixes to doc strings etc * scala style, fussy fixes to doc strings etc * scala style, fussy fixes to doc strings etc * some house keeping to clean up a util function, removing a source of bugs related to computing sizes * refactoring to change the definition of typed name to have an actual type in it, not just its string representation * adding clause to type name helper for contracts * whitespace in obsidian files * housekeeping, adding docs and removing stale todos * Adding another passing test * updating travis test script to work with solc 0.8.10 * iterating on travis script * allocate fields with pointers, rewrite yulobject and mustache interaction to reflect current object layout (#388) * adding test case for nonprimitive fields in the main contract * adding an ownership to the test field to make it valid obsidian * removing some dead code * removing unused childContracts argument from YulObject * scraps * new test * fixing local util script for new version of solc, adding a little code for #389 but not a complete idea yet * scraps * rewriting yulobject to take an optional int to indicate the size of the main contract and if the contract being elaborated is the main one (if it has as size). this is temporary and not going to last * removing what i am nearly positive is dead code * first half of a big rewrite of the YulObject class * back half of the refactor to update code generation * adding some todos and notes * scraps, scala style and whitespace * scraps * updating the implementation of switch toString to not print out invalid switches with no branches. this fixes the emptycontract dispatch table. * scraps * add the this argument uniformly in codegen not printing, and only remove it for the hashed dispatch table entries * pulling the body of local invocation out into a helper so that it can get called in both places with different arguments * documentation, cleaning up todos * scraps * adding a little more documentation * functions with arguments (#394) * adding test case for nonprimitive fields in the main contract * adding an ownership to the test field to make it valid obsidian * removing some dead code * removing unused childContracts argument from YulObject * scraps * new test * fixing local util script for new version of solc, adding a little code for #389 but not a complete idea yet * scraps * rewriting yulobject to take an optional int to indicate the size of the main contract and if the contract being elaborated is the main one (if it has as size). this is temporary and not going to last * removing what i am nearly positive is dead code * first half of a big rewrite of the YulObject class * back half of the refactor to update code generation * adding some todos and notes * scraps, scala style and whitespace * scraps * updating the implementation of switch toString to not print out invalid switches with no branches. this fixes the emptycontract dispatch table. * scraps * add the this argument uniformly in codegen not printing, and only remove it for the hashed dispatch table entries * pulling the body of local invocation out into a helper so that it can get called in both places with different arguments * documentation, cleaning up todos * scraps * renaming some files * removing half baked linked list test * typo * adding json so this test gets run in Travis * translate constrctors to transactions on the flattened object (#396) * updating test case so that it compiled and hits missing code * Adding new test cases * adding code so that constructors get emitted, named with a hash of their arguments so that objecst with multiple consturctors can call whichever one * fixed constructors with arguments * updating ganache test suite with a few constructor examples * adding code to constructor elaboration to not generate a call to a constructor that doesn't exist * monkeying around with the install script * monkeying around with the install script * monkeying around with the install script * monkeying around with the install script * monkeying with travis more * adding version info to output * Adding debugging output * debugging output * monkeying with the host name * test script * trying running ganache on localhost directly * following some shellcheck fixes * moving npm commands around * checking for ganache in the processes * unhiding ganache output * downgrading node to LTS version 16 * moving some install commands around, trying to get a lower version of node * adding possible node work around * env vars * undoing env vars, since the 16 now got picked up * removing debug messages * removing the ganache install script since that's in travis.yml now * scraps
mcoblenz · Nov 19, 2021 · 61496ad · 61496ad
1 parent 440bdd6
commit 61496ad
Show file tree

Hide file tree

Showing 13 changed files with 218 additions and 63 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -3,12 +3,17 @@ language: scala
 scala:
     - 2.13.5
 
-node_js:
-     - 15.11.0
-
 jdk:
   - oraclejdk12 # as of 10 june 2021, up to jdk12 works but 13+ fail
 
+before_install:
+  - nvm install 16 # as of 19 nov 2021, this version of node works with ganache and 17 does not
+  - npm install -g npm # update npm
+  - npm install -g ganache-cli # install ganache
+  - sudo apt update
+  - sudo apt install libcryptx-perl # crypto library needed for hashing in ganache interactions
+  - sudo snap install jq # JSON tool needed for ganache interactions
+
 install:
   - gradle publish -b Obsidian_Runtime/build.gradle
 
@@ -35,5 +40,4 @@ before_cache:
   - find $HOME/.sbt        -name "*.lock"               -print -delete
 
 addons:
-  hosts:
-    - localhost
+  hostname: localhost
diff --git a/resources/tests/GanacheTests/SetGetConstructorArgs.json b/resources/tests/GanacheTests/SetGetConstructorArgs.json
@@ -0,0 +1,8 @@
+{
+    "gas" : 30000000,
+    "gasprice" : "0x9184e72a000",
+    "startingeth" : 5000000,
+    "numaccts" : 1,
+    "testexp" : "main()",
+    "expected" : "12"
+}
diff --git a/resources/tests/GanacheTests/SetGetConstructorArgs.obs b/resources/tests/GanacheTests/SetGetConstructorArgs.obs
@@ -0,0 +1,22 @@
+contract IntContainer{
+    int x;
+
+    IntContainer@Owned(int init) {
+        x = init;
+    }
+
+    transaction set(int value) {
+        x = value;
+    }
+    transaction get() returns int{
+        return x;
+    }
+}
+
+main contract SetGetConstructorArgs{
+    transaction main() returns int{
+        IntContainer ic = new IntContainer(5);
+        ic.set(12);
+        return (ic.get());
+    }
+}
diff --git a/resources/tests/GanacheTests/SetGetConstructorNoArgs.json b/resources/tests/GanacheTests/SetGetConstructorNoArgs.json
@@ -0,0 +1,8 @@
+{
+    "gas" : 30000000,
+    "gasprice" : "0x9184e72a000",
+    "startingeth" : 5000000,
+    "numaccts" : 1,
+    "testexp" : "main()",
+    "expected" : "12"
+}
diff --git a/resources/tests/GanacheTests/SetGetConstructorNoArgs.obs b/resources/tests/GanacheTests/SetGetConstructorNoArgs.obs
@@ -1,22 +1,22 @@
 contract IntContainer{
     int x;
 
-    IntContainer() {
+    IntContainer@Owned() {
         x = 0;
     }
 
-    transaction set() { // take no parameters
-        x = 5;
+    transaction set(int value) {
+        x = value;
     }
     transaction get() returns int{
         return x;
     }
 }
 
-main contract SetGetNoArgs{
+main contract SetGetConstructorNoArgs{
     transaction main() returns int{
         IntContainer ic = new IntContainer();
-        ic.set();
+        ic.set(12);
         return (ic.get());
     }
 }
diff --git a/resources/tests/GanacheTests/SetGetConstructorNoArgsNoSet.json b/resources/tests/GanacheTests/SetGetConstructorNoArgsNoSet.json
@@ -0,0 +1,8 @@
+{
+    "gas" : 30000000,
+    "gasprice" : "0x9184e72a000",
+    "startingeth" : 5000000,
+    "numaccts" : 1,
+    "testexp" : "main()",
+    "expected" : "0"
+}
diff --git a/resources/tests/GanacheTests/SetGetConstructorNoArgsNoSet.obs b/resources/tests/GanacheTests/SetGetConstructorNoArgsNoSet.obs
@@ -0,0 +1,21 @@
+contract IntContainer{
+    int x;
+
+    IntContainer@Owned() {
+        x = 0;
+    }
+
+    transaction set(int value) {
+        x = value;
+    }
+    transaction get() returns int{
+        return x;
+    }
+}
+
+main contract SetGetConstructorNoArgsNoSet{
+    transaction main() returns int{
+        IntContainer ic = new IntContainer();
+        return (ic.get());
+    }
+}
diff --git a/resources/tests/GanacheTests/SetGetTwoConstructors.json b/resources/tests/GanacheTests/SetGetTwoConstructors.json
@@ -0,0 +1,8 @@
+{
+    "gas" : 30000000,
+    "gasprice" : "0x9184e72a000",
+    "startingeth" : 5000000,
+    "numaccts" : 1,
+    "testexp" : "main()",
+    "expected" : "15"
+}
diff --git a/resources/tests/GanacheTests/SetGetTwoConstructors.obs b/resources/tests/GanacheTests/SetGetTwoConstructors.obs
@@ -0,0 +1,27 @@
+contract IntContainer{
+    int x;
+
+    IntContainer@Owned(int init) {
+        x = init;
+    }
+
+    IntContainer@Owned(int a, int b) {
+        x = a + b;
+    }
+
+    transaction set(int value) {
+        x = value;
+    }
+    transaction get() returns int{
+        return x;
+    }
+}
+
+main contract SetGetTwoConstructors{
+    transaction main() returns int{
+        IntContainer ic = new IntContainer(5,5);
+	int x = ic.get();
+        ic.set(x+5);
+        return (ic.get());
+    }
+}
diff --git a/src/main/scala/edu/cmu/cs/obsidian/codegen/CodeGenYul.scala b/src/main/scala/edu/cmu/cs/obsidian/codegen/CodeGenYul.scala
@@ -1,7 +1,7 @@
 package edu.cmu.cs.obsidian.codegen
 
 import edu.cmu.cs.obsidian.CompilerOptions
-import edu.cmu.cs.obsidian.typecheck.ObsidianType
+import edu.cmu.cs.obsidian.typecheck.{NonPrimitiveType, ObsidianType, UnitType}
 
 import java.io.{File, FileWriter}
 import java.nio.file.{Files, Path, Paths}
@@ -117,13 +117,13 @@ object CodeGenYul extends CodeGenerator {
       * @return the yul statements corresponding to the declaration
       */
     def translateDeclaration(declaration: Declaration, contractName: String, checkedTable: SymbolTable, inMain: Boolean): Seq[YulStatement] = {
+        // nb there is a .asInstanceOf in the mustache glue code that only works if this really
+        // returns a sequence of FunctionDeclaration objects. that's OK for now because it's true,
+        // but as the cases below here get filled in that may not be true and we'll have to fix it.
+
         declaration match {
             case _: Field => Seq() // fields are translated as they are encountered
             case t: Transaction => Seq(translateTransaction(t, contractName, checkedTable, inMain))
-
-            // nb there is a .asInstanceOf in the mustache glue code that only works if this really
-            // returns a sequence of FunctionDeclaration objects. that's OK for now because it's true,
-            // but as the cases below here get filled in that may not be true and we'll have to fix it.
             case _: State =>
                 assert(assertion = false, "TODO")
                 Seq()
@@ -133,9 +133,35 @@ object CodeGenYul extends CodeGenerator {
             case _: JavaFFIContractImpl =>
                 assert(assertion = false, "Java contracts not supported in Yul translation")
                 Seq()
-            case _: Constructor =>
-                assert(assertion = false, "constructors not supported in Yul translation")
-                Seq()
+            case c: Constructor =>
+                // given an obsidian type, pull out the non-primitive type or raise an exception
+                def nonprim(t: ObsidianType): NonPrimitiveType = {
+                    t match {
+                        case npt: NonPrimitiveType => npt
+                        case _ => throw new RuntimeException("needed a non-primitive type")
+                    }
+                }
+
+                // constructors turn into transactions with a special name and the same body
+                Seq(translateTransaction(
+                    Transaction(
+                        //to support multiple constructors, constructors get the hash of their
+                        // argument types added to their name
+                        name = c.name + hashOfFunctionName(c.name, c.args.map(v => v.typIn.toString)),
+                        // we omit generic type information because we don't have it and would need
+                        // to reconstruct it, and we don't use it to translate to yul anyway
+                        params = Seq(),
+                        args = c.args,
+                        retType = c.retType,
+                        // we omit any ensures because that feature is largely deprecated
+                        ensures = Seq(),
+                        body = c.body,
+                        isStatic = false,
+                        isPrivate = false,
+                        thisType = nonprim(c.thisType),
+                        thisFinalType = nonprim(c.thisFinalType)
+                    ),
+                    contractName, checkedTable, inMain))
             case _: TypeDecl =>
                 assert(assertion = false, "TODO")
                 Seq()
@@ -163,10 +189,14 @@ object CodeGenYul extends CodeGenerator {
         // return the function definition formed from the above parts, with an added special argument called `this` for the address
         // of the allocated instance on which it should act
         addThisArgument(
-            FunctionDefinition(name = if (inMain) { transaction.name } else { transactionNameMapping(contractName, transaction.name) },
-            parameters = transaction.args.map(v => TypedName(v.varName, v.typIn)),
-            ret,
-            body = Block(body)))
+            FunctionDefinition(name = if (inMain) {
+                transaction.name
+            } else {
+                transactionNameMapping(contractName, transaction.name)
+            },
+                parameters = transaction.args.map(v => TypedName(v.varName, v.typIn)),
+                ret,
+                body = Block(body)))
     }
 
     /**
@@ -320,22 +350,22 @@ object CodeGenYul extends CodeGenerator {
     /** This encapsulates a general pattern of translation shared between both local and general
       * invocations, as called below in the two relevant cases of translate expression.
       *
-      * @param name the name of the thing being invoked
-      * @param args the arguments to the invokee
-      * @param obstype the type at the invocation site
-      * @param thisID where to look in memory for the relevant fields
-      * @param retvar the tempory variable to store the return
+      * @param name         the name of the thing being invoked
+      * @param args         the arguments to the invokee
+      * @param obstype      the type at the invocation site
+      * @param thisID       where to look in memory for the relevant fields
+      * @param retvar       the tempory variable to store the return
       * @param contractName the overall name of the contract being translated
       * @param checkedTable the checked tabled for the overall contract
-      * @param inMain whether or not this is being elborated in main
+      * @param inMain       whether or not this is being elborated in main
       * @return the sequence of yul statements that are the translation of the invocation so described
       */
-    def translateInvocation(name : String,
+    def translateInvocation(name: String,
                             args: Seq[Expression],
                             obstype: Option[ObsidianType],
                             thisID: Identifier,
                             retvar: Identifier, contractName: String, checkedTable: SymbolTable, inMain: Boolean
-                           ): Seq[YulStatement] ={
+                           ): Seq[YulStatement] = {
         // look up the name of the function in the table, get its return type, and then compute
         // how wide of a tuple that return type is. right now that's either 1 (if the
         // transaction returns) or 0 (because it's void)
@@ -456,27 +486,50 @@ object CodeGenYul extends CodeGenerator {
                 val recipient_yul = translateExpr(id_recipient, recipient, contractName, checkedTable, inMain)
 
                 (decl_0exp(id_recipient) +: recipient_yul) ++
-                    // todo: this may be the cause of a bug in the future. this is how non-main functions get their names translated before calling, but i'm not sure that's right at all.
+                    // todo: this may be the cause of a bug in the future. this is how non-main
+                    //  functions get their names translated before calling, but that might not
+                    //  work with multiple contracts and private transactions. i'm not sure.
                     translateInvocation(transactionNameMapping(getContractName(recipient), name),
                         args,
                         obstype,
                         id_recipient,
-                        retvar, contractName, checkedTable, inMain
-                    )
+                        retvar, contractName, checkedTable, inMain)
 
             case Construction(contractType, args, isFFIInvocation, obstype) =>
-                // todo: currently we ignore the arguments to the constructor
-                assert(args.isEmpty, "contracts that take arguments are not yet supported")
-
+                // grab an identifier to store memory
                 val id_memaddr = nextTemp()
 
-                Seq(
-                    // grab the appropriate amount of space of memory sequentially, off the free memory pointer
+                // store the names of the types of the arguments
+                val typeNames = args.map(e => e.obstype.get.toString)
+
+                // given a declaration, test if it's a constructor with type arguments that match the usage here
+                def isMatchingConstructor(d: Declaration): Boolean =
+                    d match {
+                        case c: Constructor => typeNames == c.args.map(v => v.typIn.toString)
+                        case _ => false
+                    }
+
+                // check to to see if there is a constructor to call, and if so translate the
+                // arguments and invoke the constructor as normal transaction with the hash appended
+                // to the name to call the right one
+                val conCall =
+                    if (checkedTable.contract(contractType.contractName).get.contract.declarations.exists(d => isMatchingConstructor(d))) {
+                        translateInvocation(name = transactionNameMapping(contractType.contractName, contractType.contractName) + hashOfFunctionName(contractType.contractName, typeNames),
+                            args = args,
+                            obstype = Some(UnitType()),
+                            thisID = id_memaddr,
+                            retvar = retvar, contractName = contractName, checkedTable = checkedTable, inMain = inMain)
+                    } else {
+                        Seq()
+                    }
+
+                Seq(// grab the appropriate amount of space of memory sequentially, off the free memory pointer
                     decl_1exp(id_memaddr, apply("allocate_memory", intlit(sizeOfContractST(contractType.contractName, checkedTable)))),
 
                     // return the address that the space starts at
-                    assign1(retvar, id_memaddr)
-                )
+                    assign1(retvar, id_memaddr)) ++ conCall
+
+
             case StateInitializer(stateName, fieldName, obstype) =>
                 assert(assertion = false, "TODO: translation of " + e.toString + " is not implemented")
                 Seq()

diff --git a/src/main/scala/edu/cmu/cs/obsidian/codegen/Util.scala b/src/main/scala/edu/cmu/cs/obsidian/codegen/Util.scala
@@ -212,7 +212,7 @@ object Util {
                 case Int256Type() => 1
                 case UnitType() => 0
             }
-            case _: NonPrimitiveType => assert(assertion = false, "width not implemented for non-primitive types!"); -1
+            case _: NonPrimitiveType => 1
             case BottomType() => assert(assertion = false, "width not implemented for the bottom type!"); -1
         }
     }