finish wasm backend

Author: yangdanny97

README.md

@@ -1,6 +1,6 @@
 # chocopy-python-compiler
 
-Ahead-of-time compiler for [Chocopy](https://chocopy.org/), a subset of Python 3.6 with type annotations static type checking. 
+Ahead-of-time compiler for [Chocopy](https://chocopy.org/), a subset of Python 3.6 with type annotations and static type checking. 
 
 Chocopy is used in compiler courses at several universities. This project has no relation to those courses, and is purely for my own learning/practice/fun. 
 
@@ -13,12 +13,13 @@ Progress is documented on my [blog](https://yangdanny97.github.io/blog/):
 
 This compiler is written entirely in Python. Since Chocopy is itself a subset of Python, lexing and parsing can be entirely handled by Python's `ast` module.
 
-This compiler matches the functionality of the first 2 passes (parsing & typechecking) Chocopy's reference compiler implementation, and outputs the AST in a JSON format that is compatible with the reference implementation's backend. That means that you can parse and typecheck the Chocopy file with this compiler, then use the reference implementation's backend to handle assembly code generation.
+The frontend of this compiler matches the functionality of the first 2 passes (parsing & typechecking) Chocopy's reference compiler implementation, and outputs the AST in a JSON format that is compatible with the reference implementation's backend. That means that you can parse and typecheck the Chocopy file with this compiler, then use the reference implementation's backend to handle assembly code generation.
 
-Additionally, this compiler contains 2 backends not found in the reference implementation: 
+This compiler contains multiple backends not found in the reference implementation: 
 - Untyped Python 3 source code
 - JVM bytecode, formatted for the Krakatau assembler
 - CIL bytecode, formatted for the Mono ilasm assembler
+- WASM, in WAT format
 
 The test suite includes both static validation of generated/annotated ASTs, as well as runtime tests that actually execute the output programs to check correctness. Many of the AST validation test cases are taken from test suites included in the release code for Berkeley's CS164, with some additional tests written for more coverage.
 
@@ -42,6 +43,7 @@ The input file should have extension `.py`. If the output file is not provided,
 - Python source outputs will be written to a file of the same name/location as the input file, with extension `.out.py`
 - JVM outputs will be written to the same location as the input file, with the extension `.j`
 - CIL outputs will be written to the same location as the input file, with the extension `.cil`
+- WASM outputs will be written to the same location as the input file, with the extension `.wat`
 
 **Flags:**
 
@@ -56,7 +58,7 @@ The input file should have extension `.py`. If the output file is not provided,
     - `hoist` - output untyped Python 3 source code w/o nonlocals or nested function definitions
     - `jvm` - output JVM bytecode formatted for the Krakatau assembler
     - `cil` - output CIL bytecode formatted for the Mono ilasm assembler
-    - `wasm` - output WASM as plaintext in WAT format (WIP)
+    - `wasm` - output WASM as plaintext in WAT format
 
 ## Differences from the reference implementation:
 
@@ -110,8 +112,6 @@ The `demo_cil.sh` script is a useful utility to compile and run files with the C
 
 ## WASM Backend Notes:
 
-This is WIP, not all features are supported (the binary tree example itself actually does not work, but you can try another one). 
-
 The WASM backend for this compiler outputs WASM in plaintext `.wat` format which can be converted to `.wasm` using `wat2wasm`:
 1. Use this compiler to generate plaintext WebAssembly
     - Format:  `python3 main.py --mode wasm <input file> <output dir>`
@@ -126,26 +126,22 @@ The WASM backend for this compiler outputs WASM in plaintext `.wat` format which
 The `demo_wasm.sh` script is a useful utility to compile and run files with the WASM backend with a single command (provide the path to the input source file as an argument). 
 - To run the same example as above, run `./demo_wasm.sh tests/runtime/binary_tree.py`
 
-### WASM Backend - Supported Features:
-- int, bool, string, list
-- most operators
-- assignment
-- control flow
-- stdlib: print, len, and assert
-- globals
+The `wasm.js` file contains all the runtime support needed to run the WASM generated by this compiler. This backend was designed was to minimize runtime JavaScript dependencies, so the only imported functions are for assertions and printing strings/integers/booleans.
 
 ### WASM Backend - Unsupported Features:
-- nonlocal referencing function param
-- stdlib: input (node.js does not have synchronous I/O out of the box so this is difficult)
+- `input` stdlib function (node.js does not have synchronous I/O out of the box so this is difficult)
 
 ### WASM Backend - Memory Format, Safety, and Management:
 
 - strings (utf-8) - first 4 bytes for length, followed by 1 byte for each character
 - lists - first 4 bytes for length, followed by 8 bytes for each element
 - ints - i64
 - pointers (objects, strings, lists) - i32, where `None` is 0
+- objects - first 8 bytes for vtable offset, followed by 8 bytes for each attribute, followed by 8 bytes for each method index. inherited attribute/method positions are same as parent.
+
+Strings, lists, objects, and refs holding nonlocals are stored in the heap, aligned to 8 bytes. Right now, memory does not get freed/garbage collected once it is allocated, so large programs may run out of memory.
 
-Strings, lists, objects, and refs holding nonlocals are stored in the heap, aligned to 8 bytes. Right now, memory does not get freed/garbage collected once it is allocated. To provide memory safety, string/list indexing have bounds checking and list operations have a null-check, which crashes the program with a generic "unreachable" instruction.
+To provide memory safety, string/list indexing have bounds checking and list operations have a null-check, which crashes the program with a generic "unreachable" instruction.
 
 ## FAQ
 

compiler/astnodes/typedvar.py

@@ -21,8 +21,3 @@ def toJSON(self, dump_location=True):
         d["identifier"] = self.identifier.toJSON(dump_location)
         d["type"] = self.type.toJSON(dump_location)
         return d
-
-    def getWasmParam(self, paramIdx, funcType):
-        isRef = paramIdx in funcType.refParams
-        t = "i32" if isRef else self.t.getWasmName()
-        return f"(param ${self.identifier.name} {t})"

compiler/types/functype.py

@@ -50,6 +50,20 @@ def getJavaSignature(self) -> str:
             params.append(sig)
         return "({}){}".format("".join(params), r)
 
+    def getWasmSignature(self, names=None) -> str:
+        params = []
+        for i in range(len(self.parameters)):
+            p = self.parameters[i]
+            paramName = ("$" + names[i]) if names else ""
+            if i in self.refParams and isinstance(p, ClassValueType):
+                sig = f"(param {paramName} i32)"
+            else:
+                sig = f"(param {paramName} {p.getWasmName()})"
+            params.append(sig)
+        params = " ".join(params)
+        result = "" if self.returnType.isNone() else f" (result {self.returnType.getWasmName()})"
+        return params + result
+
     def methodEquals(self, other):
         if isinstance(other, FuncType) and len(self.parameters) > 0 and len(other.parameters) > 0:
             return self.parameters[1:] == other.parameters[1:] and self.returnType == other.returnType

compiler/wasm_backend.py

@@ -34,13 +34,9 @@ def _else(self):
         self.newLine(f"(else")
         self.indent()
 
-    def func(self, name: str, params: List[str] = [], resType=None) -> Builder:
+    def func(self, name: str, sig: str="") -> Builder:
         # return new block for declaring extra locals
-        params = " ".join(params)
-        result = ""
-        if resType is not None:
-            result = f" (result {resType})"
-        self.newLine(f"(func ${name} {params}{result}")
+        self.newLine(f"(func ${name} {sig}")
         self.indent()
         return self.newBlock()
 
@@ -231,18 +227,16 @@ def FuncDef(self, node: FuncDef):
         self.funcDefHelper(node, node.name.name)
 
     def funcDefHelper(self, node: FuncDef, name: str):
-        params = []
-        for i in range(len(node.params)):
-            params.append(node.params[i].getWasmParam(i, node.type))
         self.returnType = node.type.returnType
         ret = None if self.returnType.isNone() else self.returnType.getWasmName()
-        self.locals = self.builder.func(name, params, ret)
+        paramNames = [x.identifier.name for x in node.params]
+        self.locals = self.builder.func(name, node.type.getWasmSignature(paramNames))
         for d in node.declarations:
             self.visit(d)
         self.visitStmtList(node.statements)
         # implicitly return None if possible
         if ret is not None and not isinstance(node.statements[-1], ReturnStmt):
-            if self.returnType.getWasmName() == "i32":
+            if self.returnType.getWasmName() == "i32" and not self.returnType.isSpecialType():
                 self.instr("i32.const 0")
             else:
                 self.instr("unreachable")
@@ -532,9 +526,7 @@ def MethodCallExpr(self, node: MethodCallExpr):
         self.instr("call $log_int")
         self.getLocal(temp)
 
-        params = " ".join([f"(param {t.getWasmName()})" for t in funcType.parameters])
-        result = "" if funcType.returnType.isNone() else f"(result {funcType.returnType.getWasmName()})"
-        self.instr(f"call_indirect {params} {result}")
+        self.instr(f"call_indirect {funcType.getWasmSignature()}")
 
         if funcType.returnType.isNone():
             self.NoneLiteral(None)  # push null for void return

test.py

@@ -12,12 +12,12 @@
 error_flags = {"error", "Error", "Exception", "exception", "Expected", "expected", "failed"}
 
 def run_all_tests():
-    # run_parse_tests()
-    # run_typecheck_tests()
-    # run_python_backend_tests()
-    # run_closure_tests()
-    # run_jvm_tests()
-    # run_cil_tests()
+    run_parse_tests()
+    run_typecheck_tests()
+    run_python_backend_tests()
+    run_closure_tests()
+    run_jvm_tests()
+    run_cil_tests()
     run_wasm_tests()
 
 def run_parse_tests():