Enso Compiler 2.0 (Alpha 2)

[iamrecursion]

Summary

The current interpreter treats the entirety of the static analysis and compilation phase as an afterthought, but this is not a state of affairs that can be maintained if we want to evolve the language.

In order to support the new syntax and more sophisticated language features, including typechecking in the future, we need to have a proper 'compiler' front end to the Enso interpreter. This, along with the interpreter and the language server, is what makes up the totality of the Enso Engine. The compiler is intended to support the following capabilities:

• Static analysis of Enso programs
• Transformation of Enso programs (e.g. desugaring and optimisation)
• Typechecking and other static analysis processes.

The initial compiler implementation is based purely on the existing machinery, but it is organised in a fashion more amenable to evolution and further development.

Value

The first phase of the development of the compiler component aims to provide all the facilities necessary for integrating the new syntax with the interpreter. This will start off with a limited subset of the syntax, and then evolve the featureset to support all the necessary capabilities, giving us a usable form of the new syntax as soon as is possible.

Specification

The following elements are presented in order as a plan for integrating the new parser and new compiler front-end with the existing interpreter. After the completion of each of these steps the interpreter should be in a working state.

• Implement a translation from AST (the parser AST type) to AstExpression (luna-package fails to compile on Windows #276).
• Thread the SourceLocation information from core through the compiler transformations (Class fields have incorrect spans #290).
• Implement the new core language (Implement Requests API for Searcher #439, luna/luna#802, luna/luna#704, luna/luna#703).
• Initial cleanup and preparation for the stopgap compiler (Implement a Bytes Type #490, luna/luna#694, luna/luna#693).
• Implement the desugaring of operators to functions (A temporary fix for the WS library #289).
• Implement Scope Identification and Alias Analysis on the IR (Find a better name for LQuery #287, luna/luna#691).
• Implement demand analysis to determine where to force and suspend arguments with regards to laziness (Being able to convert JSON stream to JSON definition/object #288).
• Write a local data-flow analysis pass on the IR (Hack previous parser behaviour for AccSection #311).
• Implement an error-reporting pass over the IR (Port appearance and inference when connecting nodes #280).
• Add support for and format string literals (Luna REPL #366).
• Implement a Lambda consolidation pass (Implement a File Access Library #495).
• Pay some technical debt in the compiler (Specify Language Server API #702).
• Enhance support for literals (Starting Subset of LSP - Part 5 #717).
• Implement desugaring the various kinds of syntax sugar (Implement a Enso Vector Type #496, luna/luna#795, luna/luna#794).
• Update the IR Identifier Format (Implement Multiclient Write Lock #714).
• Handle pass dependencies properly (Implement Value Caching #782).
• Add support for nested patterns (Report errors for duplicate definitions #294).
• Handle type signatures (Clean the initial Rust AST implementation #758).

It should be noted that these steps provide gradual support for syntactic sugar. This is in order to ensure that analysis passes do not have to cope with extraneous nodes from Core.Sugar, and can instead deal (where relevant) only with Core.Primitive structures.

Acceptance Criteria & Test Cases

• The above-listed specification points have been completed and have all landed on master.
• They are accompanied by rigorous tests and benchmarks to ensure that there are no functionality regressions and that performance is acceptable.