Provide two-stage interface to front module

[jclark]

In order to support #438, we need to provide a two stage interface to the front module:

1. In the first stage, it provides a list of the module’s imports. In this stage, it runs the parser/tokenizer only up till the end of the import declarations. It doesn’t yet build a complete AST.
2. In the second stage, it generates a BIR module; it is provided with the public interface to each of the modules that it imports. In this stage , it builds the complete AST.

[jclark]

I envisage a public class ScannedModule and a function

public function scanModule(SourcePart[] sourceParts, bir:ModuleId id) returns ScannedModule|err:Any|io:Error;

scanModule would perform parsing up to the end of the imports.

Then ScannedModule provides

• a getImports method to provide the required imports, and
• a method to build a bir:Module, which would take a t:Env, plus something TBD that provides access to the results of compilation of its imports.

[jclark]

This will need #443.

[manuranga]

What should the front:ScannedModule object keep in order to later build the bir:Module

opt1) keep s:SourceFile[], re-parse imports and throw them away
opt2) keep s:SourceFile[] and set of offsets - need API change form Tokenizer
opt3) keep Tokenizer[] - Tokenizer needs to be public

I feel like opt1 is the simplest.

[jclark]

I would wrap Tokenizer in a class that provides a parse method that continues the parse.

[jclark]

Public function creates a s:Parser object from an array of lines and a FilePath; Parser class provides parseImports and parse methods. These replaces parseModulePart. Parser contains private fields with Tokenizer plus a ModulePart. You can use Parser instead of LoadedSourcePart.

[jclark]

There's a problem with what I suggested here #451 (comment) in that eventually we will want ScannedModule to be readonly (to facilitate doing stage 2 in parallel). Otherwise ScannedModule will need to be an isolated object, which is harder. We can address this as a follow on issue.