Crate system

[fmease]

The package system of lushui.

• Path hanger extern instead of crate declarations #49
• Type-checking and interpreting
  • CrateIndex should be renamed since with a crate system, it might get confused with numeric identifiers of crates
    Suggestions: DeclarationIndex, DefinitionIndex, ModuleLevelIndex
  • The issue arises that the indices mentioned above are no longer globally unique if we define them to be crate-local
    (which is what I intended them to be). So we should have a distinction between local and global indices or just use
    global indices which consist of the index of the crate (the new CrateIndex) and a crate-local index (maybe DeclarationIndex) named GlobalDeclarationIndex (maybe)
  • a big question is how we restructure our code to handle multiple crates: Contrary to Rust, we cannot (yet) compile
    crates separately since we cannot compile at all right now (no working byte code and no working compiler yet),
    which means crates are not compilation units/boundaries (yet). Should we have multiple CrateScopes and put the into
    a Session or ProjectScope? The way we register foreign and inherent bindings needs to change, too.
  • new CLI-option --link <path> which either links to a lushui file (in which case the dependency is a "single file/simple crate"
    and we assign it some default meta data i.e. its name (name of the file), its version (0.0.0?)) or to a folder with a config file in
    it. We somehow need a way/design to refer to known locations (as env variables prob.) for example the equivalent of target/ or node_modules/ or the shared dependency folder $HOME/.lushui/shared-deps/ (…)
• Package manifest file support
• Compilation making it the a unit of compilation (memory reduction) (only once we actually have a compilation mechanism, Bytecode backend #4)
• Make the standard library core a package
  • link it by default for simple/single file crates with the option to unlink it --unlink-core
  • place it into libraries/core/ libs/core/ in this repository
  • rename src/ to compiler/

[fmease]

Implementation sketch:

pub struct Session {
    options: Options, // compilation flags coming from CLI or similar: unstable options, etc.
    crates: IndexVec<CrateIndex, Crate>, // `Crate` formerly `CrateScope`, now also contains crate metadata (version, …)
    map: SourceMap, // maybe
    errors: RefCell<Diagnostics>, // maybe, depends on our error handling strategy here
    warnings: RefCell<Diagnostics>, // leave out for now
    lowering: LoweringSession, // or similar: parallel lexing, parsing and lowering: contains queue of SourceFiles
}

Also, I'd like to have methods/"queries" for Session which abstract a bit over the compiler passes so we can write
more succinct code in main.rs like Session::front_end(…), Session::type_check(…), Session::document(…).

Additionally, I'd like to see a more centralized error handling infrastructure like an ErrorHandler/Communicator/Reporter inside of that session that controls the output of diagnostics (errors and maybe also warnings). This becomes increasingly important when we decide to implement allowing and denying of lints/warnings (those can update settings of the Reporter) and
would like to support different error formats like short (for the use in a REPL or an IDE) or JSON. Kinda unrelated to this issue but
I still regard it as useful since I'd like to architect this transition a bit more.

[fmease]

Since crates won't be compilation boundaries for now, crates are just a new kind of entity for the name resolver (not totally since
we are going to use separate classes of indices for separate crates).

• extern.core: the crate is core (with a crate index) but the actual entity is a module, the root module, also named core (maybe we should change this duplicated(?) representation unless it complicates the name resolver).
  So its index might be: Index::Declaration { crate: CrateIndex(0u32), local: LocalDeclarationIndex(0u32) }
• extern.core.nat: Its index may be Index::Declaration { crate: CrateIndex(0u32), local: LocalDeclarationIndex(1u32) }

There's a good chance that the index examples above are very error-prone since parts of the name resolver might access and use
the local index w/o checking the crate. I need to experiment with this. Of course, we could define the index as an u64 (or u128) and use some bytes of it for the crate index and some for the local part (with methods to abstract over this).