Multi-module compilation preparation

[nattress]

Add a new command-line switch to ILC: /multifile. When specified, only
the assemblies passed as input will have methods compiled. Referenced
types / methods from other assemblies are not compiled into the output
object file. This switch is most likely temporary as we hone our
compilation story and implementation.

Each managed module adds pointers to the start and end of a module
global data header to a custom section of the object file, .modules$I.
These entries are merged (on Windows, OSX / Linux needs a tweak to CLI
first) at link time producing a list of module headers.

In StartupCodeHelpers.cs, initialize global tables from each module
using the list of pointers that was written to .modules$I. This data is
discovered through two exports (__modules_a and __modules_z) which
through linker section merging, are placed either side of the module
header pointers in the final binary.

Alter interface dispatch to store its dispatch map table as an
ArrayOfEmbeddedDataNode and place it in the module header list. This
allows each module's EETypes to continue using index-based lookup of
dispatch maps.

Add a new field to EETypes which points at a ModuleInfo* through an
indirection cell. This indirection cell is filled in at runtime
initialization and allows a type to find its dispatch map table.

ModuleHeaderSection.cs|h files define the section headers currently
supported

Moved the module info lookup out of the bootstrapping code and into the
runtime

[jkotas]

We should have only one row per section - that has both start and end.

Something like:

struct ModuleInfoRow {
    int32 SectionID;
    int32 Flags; // Eventually, these flags can say whether start/end are actual pointers, RVAs, GP-relative, ...
    void * Start;
    void * End;
};

[nattress]

Yeah that's better. The Start / End pattern felt awkward.

[MichalStrehovsky]

Do we actually need to compare anything but the name? A readonly blob with same name but different content sounds like an internal compiler error. We would end up with the same symbol for two different blobs.

[nattress]

You're right - I'll alter this. We were failing to compare Tuple objects as the same when the names in each were identical. I got a bit overly enthusiastic and compared all the contents.

[MichalStrehovsky]

@nattress It feels like the compilation group management is split into multiple places (some stuff in Compilation.cs, some in NodeFactory.cs).

Would it make sense to introduce a new abstract class CompilationGroup that would have two classes deriving from it (SingleFileCompilationGroup and MultifileCompilationGroup)?

CompilationGroup would provide methods to determine whether a type/method is part of it. Maybe it could also provide compilation roots, etc.

We would new up the right thing in Compilation and pass the instance to NodeFactory. Then NodeFactory won't have to care about isMultifile at all.

I'm worried that the compilation group logic will get complex over time (e.g. if we introduce a concept of TOC files to deal with generics). People tend to just add more ifs into various places to handle the complexities and I would prefer to have some design that guides them to do the right thing.

[MichalStrehovsky]

I'm conflicted about the location of this file. While it's nice to have both .cs and .h in a single places, we only put managed stuff into src/Common.

Everything native lives under src/Native, and we maintain that split even for common things like EEType flag definitions.

[jkotas]

I agree - we should follow the same convention as what we use for other .h/.cs files.

[nattress]

I'll move it and use comments to hopefully avoid future disaster. Maybe we ought to look at tooling to keep these kinds of splits in sync. We'll hopefully remember to edit both places but other contributors may not. We have cspp a la AsmOffsets but it muddies the build imo.

[jkotas]

This should be eventually reconciled with ReadyToRunSectionType from https://github.com/dotnet/coreclr/blob/master/src/inc/readytorun.h. Could you please add comment about it?

Looking at what is there today - we should start this set of IDs at 200.

[nattress]

Done