Install compilers automatically.

[veelo]

RFC.

Observation: toolchainRequirements enable historically reproducible builds, by pinning the used compiler version. It also aides in controlled preparation of a code base for a compiler upgrade.

The next step would be that dub, if it cannot find a compiler that satisfies the buildRequirements, automatically downloads, installs and uses one that does. This would have great advantages especially in a commercial setting:

• Developers need not concern themselves with compiler versions. Compiler upgrades can be prepared by one individual (who resolves deprecation warnings) and after that change is pushed everyone proceeds without noticing.

• When checking out older revisions to determine the introduction of a regression, compilers are automatically downgraded and compilation just works without errors or deprecation warnings.

• Build farms and CI nodes need no maintenance.

I have considered various approaches:

Using preBuildCommands to execute install.sh

This currently does not work because changing the PATH does not propagate into dub's environment. Besides, it seems wrong to rely on a shell script for something that is one of dub's core businesses: downloading and installation. On top op of that are security concerns.

Introducing buildDependencies in recipes

This would probably mean publishing dub packages for all compilers and versions. Possibly a whole new package format would be required to support binary distributions. It could also support other binaries necessary for building, such as dpp. With this we would be merging into the territory of OS package managers.

Just do what I mean

The above two solutions have the downside that information is repeated that can already be represented by toolchainRequirements, and possibly can conflict. Better would be to look for an existing installation as per current procedures. When this fails, instead of giving up with an error message, decide on a compiler to download and install, and continue building with this. This would require the following additional steps:

1. Consider the intersection of the toolchainRequirements of the current package and of its dependencies.
2. Filter this through the optional --compiler= argument or defaultCompiler from settings.json.
3. Select the newest, most relevant compiler in this set.
4. Mimic install.sh functionality (download, verify, unpack). An appropriate path for the compiler cache would be ~/.dub/compilers or %LOCALAPPDATA%\dub\compilers, i.e., alongside packages.
5. Retry with implicit --compiler=<path>.

Examples

Switch to ldc without consulting dlang.org/download:

dub build --compiler=ldc

Test the master branch:

dub build --compiler=dmd-nightly

Any existing compiler version will just work:

dub build --compiler=dmd-2.088

Use ldc-1.22:

    "toolchainRequirements": {
        "frontend": "==2.092",
        "dmd": "no",
        "gdc": "no"
    }

dub build

Observations

• As long as a satisfying compiler version is locally available, no new downloads are initiated (except for -latest and -nightly suffixes).
• The compiler in PATH takes precedence (as long as it satisfies toolchainRequirements).
• When no satisfying versions are found in the usual places, but multiple versions satisfy in the compiler cache, we would probably want to select the latest.
• Error on conflicting --compiler= and toochainRequirements.

[WebFreak001]

I disagree about this being in dub, I think this would fit very well in dvm however.

[Geod24]

Same, and we had this discussion in the past. Closing as WONTFIX.