Add build commands for host dmd

[andralex]

This automatically downloads and installs the dmd compiler needed to boostrap the dmd build. cc @yebblies

[yebblies]

I think this is too much complication for what it gives us.

• None of the targets should depend on HOST_DC
• Passing -conf is not always desired

I don't really understand what the rest does but it should be ok so long as it's only every manually invoked.

[braddr]

Dear god no. Do not have automatic downloading of anything please. I'm fine with detection of missing components and instructions of a manual step to take for installation, but this is horrid. It's also broken. The auto-tester is already properly setup and this change is causing downloads to occur are absolutely not needed.

[jacob-carlborg]

This is only for Posix, not Windows.

[andralex]

Whoa. Folks. Let's leave emotion out of this, "dear god no" etc hardly qualifies as a convincing argument.

@yebblies: The benefit here is that the setup is more user friendly to folks who don't have a previous version of dmd installed, or those who set up dmd to be the one under development. I belong to the latter category, so building dmd is quite difficult for me - e.g. make clean ruins all my prospects etc. I agree there should be a way for folks who already have dmd to just use it, so I'll update this. Specifying the configuration is necessary because there might be a ~/dmd.conf set up to use the development dmd (or may be corrupt or whatever). Self reliance and robustness is what we're looking for here.

@braddr: Could you please substantiate what the matter is with downloads. There is precedent in Phobos, a specific dmd is downloaded to build dub. After a few grudges I've learned to appreciate it. cc @MartinNowak

One more note: these few lines obviate the proposed use of dvm - another external tool with its own world.

[andralex]

@braddr: I take it you'd be favorable with a solution that only downloads anything if $(HOST_DC) does not exist?

[MartinNowak]

Please use http://downloads.dlang.org/releases/2.x/$(HOST_DMD_VER)/dmd.$(HOST_DMD_VER).$(OS).zip.

[MartinNowak]

It makes a lot of sense to add bootstrapping to our build system.
Pinning the version of the host compiler also gives a more controlled build environment, e.g. avoids adding changes to ddmd that won't build with an older version of dmd.

One more note: these few lines obviate the proposed use of dvm - another external tool with its own world.

Yeah, let's please not go that path. Last attempt at deploying DVM for something useful was a disaster (travis-ci/travis-ci#730 (comment)). It's also a 5KLOC tool with a 15KLOC dependency on mambo.

@braddr: I take it you'd be favorable with a solution that only downloads anything if $(HOST_DC) does not exist?

Makes sense, e.g. the release build environment already comes with it's own host compiler.

[braddr]

The auto-tester is going to in the not terribly distant future disallow network access during a build. All prerequisites are required to be installed pre-build, not during. Validation of the build environment is a good practice. Construction of it is not. Providing functionality to assist with construction is fine, just not as a part of the build itself.

For any continuous integration system, they all aim to start with a clean, specifically constructed, environment, isolated from any past environments -- with varying levels of success. Having them downloading anything during the build step quickly multiplies. The auto-tester I run does about 1200 builds a day. Multiply that by 10 megs per build. I think it's safe to say that 12 gigs of redundant downloads is the wrong thing to do.

[andralex]

All: thanks for the input. I'll change things to make auto-bootstrapping opt-in. Stay tuned.

[yebblies]

The benefit here is that the setup is more user friendly to folks who don't have a previous version of dmd installed, or those who set up dmd to be the one under development. I belong to the latter category, so building dmd is quite difficult for me - e.g. make clean ruins all my prospects etc.

I would advise against using the just-built dmd for exactly this reason. That's what I did, up until 2.067.

I agree there should be a way for folks who already have dmd to just use it, so I'll update this. Specifying the configuration is necessary because there might be a ~/dmd.conf set up to use the development dmd (or may be corrupt or whatever).

If your installed dmd is 2.067, you might want to be using ~/dmd.conf.

I agree with Brad that any fully automatic download would be a bad idea. Manually invoking make download-dmd should be much better.

[MartinNowak]

If your installed dmd is 2.067, you might want to be using ~/dmd.conf.

Right, but you don't want to use it with the downloaded DMD.

[yebblies]

Right, but you don't want to use it with the downloaded DMD.

My understanding is that the line

+HOST_DC_RUN=$(HOST_DC) -conf=$(dir $(HOST_DC))dmd.conf

forces explicit conf on every setup.

[MartinNowak]

You can override HOST_DC_RUN.

[andralex]

OK, so now AUTO_BOOTSTRAP must be used explicitly for automatically taking care of old dmd.

[MartinNowak]

Seems like you should still allow env variables, i.e. HOST_DC?=dmd.

[andralex]

OK

[andralex]

@MartinNowak: dunn. Destroy.

[jacob-carlborg]

How about squashing the commits.

[andralex]

@jacob-carlborg squashed

All: ping

[jacob-carlborg]

It looks like you haven't addressed one of Martin's comments: #4645 (comment).

[jacob-carlborg]

Can we do this for Windows as well? http://stackoverflow.com/questions/4619088/windows-batch-file-file-download-from-a-url

[MartinNowak]

Auto-merge toggled on

[MartinNowak]

The best way to download a file on Windows is Powershell.
https://gist.github.com/MartinNowak/8270666#file-setup-ps1-L19

[MartinNowak]

It looks like you haven't addressed one of Martin's comments: #4645 (comment).

We can fix the url next time we update the compiler.

[yebblies]

This has broken specifying HOST_DC with an explicit -conf switch.