Delete the makefile build system

[alexcrichton]

This PR deletes the makefile build system in favor of the rustbuild build system. The beta has now been branched so 1.16 will continue to be buildable from the makefiles, but going forward 1.17 will only be buildable with rustbuild.

Rustbuild has been the default build system since 1.15.0 and the makefiles were proposed for deletion at this time back in November of last year.

And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!

[rust-highfive]

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

[alexcrichton]

r? @brson

[steveklabnik]

And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!

I know I already did a 🎊 reaction, but this deserves many more.

🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊 🎊

[petrochenkov]

Please fix #36385 before removing the old build system, this is a blocker.

I planned to do it myself, but I won't be able to do it before this PR is merged.

[brson]

The patch looks fine to me.

@petrochenkov #36385 seems like both a difficult problem to solve and one that isn't causing a great deal of pain (Rust devs are still developing and nobody has fixed it). Should we really stop and tackle that right now?

[petrochenkov]

@brson
@alexcrichton already suggested a solution, is well familiar with the build and is a master of hacking up kinda working systems quickly.
My estimation for the simplest (but very helpful) solution with timestamps (one timestamp for a whole test group, aux files may be not considered at all) is a couple of hours.

Rust devs are still developing

That's because the old build systems still exists! Otherwise, they probably don't develop on Windows, or irresponsibly submit untested PRs.

[alexcrichton]

Let's just put that issue to rest, a commit's now attached to this PR. I was hoping to give someone else a chance to get their feet wet with the build system and/or compiletest, but oh well.

[alexcrichton]

@petrochenkov FWIW I've found the way you've approached #36385 as somewhat frustrating and unproductive. You've repeatedly brought it up and as you've notice I tried to outline clear solutions. There's been ample time (2 whole release cycles) where the deprecation of the makefiles has been announced, so this PR shouldn't come as a surprise.

In the future it'd be great to take a more proactive and less passive-aggressive stance towards solving these sorts of issues.

[ishitatsuyuki]

There's many critical issues breaking shared linking build, like #39215.

Please make a list to track them.

[pnkfelix]

@alexcrichton

Let's just put that issue to rest, a commit's now attached to this PR. I was hoping to give someone else a chance to get their feet wet with the build system and/or compiletest, but oh well.

Wait, why isn't that commit a separate PR?

Surely that would be a step forward to let people concerned about the makefiles going away verify that they can use rustbuild before landing this whole PR too?

---

Update: I guess the idea was to ensure that this PR didn't land without ensuring that the test result caching didn't land as well? (Too bad github doesn't have a direct way to express such dependencies between PR's, as far as I know...)

[alexcrichton]

@pnkfelix well I don't really mind any particular organization, I'd just like to land this!

[brson]

@bors r+

[bors]

📌 Commit 39321aa has been approved by brson

[petrochenkov]

@pnkfelix
I've cherry-picked 39321aa and tested it locally (report).

[ishitatsuyuki]

What are you guys doing?
Rust 1.15 came to stable this morning, you guys have already annoyed package maintainers by releasing a **** flawed buildsystem.

Please, delay the removal until the (dynamic) linking issues are fixed.

[alexcrichton]

@ishitatsuyuki as explained in the PR this is a long time coming. We've still got 3 months before this hits stable anyway to fix more issues as they arise.

Rewriting a build system is almost guaranteed to not be bug-for-bug compatible, so many quirks and bugs of the makefiles are not carried over (either accidentally or intentionally). Issues like #39215 were barely ever supported by the makefiles (if at all, only by luck) and aren't really a great reason to completely block this in my opinion.

These bug fixes tend to just be a line here or there, and PRs are always welcome!

[ishitatsuyuki]

@alexcrichton I'm concerned about breaking distro packaging entirely by only testing on rustup.

A buildsystem definitely should not be flawed (flawed by default is annoying enough though), and we will completely lose dynamic linking if you removed the old one.

Be patient. You didn't tend to break old rust code, but you're going to break up-to-date distro build script now. (By the way, this would also break the rust-git package in AUR)

[ishitatsuyuki]

Issue unblocked, I'm okay with the merge.

[bstrie]

As much as I'd love to forge ahead with Cargo, I agree with @ishitatsuyuki that we should make explicit overtures to Rust packagers to make sure they have everything they need to exist in a world without makefiles.

[bors]

☔️ The latest upstream changes (presumably #39463) made this pull request unmergeable. Please resolve the merge conflicts.

[alexcrichton]

@bors: r=brson

[bors]

📌 Commit 2d51781 has been approved by brson

[tamird]

how come these files are no longer needed? where do valgrind suppressions come from these days? I grepped around and couldn't find it.

[petrochenkov]

@alexcrichton
The test caching commit has suddenly disappeared after rebase.

[bors]

🔒 Merge conflict

[vi]

@cuviper, As far as I know there are projects for making Rust bootstrappable without repeating it's entire history from Ocaml across all intermediate releases. Like a simplified non-verifying non-optimizing Rust interpreter or compiler written in C.

[frewsxcv]

@alexcrichton Does this fail from my rollup seem relevant to your changes here?

[alexcrichton]

@frewsxcv nah that's #38878

[Nashenas88]

Shouldn't the main README.md also be updated? It still says to use make.

[alexcrichton]

Nah we still have a bare-bones makefile at least for familiarity, so that documentation is still correct. We may wish to feature x.py more prominently though!

[J-F-Liu]

It will also be good to split current compiler into several crates, e.g. lexer, parser, hir, mir etc. This is a win-win situation.

• The compiler components can be reused in other projects.
• Crates can compete with each other which results in a better compiler.

[steveklabnik]

@J-F-Liu this is already true, check the subdirectories of https://github.com/rust-lang/rust/tree/master/src, for example, https://github.com/rust-lang/rust/blob/master/src/librustc_mir/Cargo.toml is for MIR.