Tracking issue for nightly-1.29 branch

[thepowersgang]

Dumping ground for progress with the 1.29 upgrade branch (targeting rustc 1.29 stable, with intent to not break support for 1.19)

[thepowersgang]

Blocked on type inferrence bugs when processing "compiler_builtins"::float::cmp::cmp.

The float module is full of complex generics and trait resolution interactions with inferrence.

[thepowersgang]

Current revision (ae0027a) builds 1.19 rustc entirely (cargo fails due to missing f516811)

Fixes from getting 1.19 working again have advanced 1.29 past float::cmp to float::div, which is experiencing inference errors.

[thepowersgang]

As of f4c39e9, libcompiler_builtins is passing typecheck (for now). Current blocker is a potential issue in MIR lowering or other type annotations.

[thepowersgang]

As of 883cf34, libcompiler_builtins is compiling (as is libstd_unicode).

Current blocker is the new allocator API, still to be investigated.

[bjorn3]

Just posting some links regarding the allocator api which I think may be helpful:

• https://github.com/rust-lang/rust/blob/master/src/librustc_allocator
• https://github.com/rust-lang/rust/blob/master/src/librustc_codegen_llvm/allocator.rs
• https://github.com/rust-lang/rust/blob/059e6a6f57f4e80d527a3cd8a8afe7f51f01af8e/src/librustc_codegen_ssa/base.rs#L630-L663

[thepowersgang]

Thanks, but I more meant some of the additions in marking where the allocator comes from, in particular the #[allocator] attribute in liballoc/alloc.rs. Currently investigating

[bjorn3]

#[global_allocator] is expanded in https://github.com/rust-lang/rust/blob/master/src/librustc_allocator/expand.rs

#[allocator] is matched at https://github.com/rust-lang/rust/blob/master/src/librustc_typeck/collect.rs#L2276 setting CodegenFnAttrFlags::ALLOCATOR which is used at https://github.com/rust-lang/rust/blob/master/src/librustc_codegen_llvm/attributes.rs#L194

[thepowersgang]

Oh good, I don't need to worry about the #[allocator] :)

[thepowersgang]

Update before I vanish for a few days: The allocator traits (Boxer/Placer) have been removed, so handling of the box keyword needs to be updated.

[bjorn3]

https://github.com/rust-lang/rust/blob/f1d61837d1cf058cfbd0902b0bf79a2657b81187/src/liballoc/alloc.rs#L182-L199

[thepowersgang]

Revision 2744a49 replaced the box and <- operator handling in HIR typecheck with just support for box returning an instance of owned_box. This works for now, but will need cleanup in the future (e.g. disabling parsing of the other forms, and handling in MIR lowering).

Currently working on "Match Ergonomics" support.

[thepowersgang]

And now (2408628) up to C generation for libstd. Running into C compilation errors caused by box_free changing its signature, combined with #[repr(transparent)] currently being ignored.

[thepowersgang]

Revision 6fb11a5 successfully generated a running "hello world" executable using libstd :)

And 8a1ed07 fixed the test harness generation such that Travis can pass.

[thepowersgang]

Progress on building rustc (as of d1ea840): Currently "blocked" on one of librustc's macros, which breaks the current macro_rules evaluator.

Open question: Where does the getopts library in the sysroot come from? librustc uses it without specify a version in Cargo.toml

[bjorn3]

getopts is a dependency of test.

[thepowersgang]

So it is, but it's a crates.io dependency, so doesn't actually get built as part of the "sysroot" in the current mrustc build system... looks like that needs fixing.

[thepowersgang]

Still on librustc, macro_rules updated to support the macros used, and now up to solving typecheck bugs. (Revision f907e91)

[thepowersgang]

Revision ebe84a8, still on librustc but many typecheck holes and bugs are now fixed. Match ergonomics are being annoying though.

[thepowersgang]

librustc is now building :) (as of 8777573)

Up to crate 69/83 in rustc, rustc_traits which uses the 2018-style paths (#![feature(extern_prelude)])

[thepowersgang]

a473a4e (70/83) BUILDING rustc_mir v0.0.0 (71st crate)

Uses a bunch of new features, including do catch { ... } (stablised in later rustc versions as try { ... }) and macro (which I need to research, looks similar to a single-arm macro_rules)

[bjorn3]

macro supports multi arm syntax too. The main difference is that items are hygienic with it, unlike macro_rules!.

[thepowersgang]

Working on ensuring that the branch still works for 1.19, revision fcb473d can compile libstd 1.19 but fails in type inference for liblog (as part of rustc)

Getting inference correct is always going to be a problem, I fear.

[thepowersgang]

Update for 3bf9a10

• 1.19.0: rustc builds and runs, cargo fails on libgit2 0.6.6 (part of 1.19 cargo) - incorrect type inference.
• 1.29.0: rustc fails on librustc, again with incorrect inference

[eddyp]

Update for 3bf9a10

* 1.19.0: rustc builds and runs, cargo fails on libgit2 0.6.6 (part of 1.19 cargo) - incorrect type inference.

* 1.29.0: rustc fails on librustc, again with incorrect inference

@thepowersgang: Was 1.29.0 chosen by some criteria like "I know 1.30 has that feature X which we don't support, so no use in trying", or is it something like you started some time ago on 1.29.0 and will have to resync later?

[thepowersgang]

@eddyp 1.29 was nightly when the branch started, so I chose it as the new target version.

[thepowersgang]

Update for 406fe84

librustc 1.29 is now passing, as is librustc_mir (although, both take over 10 mins each to compile). Looking like progress will pause librustc_lints which makes an incorrect inference choice (that isn't caught during validation) with a match foo.as_ref() { "..." => ..., ... } picking &OsStr instead of &str

1.19 is still blocking in libgit2 - difficult to make that code compile without breaking other crates.

[thepowersgang]

Annoying excerpt from librustc_typeck, where the code appears to play fast and loose with match ergonomics... or there's a coercion point that I missed. (librustc_typeck/check/method/suggest.rs line 260)

                            hir::ExprKind::Path(ref qpath) => {  // local binding
                                if let &hir::QPath::Resolved(_, ref path) = &qpath {

[thepowersgang]

Another quirk, which I originally solved by makign ref auto-deref to make &T from &&T

rustc-1.29.0-src/src/librustc_borrowck/borrowck/mod.rs line 880 (for the Ok pattern)

                            match (self.tcx.sess.codemap().span_to_snippet(sp), &err.cmt.cat) {
...
                                (Ok(ref snippet), ref cat) => {
                                    let msg = &format!("consider making `{}` mutable", snippet);
                                    let suggestion = format!("mut {}", snippet);

                                    if let &Categorization::Deref(ref cmt, _) = cat {

[thepowersgang]

Update from 1f432b8 - The above change to match ergonomics worked, now hitting a name resolution problem around the treatment of declarative macros as being the same as macro_rules. They instead resolve names in the context of the defining module (not the using module).

[thepowersgang]

?Final update for the week(end), revision 92afcb3 can compile all of rustc's crates, but is tripping up with a duplicate symbol during final link.

[thepowersgang]

Update for 64933e0
1.29 rustc is now running, and librustc_codegen_llvm compiles (but doesn't link correctly, due to missing dylib support). This is blocking rustc from actually doing anything except showing its help.

Cargo not yet started, due to the use of the rustc-workspace-hack crate.

1.19 cargo still blocked on libgit2, quite a pain that one is. But rustc should still be compiling

[bjorn3]

(but doesn't link correctly, due to missing dylib support)

You can try making it a rlib, add all necessary rustc crates as dependencies. Then depend on it in rustc_driver and add a line like "llvm" => rustc_codegen_llvm::__rustc_codegen_backend, before https://github.com/rust-lang/rust/blob/17a9dc7513b9fea883dc9505f09f97c63d1d601b/src/librustc_driver/lib.rs#L260 as workaround.

[thepowersgang]

Or, I can add dylib support and avoid needing patches to the rustc source. That's the more correct (but slower) route

[snf]

Hi Mutabah, this is an amazing project and thanks for dedicating so much work to it. Do you have a list of incomplete tasks that need to be finished for compiling nightly-1.29?

[thepowersgang]

A short list of things that need to work

• rustc compiling and running successfully (I got it compiling recently, but then broke it again it seems)
• cargo doing the same (may be stuck on libgit2, same as 1.19 cargo)
• 1.19 support mainained (so rustc/cargo working for that version/mode too)

I'd have a more complete list, but my build box is currently offline.

[snf]

Thanks! I was thinking more about tasks that could be completed and contributed to mrustc. I'll have a look to see how far it gets.

[thepowersgang]

I've done some fixes over this weekend, and rustc (both 1.29 and 1.19) is building, but neither are working (1.19 fails with a duplicate symbol, 1.29 fails with an ICE).

1.19 cargo is still failing on the libgit2 inference bug, while 1.29 cargo is currently failing on a trait impl on a very-large array (2^32 elements)

[thepowersgang]

With a94abe4 rustc 1.19 is running again (can build libcore).

Working on fixing rustc 1.29 (still ICEs)

Once that works, plan is to get 1.29 cargo working. 1.19 cargo is half in the too-hard basket for now, as that inference error is REALLY hard to fix without breaking other code (I'm tempted to just do source patch, but that is very much cheating).

[thepowersgang]

As of 49a3ab2 rustc 1.29 can build libcore :)

Now working on a niggling MIR optimisation and then will see about getting cargo working (either fixing the array size deficiency, or the type-check bug - or just patch both of those out)

[thepowersgang]

Got distracted by doing the MIR optimisations when one broke rustc (and tests), got standalone_miri working for tests and finally fixed the broken optimisation today. rustc 1.29 can build libcore, but aborts in building libcompiler_builtins due to calling SSE functions that mrustc doesn't support.

[thepowersgang]

Through some fiddling with handling of as in typechecking, libgit2 is now compiling :)

Current blocker: libcargo 1.29 is tripping incorrect inference with map indexing :(

[thepowersgang]

Slow work, managed to get cargo 1.19 up to linking (it's not linking currently, doesn't seem to be including libstd and friends)

Cargo 1.29 still blocked on inference with indexing, might be related to index values being a coercion point (trying a build with that assumption).

[thepowersgang]

Cargo 1.29 now through the inference problems. Updates needed to proc macro derive (not passing attributes through to the proc macro) before it'll compile fully. Getting so close

[thepowersgang]

Cargo 1.29 is now through typechecking. Currently fails due to a closure being FnOnce instead of FnMut

[thepowersgang]

Both cargo and rustc are building 🎉

Currently working on getting the bootstrap framework working for 1.29 (have to first build libstd manually, then can run cargo on it to get a proper build)

[thepowersgang]

As of c1b32b6
Current blocker: rustc's bootstrap application tries to use serde_derive derives, but those are erroring out (rustc doesn't think they're derive macros...)

[thepowersgang]

Update: problem is caused by vtable disagreement between the mrustc-built rustc and a rustc-built serde_derive

[thepowersgang]

Worked around the vtable disagreement by building rustc with itself "manually" (invoking cargo on it).

Found some probable codegen errors showing up as possible non-deterministic symbol hashes.

[thepowersgang]

bootstrap is working on my primary test machine (Debian 9.9), but there's a few compilation errors on my secondary test machine (Mint 19.1) - looks like differences in libc++ implementations. Working on fixing those bugs.

[bjorn3]

🚀 Finally!

[thepowersgang]

As of abd2a15, rustc/cargo 1.29 are building on mint too.

Current tasks:

• Fix windows build
• Merge master into nightly-1.29 and re-test
• Fast-forward master to match nightly-1.29

[thepowersgang]

master now merged into nightly-1.29, still need to run a bootstrap test (but rustc builds)

[thepowersgang]

And nightly-1.29 is now merged into master.

[bjorn3]

🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈🎉🎈