Add ppc64, ppc64le, s390 host release builds for linux

[brson]

All three of these architectures are supported by Debian and Fedora (and RHEL). To make it easier for them to package Firefox it would be proper for us to have compilers for these architectures. End of year is a reasonable timeframe.

• s390
• ppc64
• ppc64le

cc @hsivonen @stransky @glandium @fabiand @sylvestre @infinity0

[infinity0]

Nice, thanks! Here are the pages describing supported architectures for Debian and Fedora.

You might also want to add mips64el to your list - Fedora already supports it, and it's due to be supported by Debian with the next stable release this November (and both of us call it mips64el, unlike the Fedora:pcc64le/Debian:ppc64el difference).

Also we both call it "s390x" for the newer 64bit chips.

[nagisa]

s390 corresponds to the SystemZ in LLVM, which we have no support for in rust (yet?).

[brson]

@infinity0 Today we have target builds for mips and mipsel, but I haven't considered it a priority to make host builds for them yet. With mips64el we don't even have target builds though. I will open seperate issues for the mipses: #36015

Frankly, the reason I've prioritized these three together, and highly, is because they are IBM platforms, important to RHEL. Having them ready in Fedora will make things go smoother for the next RHEL when it comes time to package Firefox.

[japaric]

@infinity0 Do you know what's the minimum/oldest glibc each port is expected to run against? rustc
is dynamically linked to glibc and we should pick a glibc to link to that's not newer than the
glibc installed in any of these ports.

This is what's need to be done for each target:

For mips and mipsel:

b) Try to cross compile rustc for this target (locally)
c) Fix any error that pops up during the bootstrap
d) Smoke test the produced rustc
e) Add a build slave for this target and start producing nightly releases

For ppc64 and ppc64le :

a) Get/build an appropriate C cross toolchain (gcc + glibc). By appropriate, I mean that the
toolchain's glibc should be old enough to support the oldest supported release of
Debian/Fedora/RHEL.

• do (b), (c), (d) and (e)

mips64 and s390x are more complicated because there are no built-in targets for these:

• Add a basic target definition to librustc_back/target
• Enable the corresponding backend when building LLVM
• Teach rust-lang/libc about these architectures
• do (a), (b), (c), (d) and (e)

I'm look into this and report back here (or in #36015); it may take a while though.

[infinity0]

@japaric I'm not too sure myself, but I would guess whatever version is listed here in ~3 months' time when stretch is finally released as Debian 9. Right now this is glibc 2.23. I'll try to find someone who knows this better than I. Fedora probably has something similar.

[japaric]

glibc 2.23

Great! That's what we are using in the build slaves (for ppc* and mips*) right now. If we find out later that we need an older version, we can always revisit.

[sylvestre]

FYI, LLVM 3.8 is supported correctly on Debian for these three archs: https://buildd.debian.org/status/package.php?p=llvm-toolchain-3.8

[japaric]

OK, I got a functional rustc for ppc64 and also managed to build rustc for ppc64el but couldn't test the latter under qemu because it segfaulted. But qemu-ppc64el segfaulted with every other binary I threw at it so ... yeah, perhaps that rustc also works but I have no way to know (I don't have powerpc hardware lying around).

I sent a PR (rust-lang-deprecated/rust-buildbot#121) with the required changes to the buildbot repository. Now it's @alexcrichton's turn to do the heavy work of spinning up more build slaves that will produce the rustcs for these targets.

I'll look into s390x next.

[japaric]

I'll look into s390x next.

#36028 adds initial support for s390x

[alexcrichton]

@brson so today we're producing six cross-compiled host builds on our nightly builder which takes about 9 hours (!!!). This, added with #36015, is another six hosts to build. Surely we can't have an 18 hour build cycle, so we'll need to add some parallelization, but this represents even more of an infrastructure investment than we had before. Are we going to be able to get money for all that?

[nagisa]

Surely we can't have an 18 hour build cycle

Why? It seems to me it would be fine as long as it is not 24 hours.

[brson]

Regardless of whether we parallelize or not we're going to be paying for the additional CPU hours. It's definitely something we need to discuss internally.

[japaric]

I've managed to acquire access to a s390 box 😈 . I'm going to look into adding s390 support to libc next.

[infinity0]

Quite a lot of tests such as those mentioned in #34889 are x86-only (uses "stdcall" or other x86-specific features). For some of these, we have architecture-specific blacklists for e.g. arm/arm64 but this should probably be inverted, i.e. instead whitelist them for x86.

[nagisa]

@infinity0 I disagree. Consider a new target is added; its trivial to inspect and fix or blacklist the failing tests. In the whitelisting scheme, the new target simply wouldn’t run any of these tests under whitelists, resulting in bugs not being tested for and missing coverage that is hard to recover.

[infinity0]

You misunderstand me. Most things should use a blacklist. Things that are clearly x86-only such as "stdcall" should use a whitelist.

[nagisa]

@infinity0 what about stuff that’s clearly “x86-only” given the list of stuff we (or LLVM or whatever) support now, but might not be “x86-only” tomorrow (LLVM implements something/we add support for weird x86-like targets which support those not-quite-“x86-only”-anymore features/etc)?

E.g. Javascript/wasm targets being mostly ABI agnostic, they could be made to accept any and all ABI strings on “just because we can” basis, including the stdcall.

[infinity0]

It's a judgement call. To extend your example, a "stdcall" function with 0 args + 0 return value compiles fine even on arm64 - see my lengthy investigation on #36348. The main thing to evaluate I guess, is it really worth spending the time to fix the cases for which they fail? "Just because we can" is fine theoretically, but not when I spend 6+ total hours chasing a bug where "we can't" was also acceptable and should have been allowed by the original test writer... (granted I'm new to this so some of it was learning to work with the tools)

[amboar]

@alexcrichton @brson I can provide at least a ppc64le machine (possibly ppc64 as well). I've cross-built rust for it and have it configured as a buildslave (currently with its own master polling github for testing purposes). If a ppc64le stage0 is hosted on rust-lang.org it's ready to go.

[alexcrichton]

@amboar awesome! We should have ppc64 compiler nightlies up and running shortly (hopefully by the end of the week). After that we should be able to just connect the buildslave and the bootstrap should proceed as normal I believe.

[alexcrichton]

Ok, I've deployed these changes, and hopefully we'll have some builds tomorrow morning! In the inevitable event that something doesn't go quite right, I'll debug the failure tomorrow as well.

@amboar we're going to build a new beta soon, and I believe all these changes are set up for there as well, so we may pick up a new beta very soon to enable bootstrapping on ppc64le

[hsivonen]

Ok, I've deployed these changes, and hopefully we'll have some builds tomorrow morning!

Nice! Thank you.

Is a separate issue needed for making the builds available via rustup.rs or is that an automatic part of this deployment?

[alexcrichton]

Nah that should all happen automatically once we get successful builds. Looks like last night didn't succeed for various reasons, so let's hope tonight will!

[alexcrichton]

Complications last night prevented these nightlies from going out, hopefully tonight though!