[bootstrap] Add gcc to dist generation #125419

GuillaumeGomez · 2024-05-22T19:15:11Z

As @eholk summarized below:

From my understanding, this change would add libgccjit as an optional component to the Rust distribution. This library is licensed under GPLv2 and currently we do not have any other components under that license so it would be a new license, and one that is generally more restrictive than the other licenses we use.

It'll greatly improve the experience for anyone wanting to work on the GCC backend from the compiler.

Should help with #124172.
Will unblock #124353.

r? @Kobzol

rustbot · 2024-05-22T19:15:19Z

⚠️ Warning ⚠️

These commits modify submodules.

rustbot · 2024-05-22T19:15:21Z

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

GuillaumeGomez · 2024-05-23T13:33:04Z

From what I can see, it's not possible to tell reuse to allow deprecated licenses. So in this case, I have no clue how we can go around this issue. If someone has an idea?

Kobzol · 2024-05-23T15:57:19Z

@davidtwco Any suggestions on what to do about the GPL-2.0 license of GCC?

davidtwco · 2024-05-28T13:01:45Z

@davidtwco Any suggestions on what to do about the GPL-2.0 license of GCC?

Will get back to you about this

davidtwco · 2024-05-28T14:28:28Z

@wesleywiser and I discussed this and we're not sure about how to get REUSE to play ball (@pietroalbini might know), but we've forwarded this on to t-compiler's council rep to double-check that adding gcc like this is okay, license-wise.

pietroalbini · 2024-06-05T09:20:40Z

From what I can see, it's not possible to tell reuse to allow deprecated licenses. So in this case, I have no clue how we can go around this issue. If someone has an idea?

Note that REUSE doesn't mark the GPL 2.0 license as deprecated. What is deprecated is referring to it as GPL-2.0. You should refer to the license as either GPL-2.0-only or GPL-2.0-or-later, depending on what is appropriate.

Also, the warning doesn't come from REUSE itself, but from the SPDX License List (you'll find GPL-2.0 as deprecated in there).

eholk · 2024-06-07T16:44:49Z

We discussed this at the Council meeting today and we don't have any concerns with this change. However, we would like to confirm with the Foundation. @abibroom - do you know of any concerns from the Foundation side (feel free to ping me on Zulip or reach out via email if you'd rather not discuss this publicly.

From my understanding, this change would add libgccjit as an optional component to the Rust distribution. This library is licensed under GPLv2 and currently we do not have any other components under that license so it would be a new license, and one that is generally more restrictive than the other licenses we use.

Kobzol · 2024-06-09T12:59:45Z

@bors try

Just to see what happens on Linux dist.

Kobzol · 2024-06-09T12:58:42Z

src/bootstrap/src/core/build_steps/dist.rs

@@ -2365,6 +2366,10 @@ impl Step for RustDev {
        // just broadly useful to be able to link against the bundled LLVM.
        tarball.add_dir(builder.llvm_out(target).join("include"), "include");

+        tarball.add_dir(builder.gcc_out(target).join("install/lib/libgccjit.so"), "libgccjit.so");


What will this do on Windows? 🤔 Does the gcc build also work on non-Linux OSes, in general?

No it doesn't, good catch!

bors · 2024-06-09T13:00:58Z

⌛ Trying commit f302cfd with merge 9e6f6bf...

[bootstrap] Add gcc to dist generation As `@eholk` summarized below: > From my understanding, this change would add libgccjit as an optional component to the Rust distribution. This library is licensed under GPLv2 and currently we do not have any other components under that license so it would be a new license, and one that is generally more restrictive than the other licenses we use. It'll greatly improve the experience for anyone wanting to work on the GCC backend from the compiler. Should help with rust-lang#124172. Will unblock rust-lang#124353. r? `@Kobzol`

nikic · 2024-06-09T13:18:59Z

Can this be done without adding gcc as a submodule?

bors · 2024-06-09T14:35:57Z

💔 Test failed - checks-actions

abibroom · 2024-06-10T12:53:59Z

We discussed this at the Council meeting today and we don't have any concerns with this change. However, we would like to confirm with the Foundation. @abibroom - do you know of any concerns from the Foundation side (feel free to ping me on Zulip or reach out via email if you'd rather not discuss this publicly.

I'd like to be able to say this is all fine, but want to run it past Foundation legal counsel first just to make sure. Please bear with me whilst that happens!

abibroom · 2024-06-13T15:45:35Z

We discussed this at the Council meeting today and we don't have any concerns with this change. However, we would like to confirm with the Foundation. @abibroom - do you know of any concerns from the Foundation side (feel free to ping me on Zulip or reach out via email if you'd rather not discuss this publicly.

I'd like to be able to say this is all fine, but want to run it past Foundation legal counsel first just to make sure. Please bear with me whilst that happens!

We should be good to go with this. Legal raised two points, neither of which I believe to be blocking concerns:

Compliance with the GPLv2 license, specifically, the requirement to provide source code as well as binary distributions. This has hopefully already been thoroughly considered before deciding to add the component.
Enabling downstream compliance: "When it comes to downstream distribution, anyone providing a binary distribution of Rust incorporating this component will need to provide source code for the whole distribution. This is ultimately their problem, but if you're aware of anyone who's doing this currently, it's probably worth talking through those cases and figuring out how to provide notice to them and other potential distributors of their additional obligations with respect to this component."

eholk · 2024-06-14T17:48:05Z

Enabling downstream compliance: "When it comes to downstream distribution, anyone providing a binary distribution of Rust incorporating this component will need to provide source code for the whole distribution. This is ultimately their problem, but if you're aware of anyone who's doing this currently, it's probably worth talking through those cases and figuring out how to provide notice to them and other potential distributors of their additional obligations with respect to this component."

@abibroom - just to make sure, if someone wanted to make a downstream binary distribution without releasing source code (which strikes me as rather unlikely), they could still do so if they chose not to include the libgccjit component?

I want to make sure this change doesn't effectively mean we have to relicense Rust as GPL2.

bjorn3 · 2024-07-12T13:35:32Z

Assuming that cg_gcc does the same for distribution as cg_clif, all GCC components and the GCC backend itself will end up in a rustc-codegen-gcc-preview component which would need to be separately installed (using eg rustup component add rustc-codegen-gcc-preview) to use the GCC backend.

GuillaumeGomez · 2024-07-22T13:19:49Z

From what I can see, it's not possible to tell reuse to allow deprecated licenses. So in this case, I have no clue how we can go around this issue. If someone has an idea?

Note that REUSE doesn't mark the GPL 2.0 license as deprecated. What is deprecated is referring to it as GPL-2.0. You should refer to the license as either GPL-2.0-only or GPL-2.0-or-later, depending on what is appropriate.

Also, the warning doesn't come from REUSE itself, but from the SPDX License List (you'll find GPL-2.0 as deprecated in there).

As you can see @pietroalbini, REUSE wants GPL-2.0 specifically, any other will not work. And SPDX is considering GPL-2.0 as deprecated. So unless I'm missing something, we're stuck here. Any idea?

Skgland · 2024-07-22T14:51:25Z

Based on #125419 (comment) it looks like the deprecated license originates in three files

'GPL-2.0' found in:

../src/gcc/gcc/testsuite/c-c++-common/analyzer/asm-x86-dyndbg-2.c

../src/gcc/gcc/testsuite/c-c++-common/analyzer/taint-assert-BUG_ON.c

../src/gcc/gcc/testsuite/gcc.dg/analyzer/asm-x86-dyndbg-1.c

would it be possible to patch these to specify GPL-2.0-only or GPL-2.0-or-later instead of GPL-2.0?

All three currently contain:

/* Adapted from code in the Linux kernel, which has this: */
/* SPDX-License-Identifier: GPL-2.0 */

GuillaumeGomez · 2024-07-22T15:03:41Z

I'll simply remove them...

EDIT: Ah since it's our fork, I suppose it's ok to change the license there.

Skgland · 2024-07-22T15:06:31Z

Based on https://github.com/torvalds/linux/blob/master/LICENSES/preferred/GPL-2.0

For 'GNU General Public License (GPL) version 2 only' use:
    SPDX-License-Identifier: GPL-2.0
  or
    SPDX-License-Identifier: GPL-2.0-only

and the comment mentioning that the license is chosen based on them being derived from Linux kernel files with GPL-2.0 I think GPL-2.0-only should be considered equivalent

davidmalcolm · 2024-07-22T15:13:09Z

As @eholk summarized below:

From my understanding, this change would add libgccjit as an optional component to the Rust distribution. This library is licensed under GPLv2 and currently we do not have any other components under that license so it would be a new license, and one that is generally more restrictive than the other licenses we use.

Note that the libgccjit library is licensed under GPLv3, not GPLv2 (I'm the author/maintainer of libgccjit, but it's mostly just a thin wrapper around GCC's own code hence GPLv3).

Are you basing the GPLv2 thing on those SPDX identifiers from the three files in the testsuite? Those are taken/adapted from the Linux kernel and thus GPLv2 only, AIUI.

GuillaumeGomez · 2024-07-22T15:21:59Z

That's what I inferred as well: rust-lang/gcc#61

GuillaumeGomez · 2024-07-22T18:10:53Z

I think I fixed all issues here. Is there anything else that remains to be done?

nikic · 2024-07-22T19:10:11Z

I'd like to repeat this question:

Can this be done without adding gcc as a submodule?

Does distributing libgcc require gcc to be an actual submodule in the rust repo?

Does this make gcc part of the rustc-src tarballs?

GuillaumeGomez · 2024-07-22T19:13:27Z

I'd like to repeat this question:

Can this be done without adding gcc as a submodule?

Does distributing libgcc require gcc to be an actual submodule in the rust repo?

Does this make gcc part of the rustc-src tarballs?

I originally went for a commit hash stored in a file but I was asked (at least that's what I remember but maybe I'm wrong) to use a git submodule instead to make things simpler.

Does this make gcc part of the rustc-src tarballs?

Normally no but we'll need someone from infra to answer this question.

Kobzol · 2024-07-22T19:51:19Z

Does this make gcc part of the rustc-src tarballs?

Yes, because the submodule is under the src directory, and everything there is included by default.

As of the current shape of this PR, there is src/gcc with all its GCC glory in our source tarballs.

Edit: you can try this locally using x dist rustc-src.

GuillaumeGomez · 2024-07-24T11:36:35Z

Thanks for clarifying @Kobzol !

So what remains to be discussed here?

rustbot assigned Kobzol May 22, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels May 22, 2024

This comment has been minimized.

Sign in to view

rustbot added the A-testsuite Area: The testsuite used to check the correctness of rustc label May 23, 2024

This comment has been minimized.

Sign in to view

GuillaumeGomez force-pushed the add-gcc-to-dist branch from 5f791d3 to 6736333 Compare May 23, 2024 09:17

This comment has been minimized.

Sign in to view

rustbot added the A-meta Area: Issues about the rust-lang/rust repository. label May 23, 2024

This comment has been minimized.

Sign in to view

Kobzol reviewed Jun 9, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 9, 2024

GuillaumeGomez force-pushed the add-gcc-to-dist branch from f302cfd to b3fa1c2 Compare July 16, 2024 16:41

This comment has been minimized.

Sign in to view

GuillaumeGomez added 6 commits July 16, 2024 18:55

Add GCC submodule

da8b91f

Add libgccjit dist generation

59681be

Ignore gcc submodule for tidy checks

ffd4d5a

Add missing licenses for GCC

4d0bc04

Rename LICENSES/GPL-2.0.txt into LICENSES/GPL-2.0-only.txt

a00b670

Correctly generate stamp for gcc

ca33917

GuillaumeGomez force-pushed the add-gcc-to-dist branch from b3fa1c2 to ca33917 Compare July 16, 2024 16:56

This comment has been minimized.

Sign in to view

Remove src/gcc from rustfmt checks

863ba9f

This comment has been minimized.

Sign in to view

GuillaumeGomez mentioned this pull request Jul 22, 2024

Fix licenses rust-lang/gcc#61

Merged

Update GCC version

31be1e5

GuillaumeGomez force-pushed the add-gcc-to-dist branch from 0a58112 to fa6eb5b Compare July 22, 2024 18:20

Update code comments to mention GCC and not LLVM

7319664

GuillaumeGomez force-pushed the add-gcc-to-dist branch from fa6eb5b to 7319664 Compare July 22, 2024 18:23

antoyo approved these changes Jul 22, 2024

View reviewed changes

[bootstrap] Add gcc to dist generation #125419

Are you sure you want to change the base?

[bootstrap] Add gcc to dist generation #125419

Conversation

GuillaumeGomez commented May 22, 2024 • edited by traviscross Loading

rustbot commented May 22, 2024

rustbot commented May 22, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

GuillaumeGomez commented May 23, 2024

Kobzol commented May 23, 2024

davidtwco commented May 28, 2024

davidtwco commented May 28, 2024

pietroalbini commented Jun 5, 2024

eholk commented Jun 7, 2024

Kobzol commented Jun 9, 2024

Kobzol Jun 9, 2024

Choose a reason for hiding this comment

GuillaumeGomez Jul 16, 2024

Choose a reason for hiding this comment

bors commented Jun 9, 2024

nikic commented Jun 9, 2024

This comment has been minimized.

bors commented Jun 9, 2024

abibroom commented Jun 10, 2024

abibroom commented Jun 13, 2024

eholk commented Jun 14, 2024

bjorn3 commented Jul 12, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

GuillaumeGomez commented Jul 22, 2024

Skgland commented Jul 22, 2024 • edited Loading

GuillaumeGomez commented Jul 22, 2024 • edited Loading

Skgland commented Jul 22, 2024 • edited Loading

davidmalcolm commented Jul 22, 2024

GuillaumeGomez commented Jul 22, 2024

GuillaumeGomez commented Jul 22, 2024

nikic commented Jul 22, 2024

GuillaumeGomez commented Jul 22, 2024

Kobzol commented Jul 22, 2024 • edited Loading

GuillaumeGomez commented Jul 24, 2024

GuillaumeGomez commented May 22, 2024 •

edited by traviscross

Loading

Skgland commented Jul 22, 2024 •

edited

Loading

GuillaumeGomez commented Jul 22, 2024 •

edited

Loading

Skgland commented Jul 22, 2024 •

edited

Loading

Kobzol commented Jul 22, 2024 •

edited

Loading