Add a new `download-ci-llvm = if-unchanged` option and enable it by default for `profile = codegen` #116881

LuuuXXX · 2023-10-18T09:21:12Z

Three tasks have been implemented here.

Add a new download-ci-llvm = if-unchange option and enable if by
default for profile = codegen.

Include all build artifacts by traversing the llvm-project build output,
Keep the downloadable llvm the same state as if you have just run a full
source build.

After selecting the codegen profile during ./x.py setup, the submodule
will be automatically downloaded.

It should be noted that：
if you are using the 'codegen' profile, be aware that the default value changed. otherwise, you can ignore this.

Resolves #110087

rustbot · 2023-10-18T09:21:21Z

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

rustbot · 2023-10-18T09:21:24Z

This PR changes config.example.toml. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/lib.rs and change-id in config.example.toml.

This PR changes src/bootstrap/defaults/config.codegen.toml. If appropriate, please also update config.compiler.toml so the defaults are in sync.

rustbot · 2023-10-18T09:29:08Z

This PR changes src/bootstrap/defaults/config.codegen.toml. If appropriate, please also update config.compiler.toml so the defaults are in sync.

This PR changes config.example.toml. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/lib.rs and change-id in config.example.toml.

LuuuXXX · 2023-10-19T01:06:12Z

By the way, triagebot.toml should be updated because the bootstrap codebase is reorganized. The tips above seem out of date.

onur-ozkan

Current PR ID(which is 116881) should be appended to this constant:

rust/src/bootstrap/src/lib.rs

Lines 73 to 81 in 020d008

    
           /// Keeps track of major changes made to the bootstrap configuration. 
        
           /// 
        
           /// These values also represent the IDs of the PRs that caused major changes. 
        
           /// You can visit `https://github.com/rust-lang/rust/pull/{any-id-from-the-list}` to 
        
           /// check for more details regarding each change. 
        
           /// 
        
           /// If you make any major changes (such as adding new values or changing default values), please 
        
           /// ensure that the associated PR ID is added to the end of this list. 
        
           pub const CONFIG_CHANGE_HISTORY: &[usize] = &[115898];

This should be replaced with the current PR ID:

rust/config.example.toml

Lines 22 to 33 in 020d008

    
           # Keeps track of major changes made to this configuration. 
        
           # 
        
           # This value also represents ID of the PR that caused major changes. Meaning, 
        
           # you can visit github.com/rust-lang/rust/pull/{change-id} to check for more details. 
        
           # 
        
           # A 'major change' includes any of the following 
        
           #  - A new option 
        
           #  - A change in the default values 
        
           # 
        
           # If `change-id` does not match the version that is currently running, 
        
           # `x.py` will prompt you to update it and check the related PR for more details. 
        
           change-id = 115898

src/bootstrap/src/lib.rs

onur-ozkan

I still need some work to confirm the second task, maybe a separate PR is needed.

I think we can handle it in this PR

…k-Simulacrum fix bootstrap paths in triagebot.toml ref #rust-lang#116881 (comment)

Rollup merge of rust-lang#116928 - onur-ozkan:update-triagebot, r=Mark-Simulacrum fix bootstrap paths in triagebot.toml ref #rust-lang#116881 (comment)

rustbot · 2023-10-23T10:48:05Z

This PR modifies src/bootstrap/src/core/config. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/src/lib.rs and change-id in config.example.toml.

This PR modifies src/bootstrap/defaults. If appropriate, please also update CONFIG_CHANGE_HISTORY in src/bootstrap/src/lib.rs and change-id in config.example.toml.

bors · 2023-10-24T05:09:04Z

☔ The latest upstream changes (presumably #116998) made this pull request unmergeable. Please resolve the merge conflicts.

onur-ozkan · 2023-11-07T09:36:44Z

Can you please rebase and push the commit with a more descriptive message?

Commit history still looks inappropriate.

LuuuXXX · 2023-11-07T12:44:25Z

reworded it

onur-ozkan

Can you squash your commits into "Fix issue #110087" one ? Following commits alone doesn't make too much sense as far as I see. Specially the 2nd one.

src/bootstrap/src/core/build_steps/dist.rs

Three tasks have been implemented here. Add a new `download-ci-llvm = if-unchange` option and enable if by default for `profile = codegen`. Include all build artifacts by traversing the llvm-project build output, Keep the downloadable llvm the same state as if you have just run a full source build. After selecting the codegen profile during ./x.py setup, the submodule will be automatically downloaded.

onur-ozkan

Thank you for the fixes.

CC #110087
I still need some work to confirm the second task, maybe a separate PR is needed.

PR description seems to be outdated, it would be nice if you update it :) I will queue up the PR anyway.

onur-ozkan · 2023-11-08T07:18:25Z

@bors r+ rollup=iffy

bors · 2023-11-08T07:18:27Z

📌 Commit 1b8dee1 has been approved by onur-ozkan

It is now in the queue for this repository.

bors · 2023-11-08T08:08:53Z

⌛ Testing commit 1b8dee1 with merge 750c2ec...

bors · 2023-11-08T10:06:26Z

☀️ Test successful - checks-actions
Approved by: onur-ozkan
Pushing 750c2ec to master...

rust-timer · 2023-11-08T11:29:35Z

Finished benchmarking commit (750c2ec): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.0%	[0.5%, 1.5%]	3
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.7%	[-0.8%, -0.6%]	4
Improvements ✅ (secondary)	-3.7%	[-3.7%, -3.7%]	1
All ❌✅ (primary)	0.0%	[-0.8%, 1.5%]	7

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.4%, -0.4%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.0%	[-0.4%, 0.4%]	2

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.0%	[-0.0%, -0.0%]	4
All ❌✅ (primary)	-	-	0

Bootstrap: 663.473s -> 664.216s (0.11%)
Artifact size: 308.82 MiB -> 308.79 MiB (-0.01%)

RalfJung · 2023-11-11T10:05:52Z

src/bootstrap/src/lib.rs

@@ -77,7 +77,7 @@ const LLD_FILE_NAMES: &[&str] = &["ld.lld", "ld64.lld", "lld-link", "wasm-ld"];
 ///
 /// If you make any major changes (such as adding new values or changing default values), please
 /// ensure that the associated PR ID is added to the end of this list.
-pub const CONFIG_CHANGE_HISTORY: &[usize] = &[115898, 116998, 117435];
+pub const CONFIG_CHANGE_HISTORY: &[usize] = &[115898, 116998, 117435, 116881];


I'm getting a WARNING pointing me to this PR. Do I need to do something? Or can I keep my existing config? It's unclear to me. The PR description says there's a new config, but why does everyone have to be alerted of its existence?

Also this list is unordered now, that seems wrong?

update change-id = 116881 instead may be help.
AFAIK This is ordered by merge time to keep the source code synchronized with the config file.

update change-id = 116881 instead may be help.

That will silence the warning, sure. But the reason the warning is shown must be that I should do something before silencing it. There's no point in having people mindlessly increment a number every week just to get rid of the latest warning.

Also this list is unordered now, that seems wrong?

rust/src/bootstrap/src/lib.rs

Lines 78 to 80 in 1db4b12

/// If you make any major changes (such as adding new values or changing default values),

/// please ensure that the associated PR ID is added to the end of this list.

/// This is necessary because the list must be sorted by the merge date.

I'm getting a WARNING pointing me to this PR. Do I need to do something? Or can I keep my existing config? It's unclear to me

rust/src/bootstrap/defaults/config.codegen.toml

Lines 12 to 13 in 1db4b12

# build llvm from source

download-ci-llvm = "if-unchanged"

PR updated the default value here. I am pretty sure we have people who wants to know this change.

Then the PR description needs to say: "if you are coming here from x.py, here's what do do: if you are using the 'codegen' profile, be aware that the default value changed. otherwise, you can ignore this".

Currently I'm coming here clueless and wasting 5min trying to figure out if I should do something, before giving up, frustrated.

That's my bad, I will add the description for this PR later

RalfJung · 2023-11-11T11:41:36Z

For this PR I am also confused why we have if-available and if-unchanged as two different options. When would I ever want if-available? Shouldn't if-unchanged just be the default everywhere, since it will "do the right thing" in most situations?

LuuuXXX · 2023-11-11T13:01:31Z

When would I ever want if-available?

If there is no need to modify llvm locally, if-available may be able to give you faster build times. Since you can use Rust CI built LLVM instead of locally building it. and if-unchanged will consider whether there are local modifications in llvm.

Shouldn't if-unchanged just be the default everywhere, since it will "do the right thing" in most situations?

Is it similar to this? https://github.com/orgs/rust-lang/projects/35/views/1?pane=issue&itemId=40227883

RalfJung · 2023-11-11T13:10:07Z

If there is no need to modify llvm locally, if-available may be able to give you faster build times.

Have you measured it? How long does the change-check take?

We also have true for skipping all the checks, after all.

LuuuXXX · 2023-11-11T13:25:53Z

Not yet, the change-check is based on git diff-index it will not take too much time.

This comment became obsolete after rust-lang#116881 switched over to `download-ci-llvm = "if-unchanged"`.

rustbot assigned Mark-Simulacrum Oct 18, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Oct 18, 2023

This comment has been minimized.

Sign in to view

LuuuXXX force-pushed the issue-110087 branch from 7bd8b96 to 490169e Compare October 18, 2023 10:03

onur-ozkan mentioned this pull request Oct 19, 2023

fix bootstrap paths in triagebot.toml #116928

Merged

onur-ozkan requested changes Oct 19, 2023

View reviewed changes

onur-ozkan reviewed Oct 19, 2023

View reviewed changes

src/bootstrap/src/lib.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

LuuuXXX requested a review from onur-ozkan October 20, 2023 00:59

Mark-Simulacrum assigned onur-ozkan and unassigned Mark-Simulacrum Oct 21, 2023

onur-ozkan reviewed Oct 21, 2023

View reviewed changes

onur-ozkan added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 21, 2023

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Oct 21, 2023

Rollup merge of rust-lang#116928 - onur-ozkan:update-triagebot, r=Mar…

0d489f3

…k-Simulacrum fix bootstrap paths in triagebot.toml ref #rust-lang#116881 (comment)

rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Oct 21, 2023

Unrolled build for rust-lang#116928

cda34bf

Rollup merge of rust-lang#116928 - onur-ozkan:update-triagebot, r=Mark-Simulacrum fix bootstrap paths in triagebot.toml ref #rust-lang#116881 (comment)

This comment has been minimized.

Sign in to view

LuuuXXX force-pushed the issue-110087 branch from 1071e26 to 28ca7bb Compare October 24, 2023 02:54

LuuuXXX force-pushed the issue-110087 branch from 28ca7bb to b78d305 Compare October 24, 2023 07:13

LuuuXXX requested a review from onur-ozkan October 24, 2023 08:10

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 24, 2023

LuuuXXX requested a review from onur-ozkan November 7, 2023 09:32

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 7, 2023

LuuuXXX force-pushed the issue-110087 branch from acb18ff to ce0901d Compare November 7, 2023 12:37

onur-ozkan reviewed Nov 7, 2023

View reviewed changes

src/bootstrap/src/core/build_steps/dist.rs Outdated Show resolved Hide resolved

LuuuXXX force-pushed the issue-110087 branch from ce0901d to 1b8dee1 Compare November 8, 2023 01:24

onur-ozkan approved these changes Nov 8, 2023

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 8, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Nov 8, 2023

bors merged commit 750c2ec into rust-lang:master Nov 8, 2023
12 checks passed

rustbot added this to the 1.75.0 milestone Nov 8, 2023

This was referenced Nov 8, 2023

handle the case when the change-id isn't found #117263

Merged

Rollup of 8 pull requests #117704

Closed

RalfJung reviewed Nov 11, 2023

View reviewed changes

Zalathar added a commit to Zalathar/rust that referenced this pull request Dec 17, 2023

Update LLVM comments in config.codegen.toml

3655b83

This comment became obsolete after rust-lang#116881 switched over to `download-ci-llvm = "if-unchanged"`.

Zalathar added a commit to Zalathar/rust that referenced this pull request Dec 17, 2023

Update LLVM comments in config.codegen.toml

f6987b6

This comment became obsolete after rust-lang#116881 switched over to `download-ci-llvm = "if-unchanged"`.

dtolnay removed the has-merge-commits PR has merge commits, merge with caution. label Jan 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a new `download-ci-llvm = if-unchanged` option and enable it by default for `profile = codegen` #116881

Add a new `download-ci-llvm = if-unchanged` option and enable it by default for `profile = codegen` #116881

LuuuXXX commented Oct 18, 2023 •

edited

rustbot commented Oct 18, 2023

rustbot commented Oct 18, 2023

This comment has been minimized.

rustbot commented Oct 18, 2023

This comment has been minimized.

LuuuXXX commented Oct 19, 2023

onur-ozkan left a comment

This comment has been minimized.

onur-ozkan left a comment

rustbot commented Oct 23, 2023

This comment has been minimized.

bors commented Oct 24, 2023

onur-ozkan commented Nov 7, 2023

LuuuXXX commented Nov 7, 2023

onur-ozkan left a comment •

edited

onur-ozkan left a comment

onur-ozkan commented Nov 8, 2023

bors commented Nov 8, 2023

bors commented Nov 8, 2023

bors commented Nov 8, 2023

rust-timer commented Nov 8, 2023

RalfJung Nov 11, 2023

RalfJung Nov 11, 2023

LuuuXXX Nov 11, 2023

RalfJung Nov 11, 2023

onur-ozkan Nov 11, 2023

RalfJung Nov 11, 2023

LuuuXXX Nov 11, 2023

RalfJung commented Nov 11, 2023

LuuuXXX commented Nov 11, 2023

RalfJung commented Nov 11, 2023

LuuuXXX commented Nov 11, 2023

	/// Keeps track of major changes made to the bootstrap configuration.
	///
	/// These values also represent the IDs of the PRs that caused major changes.
	/// You can visit `https://github.com/rust-lang/rust/pull/{any-id-from-the-list}` to
	/// check for more details regarding each change.
	///
	/// If you make any major changes (such as adding new values or changing default values), please
	/// ensure that the associated PR ID is added to the end of this list.
	pub const CONFIG_CHANGE_HISTORY: &[usize] = &[115898];

	# Keeps track of major changes made to this configuration.
	#
	# This value also represents ID of the PR that caused major changes. Meaning,
	# you can visit github.com/rust-lang/rust/pull/{change-id} to check for more details.
	#
	# A 'major change' includes any of the following
	# - A new option
	# - A change in the default values
	#
	# If `change-id` does not match the version that is currently running,
	# `x.py` will prompt you to update it and check the related PR for more details.
	change-id = 115898

	/// If you make any major changes (such as adding new values or changing default values),
	/// please ensure that the associated PR ID is added to the end of this list.
	/// This is necessary because the list must be sorted by the merge date.

Add a new download-ci-llvm = if-unchanged option and enable it by default for profile = codegen #116881

Add a new download-ci-llvm = if-unchanged option and enable it by default for profile = codegen #116881

Conversation

LuuuXXX commented Oct 18, 2023 • edited

rustbot commented Oct 18, 2023

rustbot commented Oct 18, 2023

This comment has been minimized.

rustbot commented Oct 18, 2023

This comment has been minimized.

LuuuXXX commented Oct 19, 2023

onur-ozkan left a comment

Choose a reason for hiding this comment

This comment has been minimized.

onur-ozkan left a comment

Choose a reason for hiding this comment

rustbot commented Oct 23, 2023

This comment has been minimized.

bors commented Oct 24, 2023

onur-ozkan commented Nov 7, 2023

LuuuXXX commented Nov 7, 2023

onur-ozkan left a comment • edited

Choose a reason for hiding this comment

onur-ozkan left a comment

Choose a reason for hiding this comment

onur-ozkan commented Nov 8, 2023

bors commented Nov 8, 2023

bors commented Nov 8, 2023

bors commented Nov 8, 2023

rust-timer commented Nov 8, 2023

Overall result: no relevant changes - no action needed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RalfJung commented Nov 11, 2023

LuuuXXX commented Nov 11, 2023

RalfJung commented Nov 11, 2023

LuuuXXX commented Nov 11, 2023

Add a new `download-ci-llvm = if-unchanged` option and enable it by default for `profile = codegen` #116881

Add a new `download-ci-llvm = if-unchanged` option and enable it by default for `profile = codegen` #116881

LuuuXXX commented Oct 18, 2023 •

edited

onur-ozkan left a comment •

edited