Enforce in bootstrap that test must have stage at least 1 (except for compiletest) #145663

Kobzol · 2025-08-20T15:15:22Z

This PR cleans up a bunch of test steps and adds metadata to them. I didn't yet touch the most complicated step (CompileTest), I'm leaving that for another PR.

Testing anything on stage 0 is only possible for compiletest and with build.allow-compiletest-stage0. Testing anything else on stage 0 will either produce a nice error or crash with a stage being subtracted below zero.

r? @jieyouxu

try-job: dist-x86_64-linux
try-job: aarch64-gnu
try-job: arm-android
try-job: x86_64-gnu-llvm-20*
try-job: x86_64-msvc-*
try-job: aarch64-apple

rustbot · 2025-08-20T15:16:02Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

Kobzol · 2025-08-20T15:19:54Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

…0` is enabled

The tool wasn't useful for anything, it was only built as a part of the test, but we can just use `cargo test` and `cargo run` in the test, no need to (pre-)build the tool itself.

Kobzol · 2025-08-20T17:37:55Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

rust-bors · 2025-08-20T17:51:18Z

💔 Test for 06dca67 failed: CI. Failed jobs:

try - aarch64-apple (web logs, extended logs)
try - aarch64-gnu (web logs, extended logs)
try - arm-android (web logs, extended logs)
try - dist-x86_64-linux (web logs, extended logs)
(and 8 other jobs)

Kobzol · 2025-08-20T17:53:44Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

rust-bors · 2025-08-20T18:01:48Z

💔 Test for 71ba459 failed: CI. Failed jobs:

try - aarch64-apple (web logs, extended logs)
try - aarch64-gnu (web logs, extended logs)
try - arm-android (web logs, extended logs)
try - dist-x86_64-linux (web logs, extended logs)
(and 8 other jobs)

Kobzol · 2025-08-20T19:01:39Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

Kobzol · 2025-08-20T19:13:38Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

rust-log-analyzer · 2025-08-20T19:24:48Z

The job aarch64-gnu failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

##[group]Building stage1 unstable-book-gen (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Building stage1 rustbook (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/unstable-book/index.html
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/book/index.html
##[group]Documenting stage1 book redirect pages (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Documenting stage1 standalone (stage0 -> stage1, aarch64-unknown-linux-gnu)
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/index.html
##[endgroup]
##[group]Documenting stage2 library{alloc, compiler_builtins, core, panic_abort, panic_unwind, proc_macro, rustc-std-workspace-core, std, std_detect, sysroot, test, unwind} in HTML format (stage1 -> stage2, aarch64-unknown-linux-gnu)
##[endgroup]
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/alloc/index.html
Documenting error index (aarch64-unknown-linux-gnu)
##[group]Building stage1 error_index_generator (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/nomicon/index.html
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/reference/index.html
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/rustdoc/index.html
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/rust-by-example/index.html
Generating lint docs (aarch64-unknown-linux-gnu)
##[group]Building stage1 lint-docs (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Running stage2 lint-docs (stage1 -> stage2, aarch64-unknown-linux-gnu)
##[endgroup]
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/rustc/index.html
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/cargo/index.html
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/clippy/index.html
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/embedded-book/index.html
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/edition-guide/index.html
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/style-guide/index.html
##[group]Documenting stage1 releases (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Building stage1 linkchecker (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Testing stage1 Linkcheck (stage0 -> stage1, aarch64-unknown-linux-gnu)
---
##[endgroup]
Generating lint docs (aarch64-unknown-linux-gnu)
##[group]Running stage2 lint-docs (stage1 -> stage2, aarch64-unknown-linux-gnu)
##[endgroup]
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/doc/rustc/index.html
##[group]Building stage1 rustdoc-themes (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Testing stage2 compiletest suite=rustdoc-ui mode=ui (stage1 -> stage2, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Building stage1 jsondocck (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Building stage1 jsondoclint (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Testing stage2 compiletest suite=rustdoc-json mode=rustdoc-json (stage1 -> stage2, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Documenting stage1 compiler{rustc-main, rustc_abi, rustc_arena, rustc_ast, rustc_ast_ir, rustc_ast_lowering, rustc_ast_passes, rustc_ast_pretty, rustc_attr_parsing, rustc_baked_icu_data, rustc_borrowck, rustc_builtin_macros, rustc_codegen_llvm, rustc_codegen_ssa, rustc_const_eval, rustc_data_structures, rustc_driver, rustc_driver_impl, rustc_error_codes, rustc_error_messages, rustc_errors, rustc_expand, rustc_feature, rustc_fluent_macro, rustc_fs_util, rustc_graphviz, rustc_hashes, rustc_hir, rustc_hir_analysis, rustc_hir_pretty, rustc_hir_typeck, rustc_incremental, rustc_index, rustc_index_macros, rustc_infer, rustc_interface, rustc_lexer, rustc_lint, rustc_lint_defs, rustc_llvm, rustc_log, rustc_macros, rustc_metadata, rustc_middle, rustc_mir_build, rustc_mir_dataflow, rustc_mir_transform, rustc_monomorphize, rustc_next_trait_solver, rustc_parse, rustc_parse_format, rustc_passes, rustc_pattern_analysis, rustc_privacy, rustc_proc_macro, rustc_public, rustc_public_bridge, rustc_query_impl, rustc_query_system, rustc_resolve, rustc_sanitizers, rustc_serialize, rustc_session, rustc_span, rustc_symbol_mangling, rustc_target, rustc_thread_pool, rustc_trait_selection, rustc_traits, rustc_transmute, rustc_ty_utils, rustc_type_ir, rustc_type_ir_macros} (stage0 -> stage1, aarch64-unknown-linux-gnu)
Doc path: /tmp/.tmpga6kvS/build/tmp-dry-run/aarch64-unknown-linux-gnu/compiler-doc/rustc_main/index.html
##[endgroup]
##[group]Building stage1 html-checker (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
##[group]Testing stage1 rust-installer (stage0 -> stage1, aarch64-unknown-linux-gnu)
##[endgroup]
---
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot: all
Source: src/bootstrap/src/core/builder/tests.rs:1857
────────────────────────────────────────────────────────────────────────────────
Expression: ctx.config("test").args(&["--ci",
"true"]).args(&["--skip",
"rustdoc-js-std"]).args(&["--skip",
"rustdoc-js"]).args(&["--skip", "rustdoc-gui"]).render_steps()
────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬───────────────────────────────────────────────────────────────────
    0     0 │ [build] rustc 0 <host> -> Tidy 1 <host>
    1     1 │ [test] tidy <>
          2 │+[build] rustdoc 0 <host>
    2     3 │ [build] llvm <host>
    3     4 │ [build] rustc 0 <host> -> rustc 1 <host>
    4     5 │ [build] rustc 1 <host> -> std 1 <host>
    5     6 │ [build] rustc 0 <host> -> Compiletest 1 <host>
┈┈┈┈┈┈┈┈┈┈┈┈┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
   18    19 │ [test] CoverageRunRustdoc <host>
   19    20 │ [test] Pretty <host>
   20    21 │ [build] rustc 1 <host> -> std 1 <host>
   21    22 │ [build] rustc 0 <host> -> std 0 <host>
   22       │-[build] rustdoc 0 <host>
   23    23 │ [test] CrateLibrustc <host>
   24    24 │ [build] rustc 1 <host> -> rustc 2 <host>
   25    25 │ [test] crate-bootstrap <host> src/tools/coverage-dump
   26    26 │ [test] crate-bootstrap <host> src/tools/jsondoclint
┈┈┈┈┈┈┈┈┈┈┈┈┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
   54    54 │ [test] tier-check <host>
   55    55 │ [test] rustc 0 <host> -> rust-analyzer 1 <host>
   56    56 │ [doc] rustc (book) <host>
   57    57 │ [test] rustc 1 <host> -> lint-docs 2 <host>
   58       │-[doc] rustc 1 <host> -> std 1 <host> crates=[]
   59       │-[test] rustdoc-js-std 1 <host>
   60    58 │ [build] rustc 0 <host> -> RustdocTheme 1 <host>
   61    59 │ [test] rustdoc-theme 1 <host>
   62    60 │ [test] RustdocUi <host>
   63    61 │ [build] rustc 0 <host> -> JsonDocCk 1 <host>
────────────┴───────────────────────────────────────────────────────────────────
Stopped on the first failure. Run `cargo insta test` to run all snapshots.

thread 'core::builder::tests::snapshot::test_all' panicked at /cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.43.1/src/runtime.rs:679:13:
snapshot assertion for 'all' failed in line 1857
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/788da80fcfcef3f34c90def5baa32813e39a1a41/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_fmt
             at /rustc/788da80fcfcef3f34c90def5baa32813e39a1a41/library/core/src/panicking.rs:75:14
   2: <insta::runtime::SnapshotAssertionContext>::finalize
             at /cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.43.1/src/runtime.rs:679:13
   3: insta::runtime::assert_snapshot
             at /cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.43.1/src/runtime.rs:848:13
   4: bootstrap::core::builder::tests::snapshot::test_all
             at ./src/core/builder/tests.rs:1857:9
   5: bootstrap::core::builder::tests::snapshot::test_all::{closure#0}
             at ./src/core/builder/tests.rs:1855:18
   6: <bootstrap::core::builder::tests::snapshot::test_all::{closure#0} as core::ops::function::FnOnce<()>>::call_once

rust-bors · 2025-08-20T19:25:19Z

💔 Test for 456a51d failed: CI. Failed jobs:

try - aarch64-apple (web logs, extended logs)
try - aarch64-gnu (web logs, extended logs)
try - arm-android (web logs, extended logs)
try - dist-x86_64-linux (web logs, extended logs)
(and 8 other jobs)

Kobzol · 2025-08-20T20:31:35Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

rust-log-analyzer · 2025-08-20T21:27:53Z

The job x86_64-gnu-llvm-20-3 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

---- compiler/rustc_lint_defs/src/builtin.rs - builtin::OUT_OF_SCOPE_MACRO_CALLS (line 4835) stdout ----
warning: cannot find macro `in_root` in the current scope when looking from the crate root
 --> compiler/rustc_lint_defs/src/builtin.rs:4836:10
  |
2 | #![doc = in_root!()]
  |          ^^^^^^^ not found from the crate root
  |
  = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
  = note: for more information, see issue #124535 <https://github.com/rust-lang/rust/issues/124535>
  = help: import `macro_rules` with `use` to make it callable above its definition
---

test result: FAILED. 63 passed; 2 failed; 2 ignored; 0 measured; 0 filtered out; finished in 723.05ms

all doctests ran in 1.47s; merged doctests compilation took 0.71s
error: doctest failed, to rerun pass `-p rustc_lint_defs --doc`
Bootstrap failed while executing `--stage 1 test --skip src/tools/tidy`
Build completed unsuccessfully in 0:45:44
  local time: Wed Aug 20 21:27:43 UTC 2025
  network time: Wed, 20 Aug 2025 21:27:43 GMT
##[error]Process completed with exit code 1.

rust-bors · 2025-08-20T21:28:33Z

💔 Test for 4b82a4c failed: CI. Failed jobs:

try - aarch64-apple (web logs, extended logs)
try - aarch64-gnu (web logs, extended logs)
try - arm-android (web logs, extended logs)
try - dist-x86_64-linux (web logs, extended logs)
(and 8 other jobs)

rustbot assigned jieyouxu Aug 20, 2025

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 20, 2025

This comment has been minimized.

Sign in to view

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

rustbot added A-CI Area: Our Github Actions CI T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Aug 20, 2025

This comment has been minimized.

Sign in to view

Kobzol added 16 commits August 20, 2025 17:32

Add snapshot test for tier-check

67d70fd

Fixup x test tier-check

3dcc9b9

Forbid running tests on stage 0 unless `build.compiletest-allow-stage…

bb51abd

…0` is enabled

Add snapshot test for x test

9185529

Fix staging of TestFloatParse

6841177

The tool wasn't useful for anything, it was only built as a part of the test, but we can just use `cargo test` and `cargo run` in the test, no need to (pre-)build the tool itself.

Remove compiler_for from test::CodegenGCC

7a20ab0

Remove compiler_for from test::CodegenCranelift

e9c3e64

Refactor test::LintDocs

08b4e10

Remove stage0 checking

4824d80

Small refactoring around RemoteCopyLibs

b6c46d4

Add metadata to a bunch of steps, rename variables and add comments

a3d8ae7

Move the test! macro closer to its usages

9a409a1

Fix staging for compiler/std crate tests

2616a6e

Fix spacing when testing individual crates

5d715d7

Do not run tests on CI in stage 0

3b0a4ad

Add change tracker entry

c501499

Kobzol force-pushed the bootstrap-test branch from 00e0ffb to c501499 Compare August 20, 2025 15:34

This comment has been minimized.

Sign in to view

Fake nodejs in snapshot test

208ed47

This comment has been minimized.

Sign in to view

Kobzol force-pushed the bootstrap-test branch from b9c29fa to 73a1fef Compare August 20, 2025 19:13

This comment has been minimized.

Sign in to view

Fix tests and doclink

2178227

Kobzol force-pushed the bootstrap-test branch from 73a1fef to 2178227 Compare August 20, 2025 19:21

This comment has been minimized.

Sign in to view

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) #145663

Are you sure you want to change the base?

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) #145663

Conversation

Kobzol commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Aug 20, 2025

Uh oh!

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Aug 20, 2025

Uh oh!

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Aug 20, 2025

Uh oh!

This comment has been minimized.

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-log-analyzer commented Aug 20, 2025

Uh oh!

rust-bors bot commented Aug 20, 2025

Uh oh!

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

rust-log-analyzer commented Aug 20, 2025

Uh oh!

rust-bors bot commented Aug 20, 2025

Uh oh!

Uh oh!

Kobzol commented Aug 20, 2025 •

edited

Loading