From 6afedf303588816cb0b316a1cfaab6e88197f181 Mon Sep 17 00:00:00 2001 From: Alona Enraght-Moony Date: Wed, 5 Nov 2025 18:11:23 +0000 Subject: [PATCH 01/26] Say that docs.rs sometimes rebuilds crates. --- src/doc/rustc-dev-guide/src/rustdoc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/doc/rustc-dev-guide/src/rustdoc.md b/src/doc/rustc-dev-guide/src/rustdoc.md index 7901e951112c9..aba80131dac20 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc.md +++ b/src/doc/rustc-dev-guide/src/rustdoc.md @@ -175,7 +175,7 @@ When crates are published to crates.io, docs.rs automatically builds and publishes their documentation, for instance at . It always builds with the current nightly rustdoc, so any changes you land in rustdoc are "insta-stable" in that they will -have an immediate public effect on docs.rs. Old documentation is not rebuilt, so +have an immediate public effect on docs.rs. Old documentation is only sometimes rebuilt, so you will see some variation in UI when browsing old releases in docs.rs. Crate authors can request rebuilds, which will be run with the latest rustdoc. From 9c1b958b5e4d51ab4e9485c8096fcd040d62cc81 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Wed, 5 Nov 2025 20:25:39 +0200 Subject: [PATCH 02/26] sembr src/rustdoc.md --- src/doc/rustc-dev-guide/src/rustdoc.md | 128 ++++++++++++------------- 1 file changed, 64 insertions(+), 64 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/rustdoc.md b/src/doc/rustc-dev-guide/src/rustdoc.md index aba80131dac20..9e2fe07ea34a6 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc.md +++ b/src/doc/rustc-dev-guide/src/rustdoc.md @@ -1,28 +1,26 @@ # Rustdoc overview -`rustdoc` lives in-tree with the -compiler and standard library. This chapter is about how it works. +`rustdoc` lives in-tree with the compiler and standard library. +This chapter is about how it works. For information about Rustdoc's features and how to use them, see the [Rustdoc book](https://doc.rust-lang.org/nightly/rustdoc/). -For more details about how rustdoc works, see the -["Rustdoc internals" chapter][Rustdoc internals]. +For more details about how rustdoc works, see the ["Rustdoc internals" chapter][Rustdoc internals]. [Rustdoc internals]: ./rustdoc-internals.md `rustdoc` uses `rustc` internals (and, of course, the standard library), so you will have to build the compiler and `std` once before you can build `rustdoc`. -Rustdoc is implemented entirely within the crate [`librustdoc`][rd]. It runs -the compiler up to the point where we have an internal representation of a -crate (HIR) and the ability to run some queries about the types of items. [HIR] -and [queries] are discussed in the linked chapters. +Rustdoc is implemented entirely within the crate [`librustdoc`][rd]. +It runs the compiler up to the point where we have an internal representation of a +crate (HIR) and the ability to run some queries about the types of items. +[HIR] and [queries] are discussed in the linked chapters. [HIR]: ./hir.md [queries]: ./query.md [rd]: https://github.com/rust-lang/rust/tree/HEAD/src/librustdoc -`librustdoc` performs two major steps after that to render a set of -documentation: +`librustdoc` performs two major steps after that to render a set of documentation: * "Clean" the AST into a form that's more suited to creating documentation (and slightly more resistant to churn in the compiler). @@ -31,9 +29,9 @@ documentation: Naturally, there's more than just this, and those descriptions simplify out lots of details, but that's the high-level overview. -(Side note: `librustdoc` is a library crate! The `rustdoc` binary is created -using the project in [`src/tools/rustdoc`][bin]. Note that literally all that -does is call the `main()` that's in this crate's `lib.rs`, though.) +(Side note: `librustdoc` is a library crate! +The `rustdoc` binary is created using the project in [`src/tools/rustdoc`][bin]. +Note that literally all that does is call the `main()` that's in this crate's `lib.rs`, though.) [bin]: https://github.com/rust-lang/rust/tree/HEAD/src/tools/rustdoc @@ -47,15 +45,14 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) rustdoc you can run on other projects. * Add `library/test` to be able to use `rustdoc --test`. * Run `rustup toolchain link stage2 build/host/stage2` to add a - custom toolchain called `stage2` to your rustup environment. After - running that, `cargo +stage2 doc` in any directory will build with + custom toolchain called `stage2` to your rustup environment. + After running that, `cargo +stage2 doc` in any directory will build with your locally-compiled rustdoc. * Use `./x doc library` to use this rustdoc to generate the standard library docs. * The completed docs will be available in `build/host/doc` (under `core`, `alloc`, and `std`). * If you want to copy those docs to a webserver, copy all of - `build/host/doc`, since that's where the CSS, JS, fonts, and landing - page are. + `build/host/doc`, since that's where the CSS, JS, fonts, and landing page are. * For frontend debugging, disable the `rust.docs-minification` option in [`bootstrap.toml`]. * Use `./x test tests/rustdoc*` to run the tests using a stage1 rustdoc. @@ -65,7 +62,8 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) ### JavaScript CI checks -Rustdoc’s JavaScript and TypeScript are checked during CI by `eslint`, `es-check`, and `tsc` (not by compiletest). These run as part of the `tidy` job. +Rustdoc’s JavaScript and TypeScript are checked during CI by `eslint`, `es-check`, and `tsc` (not by compiletest). +These run as part of the `tidy` job. ```bash ./x.py test tidy --extra-checks=js @@ -103,22 +101,24 @@ See [Rustdoc tests suites](tests/compiletest.md#rustdoc-test-suites) for more de ## Constraints -We try to make rustdoc work reasonably well with JavaScript disabled, and when -browsing local files. We support +We try to make rustdoc work reasonably well with JavaScript disabled, and when browsing local files. +We support [these browsers](https://rust-lang.github.io/rfcs/1985-tiered-browser-support.html#supported-browsers). Supporting local files (`file:///` URLs) brings some surprising restrictions. Certain browser features that require secure origins, like `localStorage` and -Service Workers, don't work reliably. We can still use such features but we -should make sure pages are still usable without them. +Service Workers, don't work reliably. +We can still use such features but we should make sure pages are still usable without them. Rustdoc [does not type-check function bodies][platform-specific docs]. This works by [overriding the built-in queries for typeck][override queries], by [silencing name resolution errors], and by [not resolving opaque types]. This comes with several caveats: in particular, rustdoc *cannot* run any parts of the compiler that -require type-checking bodies; for example it cannot generate `.rlib` files or run most lints. +require type-checking bodies; +for example it cannot generate `.rlib` files or run most lints. We want to move away from this model eventually, but we need some alternative for -[the people using it][async-std]; see [various][zulip stop accepting broken code] +[the people using it][async-std]; +see [various][zulip stop accepting broken code] [previous][rustdoc meeting 2024-07-08] [Zulip][compiler meeting 2023-01-26] [discussion][notriddle rfc]. For examples of code that breaks if this hack is removed, see [`tests/rustdoc-ui/error-in-impl-trait`]. @@ -136,9 +136,9 @@ For examples of code that breaks if this hack is removed, see ## Multiple runs, same output directory -Rustdoc can be run multiple times for varying inputs, with its output set to the -same directory. That's how cargo produces documentation for dependencies of the -current crate. It can also be done manually if a user wants a big +Rustdoc can be run multiple times for varying inputs, with its output set to the same directory. +That's how cargo produces documentation for dependencies of the current crate. +It can also be done manually if a user wants a big documentation bundle with all of the docs they care about. HTML is generated independently for each crate, but there is some cross-crate @@ -147,68 +147,68 @@ information that we update as we add crates to the output directory: - `crates.js` holds a list of all crates in the output directory. - `search-index.js` holds a list of all searchable items. - For each trait, there is a file under `implementors/.../trait.TraitName.js` - containing a list of implementors of that trait. The implementors may be in - different crates than the trait, and the JS file is updated as we discover - new ones. + containing a list of implementors of that trait. + The implementors may be in + different crates than the trait, and the JS file is updated as we discover new ones. ## Use cases -There are a few major use cases for rustdoc that you should keep in mind when -working on it: +There are a few major use cases for rustdoc that you should keep in mind when working on it: ### Standard library docs -These are published at as part of the Rust release -process. Stable releases are also uploaded to specific versioned URLs like -. Beta and nightly docs are published to +These are published at as part of the Rust release process. +Stable releases are also uploaded to specific versioned URLs like +. +Beta and nightly docs are published to and . The docs are uploaded with the [promote-release -tool](https://github.com/rust-lang/promote-release) and served from S3 with -CloudFront. +tool](https://github.com/rust-lang/promote-release) and served from S3 with CloudFront. -The standard library docs contain five crates: alloc, core, proc_macro, std, and -test. +The standard library docs contain five crates: alloc, core, proc_macro, std, and test. ### docs.rs When crates are published to crates.io, docs.rs automatically builds -and publishes their documentation, for instance at -. It always builds with the current nightly +and publishes their documentation, for instance at . +It always builds with the current nightly rustdoc, so any changes you land in rustdoc are "insta-stable" in that they will -have an immediate public effect on docs.rs. Old documentation is only sometimes rebuilt, so -you will see some variation in UI when browsing old releases in docs.rs. Crate -authors can request rebuilds, which will be run with the latest rustdoc. +have an immediate public effect on docs.rs. +Old documentation is only sometimes rebuilt, so +you will see some variation in UI when browsing old releases in docs.rs. +Crate authors can request rebuilds, which will be run with the latest rustdoc. Docs.rs performs some transformations on rustdoc's output in order to save -storage and display a navigation bar at the top. In particular, certain static +storage and display a navigation bar at the top. +In particular, certain static files, like main.js and rustdoc.css, may be shared across multiple invocations -of the same version of rustdoc. Others, like crates.js and sidebar-items.js, are -different for different invocations. Still others, like fonts, will never -change. These categories are distinguished using the `SharedResource` enum in +of the same version of rustdoc. +Others, like crates.js and sidebar-items.js, are different for different invocations. +Still others, like fonts, will never change. +These categories are distinguished using the `SharedResource` enum in `src/librustdoc/html/render/write_shared.rs` Documentation on docs.rs is always generated for a single crate at a time, so -the search and sidebar functionality don't include dependencies of the current -crate. +the search and sidebar functionality don't include dependencies of the current crate. ### Locally generated docs -Crate authors can run `cargo doc --open` in crates they have checked -out locally to see the docs. This is useful to check that the docs they -are writing are useful and display correctly. It can also be useful for -people to view documentation on crates they aren't authors of, but want to -use. In both cases, people may use `--document-private-items` Cargo flag to +Crate authors can run `cargo doc --open` in crates they have checked out locally to see the docs. +This is useful to check that the docs they are writing are useful and display correctly. +It can also be useful for people to view documentation on crates they aren't authors of, but want to +use. +In both cases, people may use `--document-private-items` Cargo flag to see private methods, fields, and so on, which are normally not displayed. -By default `cargo doc` will generate documentation for a crate and all of its -dependencies. That can result in a very large documentation bundle, with a large -(and slow) search corpus. The Cargo flag `--no-deps` inhibits that behavior and -generates docs for just the crate. +By default `cargo doc` will generate documentation for a crate and all of its dependencies. +That can result in a very large documentation bundle, with a large (and slow) search corpus. +The Cargo flag `--no-deps` inhibits that behavior and generates docs for just the crate. ### Self-hosted project docs -Some projects like to host their own documentation. For example: -. This is easy to do by locally generating docs, and -simply copying them to a web server. Rustdoc's HTML output can be extensively -customized by flags. Users can add a theme, set the default theme, and inject -arbitrary HTML. See `rustdoc --help` for details. +Some projects like to host their own documentation. +For example: . +This is easy to do by locally generating docs, and simply copying them to a web server. +Rustdoc's HTML output can be extensively customized by flags. +Users can add a theme, set the default theme, and inject arbitrary HTML. +See `rustdoc --help` for details. From 899ff1aeee53cdc5670deb462e393ddefdf195ab Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Wed, 5 Nov 2025 20:32:19 +0200 Subject: [PATCH 03/26] use sentence case for titles --- src/doc/rustc-dev-guide/src/rustdoc-internals.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/rustdoc-internals.md b/src/doc/rustc-dev-guide/src/rustdoc-internals.md index 55b08e8893c0b..927bffc591bbc 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc-internals.md +++ b/src/doc/rustc-dev-guide/src/rustdoc-internals.md @@ -1,11 +1,11 @@ -# Rustdoc Internals +# Rustdoc internals This page describes [`rustdoc`]'s passes and modes. For an overview of `rustdoc`, see the ["Rustdoc overview" chapter](./rustdoc.md). [`rustdoc`]: https://github.com/rust-lang/rust/tree/HEAD/src/tools/rustdoc -## From Crate to Clean +## From crate to clean In [`core.rs`] are two central items: the [`rustdoc::core::DocContext`] `struct`, and the [`rustdoc::core::run_global_ctxt`] function. The latter is @@ -153,7 +153,7 @@ itself. [`librustdoc/passes`]: https://github.com/rust-lang/rust/tree/HEAD/src/librustdoc/passes [`stripper`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustdoc/passes/stripper/index.html -## From Clean To HTML +## From clean to HTML This is where the "second phase" in `rustdoc` begins. This phase primarily lives in the [`librustdoc/formats`] and [`librustdoc/html`] folders, and it all starts with @@ -247,7 +247,7 @@ generation, as well as just keeping things organized: [didn't used to be the case]: https://github.com/rust-lang/rust/pull/80090 [`SharedContext`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustdoc/html/render/context/struct.SharedContext.html -## Other Tricks Up Its Sleeve +## Other tricks up its sleeve All this describes the process for generating `HTML` documentation from a Rust crate, but there are couple other major modes that `rustdoc` runs in. It can also @@ -268,7 +268,7 @@ in `test.rs` is the function `make_test`, which is where hand-written Some extra reading about `make_test` can be found [here](https://quietmisdreavus.net/code/2018/02/23/how-the-doctests-get-made/). -## Testing Locally +## Testing locally Some features of the generated `HTML` documentation might require local storage to be used across pages, which doesn't work well without an `HTTP` @@ -284,7 +284,7 @@ $ python3 -m http.server -d build/[YOUR ARCH]/doc Now you can browse your documentation just like you would if it was hosted on the internet. For example, the url for `std` will be `rust/std/`. -## See Also +## See also - The [`rustdoc` api docs] - [An overview of `rustdoc`](./rustdoc.md) From 06d329ec338cde23af5719e9eba56b58bae80688 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Wed, 5 Nov 2025 20:56:16 +0200 Subject: [PATCH 04/26] some text improvements --- src/doc/rustc-dev-guide/src/rustdoc.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/rustdoc.md b/src/doc/rustc-dev-guide/src/rustdoc.md index 9e2fe07ea34a6..1be0ae3696b68 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc.md +++ b/src/doc/rustc-dev-guide/src/rustdoc.md @@ -102,24 +102,23 @@ See [Rustdoc tests suites](tests/compiletest.md#rustdoc-test-suites) for more de ## Constraints We try to make rustdoc work reasonably well with JavaScript disabled, and when browsing local files. -We support -[these browsers](https://rust-lang.github.io/rfcs/1985-tiered-browser-support.html#supported-browsers). +We have [a list of supported browsers]. Supporting local files (`file:///` URLs) brings some surprising restrictions. Certain browser features that require secure origins, like `localStorage` and Service Workers, don't work reliably. -We can still use such features but we should make sure pages are still usable without them. +We can still use such features, but we should make sure pages are still usable without them. Rustdoc [does not type-check function bodies][platform-specific docs]. This works by [overriding the built-in queries for typeck][override queries], by [silencing name resolution errors], and by [not resolving opaque types]. This comes with several caveats: in particular, rustdoc *cannot* run any parts of the compiler that require type-checking bodies; -for example it cannot generate `.rlib` files or run most lints. +for example, it cannot generate `.rlib` files or run most lints. We want to move away from this model eventually, but we need some alternative for [the people using it][async-std]; see [various][zulip stop accepting broken code] -[previous][rustdoc meeting 2024-07-08] [Zulip][compiler meeting 2023-01-26] [discussion][notriddle rfc]. +[previous][rustdoc meeting 2024-07-08] [Zulip][compiler meeting 2023-01-26] [discussions][notriddle rfc]. For examples of code that breaks if this hack is removed, see [`tests/rustdoc-ui/error-in-impl-trait`]. @@ -133,6 +132,7 @@ For examples of code that breaks if this hack is removed, see [zulip stop accepting broken code]: https://rust-lang.zulipchat.com/#narrow/stream/266220-rustdoc/topic/stop.20accepting.20broken.20code [notriddle rfc]: https://rust-lang.zulipchat.com/#narrow/channel/266220-t-rustdoc/topic/Pre-RFC.3A.20stop.20accepting.20broken.20code [`tests/rustdoc-ui/error-in-impl-trait`]: https://github.com/rust-lang/rust/tree/163cb4ea3f0ae3bc7921cc259a08a7bf92e73ee6/tests/rustdoc-ui/error-in-impl-trait +[a list of supported browsers]: https://rust-lang.github.io/rfcs/1985-tiered-browser-support.html#supported-browsers ## Multiple runs, same output directory @@ -200,7 +200,7 @@ use. In both cases, people may use `--document-private-items` Cargo flag to see private methods, fields, and so on, which are normally not displayed. -By default `cargo doc` will generate documentation for a crate and all of its dependencies. +By default, `cargo doc` will generate documentation for a crate and all of its dependencies. That can result in a very large documentation bundle, with a large (and slow) search corpus. The Cargo flag `--no-deps` inhibits that behavior and generates docs for just the crate. From c6b4a22abea5ae1402a54504509561e04f84b215 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Wed, 5 Nov 2025 20:56:59 +0200 Subject: [PATCH 05/26] that redirects to docs.rs --- src/doc/rustc-dev-guide/src/rustdoc.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/rustdoc.md b/src/doc/rustc-dev-guide/src/rustdoc.md index 1be0ae3696b68..0e673bf4f58bb 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc.md +++ b/src/doc/rustc-dev-guide/src/rustdoc.md @@ -206,8 +206,7 @@ The Cargo flag `--no-deps` inhibits that behavior and generates docs for just th ### Self-hosted project docs -Some projects like to host their own documentation. -For example: . +Some projects host their own documentation. This is easy to do by locally generating docs, and simply copying them to a web server. Rustdoc's HTML output can be extensively customized by flags. Users can add a theme, set the default theme, and inject arbitrary HTML. From 3a4d1467be480d805d6e94f08a61318d0fa2a5eb Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Wed, 5 Nov 2025 21:02:44 +0200 Subject: [PATCH 06/26] sembr src/rustdoc-internals.md --- .../rustc-dev-guide/src/rustdoc-internals.md | 141 ++++++++++-------- 1 file changed, 75 insertions(+), 66 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/rustdoc-internals.md b/src/doc/rustc-dev-guide/src/rustdoc-internals.md index 927bffc591bbc..199a75e99d672 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc-internals.md +++ b/src/doc/rustc-dev-guide/src/rustdoc-internals.md @@ -1,22 +1,23 @@ # Rustdoc internals -This page describes [`rustdoc`]'s passes and modes. For an overview of `rustdoc`, -see the ["Rustdoc overview" chapter](./rustdoc.md). +This page describes [`rustdoc`]'s passes and modes. +For an overview of `rustdoc`, see the ["Rustdoc overview" chapter](./rustdoc.md). [`rustdoc`]: https://github.com/rust-lang/rust/tree/HEAD/src/tools/rustdoc ## From crate to clean In [`core.rs`] are two central items: the [`rustdoc::core::DocContext`] -`struct`, and the [`rustdoc::core::run_global_ctxt`] function. The latter is -where `rustdoc` calls out to `rustc` to compile a crate to the point where -`rustdoc` can take over. The former is a state container used when crawling -through a crate to gather its documentation. +`struct`, and the [`rustdoc::core::run_global_ctxt`] function. +The latter is where `rustdoc` calls out to `rustc` to compile a crate to the point where +`rustdoc` can take over. +The former is a state container used when crawling through a crate to gather its documentation. The main process of crate crawling is done in [`clean/mod.rs`] through several -functions with names that start with `clean_`. Each function accepts an `hir` -or `ty` data structure, and outputs a `clean` structure used by `rustdoc`. For -example, [this function for converting lifetimes]: +functions with names that start with `clean_`. +Each function accepts an `hir` +or `ty` data structure, and outputs a `clean` structure used by `rustdoc`. +For example, [this function for converting lifetimes]: ```rust,ignore fn clean_lifetime<'tcx>(lifetime: &hir::Lifetime, cx: &mut DocContext<'tcx>) -> Lifetime { @@ -34,17 +35,19 @@ fn clean_lifetime<'tcx>(lifetime: &hir::Lifetime, cx: &mut DocContext<'tcx>) -> ``` Also, `clean/mod.rs` defines the types for the "cleaned" [Abstract Syntax Tree -(`AST`)][ast] used later to render documentation pages. Each usually accompanies a +(`AST`)][ast] used later to render documentation pages. +Each usually accompanies a `clean_*` function that takes some [`AST`][ast] or [High-Level Intermediate -Representation (`HIR`)][hir] type from `rustc` and converts it into the -appropriate "cleaned" type. "Big" items like modules or associated items may +Representation (`HIR`)][hir] type from `rustc` and converts it into the appropriate "cleaned" type. +"Big" items like modules or associated items may have some extra processing in its `clean` function, but for the most part these -`impl`s are straightforward conversions. The "entry point" to this module is +`impl`s are straightforward conversions. +The "entry point" to this module is [`clean::utils::krate`][ck0], which is called by [`run_global_ctxt`]. The first step in [`clean::utils::krate`][ck1] is to invoke -[`visit_ast::RustdocVisitor`] to process the module tree into an intermediate -[`visit_ast::Module`]. This is the step that actually crawls the +[`visit_ast::RustdocVisitor`] to process the module tree into an intermediate [`visit_ast::Module`]. +This is the step that actually crawls the [`rustc_hir::Crate`], normalizing various aspects of name resolution, such as: * handling `#[doc(inline)]` and `#[doc(no_inline)]` @@ -57,14 +60,15 @@ The first step in [`clean::utils::krate`][ck1] is to invoke they're defined as a reexport or not After this step, `clean::krate` invokes [`clean_doc_module`], which actually -converts the `HIR` items to the cleaned [`AST`][ast]. This is also the step where cross- +converts the `HIR` items to the cleaned [`AST`][ast]. +This is also the step where cross- crate inlining is performed, which requires converting `rustc_middle` data structures into the cleaned [`AST`][ast]. The other major thing that happens in `clean/mod.rs` is the collection of doc comments and `#[doc=""]` attributes into a separate field of the [`Attributes`] -`struct`, present on anything that gets hand-written documentation. This makes it -easier to collect this documentation later in the process. +`struct`, present on anything that gets hand-written documentation. +This makes it easier to collect this documentation later in the process. The primary output of this process is a [`clean::types::Crate`] with a tree of [`Item`]s which describe the publicly-documentable items in the target crate. @@ -90,13 +94,14 @@ which describe the publicly-documentable items in the target crate. ### Passes Anything But a Gas Station (or: [Hot Potato](https://www.youtube.com/watch?v=WNFBIt5HxdY)) Before moving on to the next major step, a few important "passes" occur over -the cleaned [`AST`][ast]. Several of these passes are `lint`s and reports, but some of -them mutate or generate new items. +the cleaned [`AST`][ast]. +Several of these passes are `lint`s and reports, but some of them mutate or generate new items. These are all implemented in the [`librustdoc/passes`] directory, one file per pass. By default, all of these passes are run on a crate, but the ones regarding dropping private/hidden items can be bypassed by passing -`--document-private-items` to `rustdoc`. Note that unlike the previous set of [`AST`][ast] +`--document-private-items` to `rustdoc`. +Note that unlike the previous set of [`AST`][ast] transformations, the passes are run on the _cleaned_ crate. Here is the list of passes as of March 2023: @@ -105,8 +110,7 @@ Here is the list of passes as of March 2023: flag. - `check-doc-test-visibility` runs `doctest` visibility–related `lint`s. This pass - runs before `strip-private`, which is why it needs to be separate from - `run-lints`. + runs before `strip-private`, which is why it needs to be separate from `run-lints`. - `collect-intra-doc-links` resolves [intra-doc links](https://doc.rust-lang.org/nightly/rustdoc/write-documentation/linking-to-items-by-name.html). @@ -121,8 +125,8 @@ Here is the list of passes as of March 2023: - `bare_urls` detects links that are not linkified, e.g., in Markdown such as `Go to https://example.com/.` It suggests wrapping the link with angle brackets: - `Go to .` to linkify it. This is the code behind the `rustdoc::bare_urls` `lint`. + `Go to .` to linkify it. + This is the code behind the `rustdoc::bare_urls` `lint`. - `check_code_block_syntax` validates syntax inside Rust code blocks (```rust) @@ -131,33 +135,37 @@ Here is the list of passes as of March 2023: in doc comments. - `strip-hidden` and `strip-private` strip all `doc(hidden)` and private items - from the output. `strip-private` implies `strip-priv-imports`. Basically, the - goal is to remove items that are not relevant for public documentation. This - pass is skipped when `--document-hidden-items` is passed. + from the output. + `strip-private` implies `strip-priv-imports`. + Basically, the goal is to remove items that are not relevant for public documentation. + This pass is skipped when `--document-hidden-items` is passed. - `strip-priv-imports` strips all private import statements (`use`, `extern - crate`) from a crate. This is necessary because `rustdoc` will handle *public* + crate`) from a crate. + This is necessary because `rustdoc` will handle *public* imports by either inlining the item's documentation to the module or creating - a "Reexports" section with the import in it. The pass ensures that all of - these imports are actually relevant to documentation. It is technically - only run when `--document-private-items` is passed, but `strip-private` + a "Reexports" section with the import in it. + The pass ensures that all of these imports are actually relevant to documentation. + It is technically only run when `--document-private-items` is passed, but `strip-private` accomplishes the same thing. - `strip-private` strips all private items from a crate which cannot be seen - externally. This pass is skipped when `--document-private-items` is passed. + externally. + This pass is skipped when `--document-private-items` is passed. There is also a [`stripper`] module in `librustdoc/passes`, but it is a -collection of utility functions for the `strip-*` passes and is not a pass -itself. +collection of utility functions for the `strip-*` passes and is not a pass itself. [`librustdoc/passes`]: https://github.com/rust-lang/rust/tree/HEAD/src/librustdoc/passes [`stripper`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustdoc/passes/stripper/index.html ## From clean to HTML -This is where the "second phase" in `rustdoc` begins. This phase primarily lives +This is where the "second phase" in `rustdoc` begins. +This phase primarily lives in the [`librustdoc/formats`] and [`librustdoc/html`] folders, and it all starts with -[`formats::renderer::run_format`]. This code is responsible for setting up a type that +[`formats::renderer::run_format`]. +This code is responsible for setting up a type that `impl FormatRenderer`, which for `HTML` is [`Context`]. This structure contains methods that get called by `run_format` to drive the @@ -168,8 +176,8 @@ doc rendering, which includes: * `after_krate` generates other global resources like `all.html` In `item`, the "page rendering" occurs, via a mixture of [Askama] templates -and manual `write!()` calls, starting in [`html/layout.rs`]. The parts that have -not been converted to templates occur within a series of `std::fmt::Display` +and manual `write!()` calls, starting in [`html/layout.rs`]. +The parts that have not been converted to templates occur within a series of `std::fmt::Display` implementations and functions that pass around a `&mut std::fmt::Formatter`. The parts that actually generate `HTML` from the items and documentation start @@ -183,11 +191,13 @@ pieces like "how should I print a where clause as part of some other item". Whenever `rustdoc` comes across an item that should print hand-written documentation alongside, it calls out to [`html/markdown.rs`] which interfaces -with the Markdown parser. This is exposed as a series of types that wrap a -string of Markdown, and implement `fmt::Display` to emit `HTML` text. It takes -special care to enable certain features like footnotes and tables and add +with the Markdown parser. +This is exposed as a series of types that wrap a +string of Markdown, and implement `fmt::Display` to emit `HTML` text. +It takes special care to enable certain features like footnotes and tables and add syntax highlighting to Rust code blocks (via `html/highlight.rs`) before -running the Markdown parser. There's also a function [`find_codes`] which is +running the Markdown parser. +There's also a function [`find_codes`] which is called by `find_testable_codes` that specifically scans for Rust code blocks so the test-runner code can find all the `doctest`s in the crate. @@ -208,11 +218,11 @@ the test-runner code can find all the `doctest`s in the crate. [video]: https://www.youtube.com/watch?v=hOLAGYmUQV0 It's important to note that `rustdoc` can ask the compiler for type information -directly, even during `HTML` generation. This [didn't used to be the case], and +directly, even during `HTML` generation. +This [didn't used to be the case], and a lot of `rustdoc`'s architecture was designed around not doing that, but a `TyCtxt` is now passed to `formats::renderer::run_format`, which is used to -run generation for both `HTML` and the -(unstable as of March 2023) JSON format. +run generation for both `HTML` and the (unstable as of March 2023) JSON format. This change has allowed other changes to remove data from the "clean" [`AST`][ast] that can be easily derived from `TyCtxt` queries, and we'll usually accept @@ -222,18 +232,17 @@ is complicated from two other constraints that `rustdoc` runs under: * Docs can be generated for crates that don't actually pass type checking. This is used for generating docs that cover mutually-exclusive platform configurations, such as `libstd` having a single package of docs that - cover all supported operating systems. This means `rustdoc` has to be able - to generate docs from `HIR`. + cover all supported operating systems. + This means `rustdoc` has to be able to generate docs from `HIR`. * Docs can inline across crates. Since crate metadata doesn't contain `HIR`, it must be possible to generate inlined docs from the `rustc_middle` data. -The "clean" [`AST`][ast] acts as a common output format for both input formats. There -is also some data in clean that doesn't correspond directly to `HIR`, such as -synthetic `impl`s for auto traits and blanket `impl`s generated by the -`collect-trait-impls` pass. +The "clean" [`AST`][ast] acts as a common output format for both input formats. +There is also some data in clean that doesn't correspond directly to `HIR`, such as +synthetic `impl`s for auto traits and blanket `impl`s generated by the `collect-trait-impls` pass. -Some additional data is stored in -`html::render::context::{Context, SharedContext}`. These two types serve as +Some additional data is stored in `html::render::context::{Context, SharedContext}`. +These two types serve as ways to segregate `rustdoc`'s data for an eventual future with multithreaded doc generation, as well as just keeping things organized: @@ -250,19 +259,20 @@ generation, as well as just keeping things organized: ## Other tricks up its sleeve All this describes the process for generating `HTML` documentation from a Rust -crate, but there are couple other major modes that `rustdoc` runs in. It can also -be run on a standalone Markdown file, or it can run `doctest`s on Rust code or -standalone Markdown files. For the former, it shortcuts straight to +crate, but there are couple other major modes that `rustdoc` runs in. +It can also be run on a standalone Markdown file, or it can run `doctest`s on Rust code or +standalone Markdown files. +For the former, it shortcuts straight to `html/markdown.rs`, optionally including a mode which inserts a Table of Contents to the output `HTML`. For the latter, `rustdoc` runs a similar partial-compilation to get relevant documentation in `test.rs`, but instead of going through the full clean and -render process, it runs a much simpler crate walk to grab *just* the -hand-written documentation. Combined with the aforementioned +render process, it runs a much simpler crate walk to grab *just* the hand-written documentation. +Combined with the aforementioned "`find_testable_code`" in `html/markdown.rs`, it builds up a collection of -tests to run before handing them off to the test runner. One notable location -in `test.rs` is the function `make_test`, which is where hand-written +tests to run before handing them off to the test runner. +One notable location in `test.rs` is the function `make_test`, which is where hand-written `doctest`s get transformed into something that can be executed. Some extra reading about `make_test` can be found @@ -271,9 +281,8 @@ Some extra reading about `make_test` can be found ## Testing locally Some features of the generated `HTML` documentation might require local -storage to be used across pages, which doesn't work well without an `HTTP` -server. To test these features locally, you can run a local `HTTP` server, like -this: +storage to be used across pages, which doesn't work well without an `HTTP` server. +To test these features locally, you can run a local `HTTP` server, like this: ```bash $ ./x doc library @@ -281,8 +290,8 @@ $ ./x doc library $ python3 -m http.server -d build/[YOUR ARCH]/doc ``` -Now you can browse your documentation just like you would if it was hosted -on the internet. For example, the url for `std` will be `rust/std/`. +Now you can browse your documentation just like you would if it was hosted on the internet. +For example, the url for `std` will be `rust/std/`. ## See also From 8808940aeb71037b5c049441dc4b3f3867730eb2 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Wed, 5 Nov 2025 21:09:24 +0200 Subject: [PATCH 07/26] some text improvements --- src/doc/rustc-dev-guide/src/rustdoc-internals.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/rustdoc-internals.md b/src/doc/rustc-dev-guide/src/rustdoc-internals.md index 199a75e99d672..40bc4aeb2ce59 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc-internals.md +++ b/src/doc/rustc-dev-guide/src/rustdoc-internals.md @@ -61,9 +61,8 @@ This is the step that actually crawls the After this step, `clean::krate` invokes [`clean_doc_module`], which actually converts the `HIR` items to the cleaned [`AST`][ast]. -This is also the step where cross- -crate inlining is performed, which requires converting `rustc_middle` data -structures into the cleaned [`AST`][ast]. +This is also the step where cross-crate inlining is performed, +which requires converting `rustc_middle` data structures into the cleaned [`AST`][ast]. The other major thing that happens in `clean/mod.rs` is the collection of doc comments and `#[doc=""]` attributes into a separate field of the [`Attributes`] @@ -101,7 +100,7 @@ These are all implemented in the [`librustdoc/passes`] directory, one file per p By default, all of these passes are run on a crate, but the ones regarding dropping private/hidden items can be bypassed by passing `--document-private-items` to `rustdoc`. -Note that unlike the previous set of [`AST`][ast] +Note that, unlike the previous set of [`AST`][ast] transformations, the passes are run on the _cleaned_ crate. Here is the list of passes as of March 2023: From 2e6ff5826d4321f8eea3742f8d75160cef0047ec Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Wed, 5 Nov 2025 21:10:44 +0200 Subject: [PATCH 08/26] rustdoc json format remains unstable today --- src/doc/rustc-dev-guide/src/rustdoc-internals.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/doc/rustc-dev-guide/src/rustdoc-internals.md b/src/doc/rustc-dev-guide/src/rustdoc-internals.md index 40bc4aeb2ce59..0b75660a0c32c 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc-internals.md +++ b/src/doc/rustc-dev-guide/src/rustdoc-internals.md @@ -221,7 +221,7 @@ directly, even during `HTML` generation. This [didn't used to be the case], and a lot of `rustdoc`'s architecture was designed around not doing that, but a `TyCtxt` is now passed to `formats::renderer::run_format`, which is used to -run generation for both `HTML` and the (unstable as of March 2023) JSON format. +run generation for both `HTML` and the (unstable as of Nov 2025) JSON format. This change has allowed other changes to remove data from the "clean" [`AST`][ast] that can be easily derived from `TyCtxt` queries, and we'll usually accept From 717449cae220e733c5b398fee51b911ec6726d2f Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Wed, 5 Nov 2025 21:17:47 +0200 Subject: [PATCH 09/26] fix code block markers --- src/doc/rustc-dev-guide/src/rustdoc-internals.md | 2 +- src/doc/rustc-dev-guide/src/rustdoc.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/rustdoc-internals.md b/src/doc/rustc-dev-guide/src/rustdoc-internals.md index 0b75660a0c32c..989a77290b6b9 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc-internals.md +++ b/src/doc/rustc-dev-guide/src/rustdoc-internals.md @@ -283,7 +283,7 @@ Some features of the generated `HTML` documentation might require local storage to be used across pages, which doesn't work well without an `HTTP` server. To test these features locally, you can run a local `HTTP` server, like this: -```bash +```console $ ./x doc library # The documentation has been generated into `build/[YOUR ARCH]/doc`. $ python3 -m http.server -d build/[YOUR ARCH]/doc diff --git a/src/doc/rustc-dev-guide/src/rustdoc.md b/src/doc/rustc-dev-guide/src/rustdoc.md index 0e673bf4f58bb..47b18f4e7e52d 100644 --- a/src/doc/rustc-dev-guide/src/rustdoc.md +++ b/src/doc/rustc-dev-guide/src/rustdoc.md @@ -65,7 +65,7 @@ Note that literally all that does is call the `main()` that's in this crate's `l Rustdoc’s JavaScript and TypeScript are checked during CI by `eslint`, `es-check`, and `tsc` (not by compiletest). These run as part of the `tidy` job. -```bash +```console ./x.py test tidy --extra-checks=js ``` From 037ee61d60148303e668aeae9df7bdb9cd678193 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Wed, 5 Nov 2025 14:26:29 -0800 Subject: [PATCH 10/26] Fix some invalid language tags This fixes some code block tags that aren't using the correct language for rust. --- src/doc/rustc-dev-guide/src/borrow_check/two_phase_borrows.md | 2 +- src/doc/rustc-dev-guide/src/coroutine-closures.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/borrow_check/two_phase_borrows.md b/src/doc/rustc-dev-guide/src/borrow_check/two_phase_borrows.md index b77ae09465ca9..d38724335c960 100644 --- a/src/doc/rustc-dev-guide/src/borrow_check/two_phase_borrows.md +++ b/src/doc/rustc-dev-guide/src/borrow_check/two_phase_borrows.md @@ -15,7 +15,7 @@ two-phase borrow are: To give some examples: -```rust2018 +```rust,edition2018 // In the source code // Case 1: diff --git a/src/doc/rustc-dev-guide/src/coroutine-closures.md b/src/doc/rustc-dev-guide/src/coroutine-closures.md index 2617c824a3911..45d6a24a65998 100644 --- a/src/doc/rustc-dev-guide/src/coroutine-closures.md +++ b/src/doc/rustc-dev-guide/src/coroutine-closures.md @@ -167,7 +167,7 @@ We introduce a `AsyncFnKindHelper` trait which allows us to defer the question o This seems a bit roundabout and complex, and I admit that it is. But let's think of the "do nothing" alternative -- we could instead mark all `AsyncFn*` goals as ambiguous until upvar analysis, at which point we would know exactly what to put into the upvars of the coroutine we return. However, this is actually *very* detrimental to inference in the program, since it means that programs like this would not be valid: -```rust! +```rust,ignore let c = async || -> String { .. }; let s = c().await; // ^^^ If we can't project `<{c} as AsyncFn>::call()` to a coroutine, then the `IntoFuture::into_future` call inside of the `.await` stalls, and the type of `s` is left unconstrained as an infer var. From 5f04888e90bd1db373011ecf649b8765d881fa9a Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Wed, 5 Nov 2025 14:27:27 -0800 Subject: [PATCH 11/26] Add code spans around HTML-like text This was incorrectly interpreting the value as an HTML tag. This wraps it in an code span to avoid that. --- src/doc/rustc-dev-guide/src/offload/usage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/doc/rustc-dev-guide/src/offload/usage.md b/src/doc/rustc-dev-guide/src/offload/usage.md index 7d1a5c9e2e0ed..9d5839334b1a9 100644 --- a/src/doc/rustc-dev-guide/src/offload/usage.md +++ b/src/doc/rustc-dev-guide/src/offload/usage.md @@ -79,7 +79,7 @@ Now we generate the device code. Replace the target-cpu with the right code for RUSTFLAGS="-Ctarget-cpu=gfx90a --emit=llvm-bc,llvm-ir" cargo +offload build -Zunstable-options -r -v --target amdgcn-amd-amdhsa -Zbuild-std=core ``` -Now find the .ll under target/amdgcn-amd-amdhsa folder and copy it to a device.ll file (or adjust the file names below). +Now find the `.ll` under target/amdgcn-amd-amdhsa folder and copy it to a device.ll file (or adjust the file names below). If you work on an NVIDIA or Intel gpu, please adjust the names acordingly and open an issue to share your results (either if you succeed or fail). First we compile our .ll files (good for manual inspections) to .bc files and clean up leftover artifacts. The cleanup is important, otherwise caching might interfere on following runs. ``` From 5a4bb891f5b7b47e734ee18837d0c972f9bca1a8 Mon Sep 17 00:00:00 2001 From: Manuel Drehwald Date: Thu, 6 Nov 2025 14:09:04 -0500 Subject: [PATCH 12/26] the download-ci-llvm experiment turned out to be too unreliable --- src/doc/rustc-dev-guide/src/autodiff/installation.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/autodiff/installation.md b/src/doc/rustc-dev-guide/src/autodiff/installation.md index c9b6c85ab7a89..5300c12459a19 100644 --- a/src/doc/rustc-dev-guide/src/autodiff/installation.md +++ b/src/doc/rustc-dev-guide/src/autodiff/installation.md @@ -4,11 +4,11 @@ In the near future, `std::autodiff` should become available in nightly builds fo ## Build instructions -First you need to clone and configure the Rust repository: +First you need to clone and configure the Rust repository. Based on your preferences, you might also want to `--enable-clang` or `--enable-lld`. ```bash git clone git@github.com:rust-lang/rust cd rust -./configure --release-channel=nightly --enable-llvm-enzyme --enable-llvm-assertions --enable-option-checking --disable-docs --set llvm.download-ci-llvm=true +./configure --release-channel=nightly --enable-llvm-enzyme --enable-llvm-link-shared --enable-llvm-assertions --enable-ninja --enable-option-checking --disable-docs --set llvm.download-ci-llvm=false ``` Afterwards you can build rustc using: @@ -47,7 +47,7 @@ Then build rustc in a slightly altered way: ```bash git clone https://github.com/rust-lang/rust cd rust -./configure --release-channel=nightly --enable-llvm-enzyme --enable-llvm-assertions --enable-option-checking --disable-docs --set llvm.download-ci-llvm=true +./configure --release-channel=nightly --enable-llvm-enzyme --enable-llvm-link-shared --enable-llvm-assertions --enable-ninja --enable-option-checking --disable-docs --set llvm.download-ci-llvm=false ./x dist ``` We then copy the tarball to our host. The dockerid is the newest entry under `docker ps -a`. From 2f00812e063d40b81befe91afb4be0f0f213de50 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sun, 9 Nov 2025 19:39:11 +0200 Subject: [PATCH 13/26] typo --- src/doc/rustc-dev-guide/src/contributing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/doc/rustc-dev-guide/src/contributing.md b/src/doc/rustc-dev-guide/src/contributing.md index b8614e5a90f0c..e30819ebccae0 100644 --- a/src/doc/rustc-dev-guide/src/contributing.md +++ b/src/doc/rustc-dev-guide/src/contributing.md @@ -28,7 +28,7 @@ conditions that trigger the bug, or part of the error message if there is any. An example could be: **"impossible case reached" on lifetime inference for impl Trait in return position**. -Opening an issue is as easy as following [thi link][create an issue] and filling out the fields +Opening an issue is as easy as following [this link][create an issue] and filling out the fields in the appropriate provided template. ## Bug fixes or "normal" code changes From ca4abb57e868bd9d27b106e470e9dc703497e593 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Nov 2025 01:52:43 +0200 Subject: [PATCH 14/26] typo --- src/doc/rustc-dev-guide/src/tests/running.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/doc/rustc-dev-guide/src/tests/running.md b/src/doc/rustc-dev-guide/src/tests/running.md index 0a99c847cce7e..f67c9a2ce420d 100644 --- a/src/doc/rustc-dev-guide/src/tests/running.md +++ b/src/doc/rustc-dev-guide/src/tests/running.md @@ -8,7 +8,7 @@ subsection after on how to run a subset of tests.
Running plain `./x test` will build the stage 1 compiler and then run the whole -test suite. This not only include `tests/`, but also `library/`, `compiler/`, +test suite. This not only includes `tests/`, but also `library/`, `compiler/`, `src/tools/` package tests and more. You usually only want to run a subset of the test suites (or even a smaller set From ed094318d52e218777d7984555e93fa4ecf72bd1 Mon Sep 17 00:00:00 2001 From: The rustc-josh-sync Cronjob Bot Date: Mon, 10 Nov 2025 04:13:13 +0000 Subject: [PATCH 15/26] Prepare for merging from rust-lang/rust This updates the rust-version file to 8401398e1f14a24670ee1a3203713dc2f0f8b3a8. --- src/doc/rustc-dev-guide/rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/doc/rustc-dev-guide/rust-version b/src/doc/rustc-dev-guide/rust-version index 0e89b4ab6ac75..04d41c96f5c08 100644 --- a/src/doc/rustc-dev-guide/rust-version +++ b/src/doc/rustc-dev-guide/rust-version @@ -1 +1 @@ -c5dabe8cf798123087d094f06417f5a767ca73e8 +8401398e1f14a24670ee1a3203713dc2f0f8b3a8 From f4e2f0b10b47abda59f1b9ea2cc272aed5cbfbae Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Nov 2025 23:06:50 +0200 Subject: [PATCH 16/26] sembr src/getting-started.md --- .../rustc-dev-guide/src/getting-started.md | 85 +++++++++++-------- 1 file changed, 49 insertions(+), 36 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/getting-started.md b/src/doc/rustc-dev-guide/src/getting-started.md index 857e8ff51b03d..e270f5c755d9b 100644 --- a/src/doc/rustc-dev-guide/src/getting-started.md +++ b/src/doc/rustc-dev-guide/src/getting-started.md @@ -1,13 +1,14 @@ # Getting Started -Thank you for your interest in contributing to Rust! There are many ways to -contribute, and we appreciate all of them. +Thank you for your interest in contributing to Rust! +There are many ways to contribute, and we appreciate all of them. If this is your first time contributing, the [walkthrough] chapter can give you a good example of how a typical contribution would go. -This documentation is _not_ intended to be comprehensive; it is meant to be a -quick guide for the most useful things. For more information, [see this +This documentation is _not_ intended to be comprehensive; +it is meant to be a quick guide for the most useful things. +For more information, [see this chapter on how to build and run the compiler](./building/how-to-build-and-run.md). [internals]: https://internals.rust-lang.org @@ -36,26 +37,31 @@ questions about how the compiler works can go in [#t-compiler/help][z-help]. [z-help]: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp **Please ask questions!** A lot of people report feeling that they are "wasting -expert's time", but nobody on `t-compiler` feels this way. Contributors are -important to us. +expert's time", but nobody on `t-compiler` feels this way. +Contributors are important to us. Also, if you feel comfortable, prefer public topics, as this means others can -see the questions and answers, and perhaps even integrate them back into this -guide :) +see the questions and answers, and perhaps even integrate them back into this guide :) -**Tip**: If you're not a native English speaker and feel unsure about writing, try using a translator to help. But avoid using LLM tools that generate long, complex words. In daily teamwork, **simple and clear words** are best for easy understanding. Even small typos or grammar mistakes can make you seem more human, and people connect better with humans. +**Tip**: If you're not a native English speaker and feel unsure about writing, try using a translator to help. +But avoid using LLM tools that generate long, complex words. +In daily teamwork, **simple and clear words** are best for easy understanding. +Even small typos or grammar mistakes can make you seem more human, and people connect better with humans. ### Experts -Not all `t-compiler` members are experts on all parts of `rustc`; it's a -pretty large project. To find out who could have some expertise on +Not all `t-compiler` members are experts on all parts of `rustc`; +it's a pretty large project. +To find out who could have some expertise on different parts of the compiler, [consult triagebot assign groups][map]. -The sections that start with `[assign*` in `triagebot.toml` file. +The sections that start with `[assign*` in `triagebot.toml` file. But also, feel free to ask questions even if you can't figure out who to ping. Another way to find experts for a given part of the compiler is to see who has made recent commits. For example, to find people who have recently worked on name resolution since the 1.68.2 release, -you could run `git shortlog -n 1.68.2.. compiler/rustc_resolve/`. Ignore any commits starting with +you could run `git shortlog -n 1.68.2.. +compiler/rustc_resolve/`. +Ignore any commits starting with "Rollup merge" or commits by `@bors` (see [CI contribution procedures](./contributing.md#ci) for more information about these commits). @@ -64,8 +70,7 @@ more information about these commits). ### Etiquette We do ask that you be mindful to include as much useful information as you can -in your question, but we recognize this can be hard if you are unfamiliar with -contributing to Rust. +in your question, but we recognize this can be hard if you are unfamiliar with contributing to Rust. Just pinging someone without providing any context can be a bit annoying and just create noise, so we ask that you be mindful of the fact that the @@ -74,13 +79,16 @@ just create noise, so we ask that you be mindful of the fact that the ## What should I work on? The Rust project is quite large and it can be difficult to know which parts of the project need -help, or are a good starting place for beginners. Here are some suggested starting places. +help, or are a good starting place for beginners. +Here are some suggested starting places. ### Easy or mentored issues If you're looking for somewhere to start, check out the following [issue -search][help-wanted-search]. See the [Triage] for an explanation of these labels. You can also try -filtering the search to areas you're interested in. For example: +search][help-wanted-search]. +See the [Triage] for an explanation of these labels. +You can also try filtering the search to areas you're interested in. +For example: - `repo:rust-lang/rust-clippy` will only show clippy issues - `label:T-compiler` will only show issues related to the compiler @@ -94,8 +102,9 @@ See below for how to find work that isn't labelled. ### Recurring work -Some work is too large to be done by a single person. In this case, it's common to have "Tracking -issues" to co-ordinate the work between contributors. Here are some example tracking issues where +Some work is too large to be done by a single person. +In this case, it's common to have "Tracking issues" to co-ordinate the work between contributors. +Here are some example tracking issues where it's easy to pick up work without a large time commitment: - *Add recurring work items here.* @@ -105,8 +114,8 @@ If you find more recurring work, please feel free to add it here! ### Clippy issues The [Clippy] project has spent a long time making its contribution process as friendly to newcomers -as possible. Consider working on it first to get familiar with the process and the compiler -internals. +as possible. +Consider working on it first to get familiar with the process and the compiler internals. See [the Clippy contribution guide][clippy-contributing] for instructions on getting started. @@ -116,19 +125,22 @@ See [the Clippy contribution guide][clippy-contributing] for instructions on get ### Diagnostic issues Many diagnostic issues are self-contained and don't need detailed background knowledge of the -compiler. You can see a list of diagnostic issues [here][diagnostic-issues]. +compiler. +You can see a list of diagnostic issues [here][diagnostic-issues]. [diagnostic-issues]: https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Aopen+label%3AA-diagnostics+no%3Aassignee ### Picking up abandoned pull requests Sometimes, contributors send a pull request, but later find out that they don't have enough -time to work on it, or they simply are not interested in it anymore. Such PRs are often -eventually closed and they receive the `S-inactive` label. You could try to examine some of -these PRs and pick up the work. You can find the list of such PRs [here][abandoned-prs]. +time to work on it, or they simply are not interested in it anymore. +Such PRs are often eventually closed and they receive the `S-inactive` label. +You could try to examine some of these PRs and pick up the work. +You can find the list of such PRs [here][abandoned-prs]. If the PR has been implemented in some other way in the meantime, the `S-inactive` label -should be removed from it. If not, and it seems that there is still interest in the change, +should be removed from it. +If not, and it seems that there is still interest in the change, you can try to rebase the pull request on top of the latest `master` branch and send a new pull request, continuing the work on the feature. @@ -136,7 +148,8 @@ pull request, continuing the work on the feature. ### Writing tests -Issues that have been resolved but do not have a regression test are marked with the `E-needs-test` label. Writing unit tests is a low-risk, lower-priority task that offers new contributors a great opportunity to familiarize themselves with the testing infrastructure and contribution workflow. +Issues that have been resolved but do not have a regression test are marked with the `E-needs-test` label. +Writing unit tests is a low-risk, lower-priority task that offers new contributors a great opportunity to familiarize themselves with the testing infrastructure and contribution workflow. ### Contributing to std (standard library) @@ -147,28 +160,28 @@ See [std-dev-guide](https://std-dev-guide.rust-lang.org/). There are a bunch of other projects that you can contribute to outside of the `rust-lang/rust` repo, including `cargo`, `miri`, `rustup`, and many others. -These repos might have their own contributing guidelines and procedures. Many -of them are owned by working groups. For more info, see the documentation in those repos' READMEs. +These repos might have their own contributing guidelines and procedures. +Many of them are owned by working groups. +For more info, see the documentation in those repos' READMEs. ### Other ways to contribute There are a bunch of other ways you can contribute, especially if you don't feel comfortable jumping straight into the large `rust-lang/rust` codebase. -The following tasks are doable without much background knowledge but are -incredibly helpful: +The following tasks are doable without much background knowledge but are incredibly helpful: - [Writing documentation][wd]: if you are feeling a bit more intrepid, you could try - to read a part of the code and write doc comments for it. This will help you - to learn some part of the compiler while also producing a useful artifact! + to read a part of the code and write doc comments for it. + This will help you to learn some part of the compiler while also producing a useful artifact! - [Triaging issues][triage]: categorizing, replicating, and minimizing issues is very helpful to the Rust maintainers. - [Working groups][wg]: there are a bunch of working groups on a wide variety of rust-related things. - Answer questions on [users.rust-lang.org][users], or on [Stack Overflow][so]. - Participate in the [RFC process](https://github.com/rust-lang/rfcs). - Find a [requested community library][community-library], build it, and publish - it to [Crates.io](http://crates.io). Easier said than done, but very, very - valuable! + it to [Crates.io](http://crates.io). + Easier said than done, but very, very valuable! [users]: https://users.rust-lang.org/ [so]: http://stackoverflow.com/questions/tagged/rust From 49143e5be8638a5f529697a3e9415162b6f99ac4 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Nov 2025 23:24:32 +0200 Subject: [PATCH 17/26] use chapter name instead of text that does not fit well --- src/doc/rustc-dev-guide/src/getting-started.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/getting-started.md b/src/doc/rustc-dev-guide/src/getting-started.md index e270f5c755d9b..9f24f5fa44661 100644 --- a/src/doc/rustc-dev-guide/src/getting-started.md +++ b/src/doc/rustc-dev-guide/src/getting-started.md @@ -8,8 +8,8 @@ how a typical contribution would go. This documentation is _not_ intended to be comprehensive; it is meant to be a quick guide for the most useful things. -For more information, [see this -chapter on how to build and run the compiler](./building/how-to-build-and-run.md). +For more information, +see [How to build and run the compiler](building/how-to-build-and-run.md). [internals]: https://internals.rust-lang.org [rust-zulip]: https://rust-lang.zulipchat.com From 7a7608cd0e47e10d455aef320cff60b423e393d4 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Nov 2025 23:25:41 +0200 Subject: [PATCH 18/26] overlong --- src/doc/rustc-dev-guide/src/getting-started.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/doc/rustc-dev-guide/src/getting-started.md b/src/doc/rustc-dev-guide/src/getting-started.md index 9f24f5fa44661..cae00306047fd 100644 --- a/src/doc/rustc-dev-guide/src/getting-started.md +++ b/src/doc/rustc-dev-guide/src/getting-started.md @@ -149,7 +149,9 @@ pull request, continuing the work on the feature. ### Writing tests Issues that have been resolved but do not have a regression test are marked with the `E-needs-test` label. -Writing unit tests is a low-risk, lower-priority task that offers new contributors a great opportunity to familiarize themselves with the testing infrastructure and contribution workflow. +Writing unit tests is a low-risk, +lower-priority task that offers new contributors a great opportunity to familiarize themselves +with the testing infrastructure and contribution workflow. ### Contributing to std (standard library) From b4a91c396beb2b4e7e8430bf9b6fdf93a7894a9d Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Nov 2025 23:26:33 +0200 Subject: [PATCH 19/26] a double dot is not end of sentence --- src/doc/rustc-dev-guide/ci/sembr/src/main.rs | 5 ++++- src/doc/rustc-dev-guide/src/getting-started.md | 3 +-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/doc/rustc-dev-guide/ci/sembr/src/main.rs b/src/doc/rustc-dev-guide/ci/sembr/src/main.rs index c056f68c31d78..99e90e9081980 100644 --- a/src/doc/rustc-dev-guide/ci/sembr/src/main.rs +++ b/src/doc/rustc-dev-guide/ci/sembr/src/main.rs @@ -27,7 +27,8 @@ static REGEX_IGNORE: LazyLock = static REGEX_IGNORE_END: LazyLock = LazyLock::new(|| Regex::new(r"(\.|\?|;|!)$").unwrap()); static REGEX_IGNORE_LINK_TARGETS: LazyLock = LazyLock::new(|| Regex::new(r"^\[.+\]: ").unwrap()); -static REGEX_SPLIT: LazyLock = LazyLock::new(|| Regex::new(r"(\.|[^r]\?|;|!)\s+").unwrap()); +static REGEX_SPLIT: LazyLock = + LazyLock::new(|| Regex::new(r"([^\.]\.|[^r]\?|;|!)\s+").unwrap()); fn main() -> Result<()> { let cli = Cli::parse(); @@ -183,6 +184,7 @@ ignore E.g. too some code. block ``` sentence with *italics* should not be ignored. truly. +git log main.. compiler "; let expected = "\ # some. heading @@ -203,6 +205,7 @@ some code. block ``` sentence with *italics* should not be ignored. truly. +git log main.. compiler "; assert_eq!(expected, comply(original)); } diff --git a/src/doc/rustc-dev-guide/src/getting-started.md b/src/doc/rustc-dev-guide/src/getting-started.md index cae00306047fd..64abd94c69e27 100644 --- a/src/doc/rustc-dev-guide/src/getting-started.md +++ b/src/doc/rustc-dev-guide/src/getting-started.md @@ -59,8 +59,7 @@ But also, feel free to ask questions even if you can't figure out who to ping. Another way to find experts for a given part of the compiler is to see who has made recent commits. For example, to find people who have recently worked on name resolution since the 1.68.2 release, -you could run `git shortlog -n 1.68.2.. -compiler/rustc_resolve/`. +you could run `git shortlog -n 1.68.2.. compiler/rustc_resolve/`. Ignore any commits starting with "Rollup merge" or commits by `@bors` (see [CI contribution procedures](./contributing.md#ci) for more information about these commits). From 299a13b06024441b56cf831982af4698db419d49 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Nov 2025 23:36:16 +0200 Subject: [PATCH 20/26] sembr tests/running.md --- src/doc/rustc-dev-guide/src/tests/running.md | 151 ++++++++++--------- 1 file changed, 76 insertions(+), 75 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/tests/running.md b/src/doc/rustc-dev-guide/src/tests/running.md index f67c9a2ce420d..3071af3b98a68 100644 --- a/src/doc/rustc-dev-guide/src/tests/running.md +++ b/src/doc/rustc-dev-guide/src/tests/running.md @@ -1,19 +1,19 @@ # Running tests -You can run the entire test collection using `x`. But note that running the -*entire* test collection is almost never what you want to do during local -development because it takes a really long time. For local development, see the -subsection after on how to run a subset of tests. +You can run the entire test collection using `x`. +But note that running the *entire* test collection is almost never what you want to do during local +development because it takes a really long time. +For local development, see the subsection after on how to run a subset of tests.
-Running plain `./x test` will build the stage 1 compiler and then run the whole -test suite. This not only includes `tests/`, but also `library/`, `compiler/`, +Running plain `./x test` will build the stage 1 compiler and then run the whole test suite. +This not only includes `tests/`, but also `library/`, `compiler/`, `src/tools/` package tests and more. You usually only want to run a subset of the test suites (or even a smaller set -of tests than that) which you expect will exercise your changes. PR CI exercises -a subset of test collections, and merge queue CI will exercise all of the test +of tests than that) which you expect will exercise your changes. +PR CI exercises a subset of test collections, and merge queue CI will exercise all of the test collection.
@@ -22,8 +22,8 @@ collection. ./x test ``` -The test results are cached and previously successful tests are `ignored` during -testing. The stdout/stderr contents as well as a timestamp file for every test +The test results are cached and previously successful tests are `ignored` during testing. +The stdout/stderr contents as well as a timestamp file for every test can be found under `build//test/` for the given ``. To force-rerun a test (e.g. in case the test runner fails to notice a change) you can use the `--force-rerun` CLI option. @@ -40,8 +40,8 @@ notice a change) you can use the `--force-rerun` CLI option. ## Running a subset of the test suites -When working on a specific PR, you will usually want to run a smaller set of -tests. For example, a good "smoke test" that can be used after modifying rustc +When working on a specific PR, you will usually want to run a smaller set of tests. +For example, a good "smoke test" that can be used after modifying rustc to see if things are generally working correctly would be to exercise the `ui` test suite ([`tests/ui`]): @@ -49,9 +49,8 @@ test suite ([`tests/ui`]): ./x test tests/ui ``` -Of course, the choice of test suites is -somewhat arbitrary, and may not suit the task you are doing. For example, if you -are hacking on debuginfo, you may be better off with the debuginfo test suite: +Of course, the choice of test suites is somewhat arbitrary, and may not suit the task you are doing. +For example, if you are hacking on debuginfo, you may be better off with the debuginfo test suite: ```text ./x test tests/debuginfo @@ -77,8 +76,8 @@ Likewise, you can test a single file by passing its path: ./x test tests/ui/const-generics/const-test.rs ``` -`x` doesn't support running a single tool test by passing its path yet. You'll -have to use the `--test-args` argument as described +`x` doesn't support running a single tool test by passing its path yet. +You'll have to use the `--test-args` argument as described [below](#running-an-individual-test). ```text @@ -97,8 +96,8 @@ have to use the `--test-args` argument as described ./x test --stage 0 library/std ``` -Note that this only runs tests on `std`; if you want to test `core` or other -crates, you have to specify those explicitly. +Note that this only runs tests on `std`; +if you want to test `core` or other crates, you have to specify those explicitly. ### Run the tidy script and tests on the standard library @@ -117,8 +116,8 @@ you avoid having to run tests for components you did not change at all.
-Note that bors only runs the tests with the full stage 2 build; therefore, while -the tests **usually** work fine with stage 1, there are some limitations. +Note that bors only runs the tests with the full stage 2 build; +therefore, while the tests **usually** work fine with stage 1, there are some limitations.
@@ -128,8 +127,8 @@ the tests **usually** work fine with stage 1, there are some limitations. ./x test --stage 2 ``` -
-You almost never need to do this; CI will run these tests for you. +
You almost never need to do this; +CI will run these tests for you.
## Run unit tests on the compiler/library @@ -140,7 +139,8 @@ You may want to run unit tests on a specific file with following: ./x test compiler/rustc_data_structures/src/thin_vec/tests.rs ``` -But unfortunately, it's impossible. You should invoke the following instead: +But unfortunately, it's impossible. +You should invoke the following instead: ```text ./x test compiler/rustc_data_structures/ --test-args thin_vec @@ -149,9 +149,9 @@ But unfortunately, it's impossible. You should invoke the following instead: ## Running an individual test Another common thing that people want to do is to run an **individual test**, -often the test they are trying to fix. As mentioned earlier, you may pass the -full file path to achieve this, or alternatively one may invoke `x` with the -`--test-args` option: +often the test they are trying to fix. +As mentioned earlier, you may pass the +full file path to achieve this, or alternatively one may invoke `x` with the `--test-args` option: ```text ./x test tests/ui --test-args issue-1234 @@ -159,14 +159,13 @@ full file path to achieve this, or alternatively one may invoke `x` with the Under the hood, the test runner invokes the standard Rust test runner (the same one you get with `#[test]`), so this command would wind up filtering for tests -that include "issue-1234" in the name. Thus, `--test-args` is a good way to run -a collection of related tests. +that include "issue-1234" in the name. +Thus, `--test-args` is a good way to run a collection of related tests. ## Passing arguments to `rustc` when running tests It can sometimes be useful to run some tests with specific compiler arguments, -without using `RUSTFLAGS` (during development of unstable features, with `-Z` -flags, for example). +without using `RUSTFLAGS` (during development of unstable features, with `-Z` flags, for example). This can be done with `./x test`'s `--compiletest-rustc-args` option, to pass additional arguments to the compiler when building the tests. @@ -176,8 +175,7 @@ additional arguments to the compiler when building the tests. If you have changed the compiler's output intentionally, or you are making a new test, you can pass `--bless` to the test subcommand. -As an example, -if some tests in `tests/ui` are failing, you can run this command: +As an example, if some tests in `tests/ui` are failing, you can run this command: ```text ./x test tests/ui --bless @@ -192,8 +190,9 @@ just like when running the tests without the `--bless` flag. There are a few options for running tests: * `bootstrap.toml` has the `rust.verbose-tests` option. If `false`, each test will - print a single dot (the default). If `true`, the name of every test will be - printed. This is equivalent to the `--quiet` option in the [Rust test + print a single dot (the default). + If `true`, the name of every test will be printed. + This is equivalent to the `--quiet` option in the [Rust test harness](https://doc.rust-lang.org/rustc/tests/). * The environment variable `RUST_TEST_THREADS` can be set to the number of concurrent threads to use for testing. @@ -202,24 +201,24 @@ There are a few options for running tests: Pass UI tests now have three modes, `check-pass`, `build-pass` and `run-pass`. When `--pass $mode` is passed, these tests will be forced to run under the given -`$mode` unless the directive `//@ ignore-pass` exists in the test file. For -example, you can run all the tests in `tests/ui` as `check-pass`: +`$mode` unless the directive `//@ ignore-pass` exists in the test file. +For example, you can run all the tests in `tests/ui` as `check-pass`: ```text ./x test tests/ui --pass check ``` -By passing `--pass $mode`, you can reduce the testing time. For each mode, -please see [Controlling pass/fail +By passing `--pass $mode`, you can reduce the testing time. +For each mode, please see [Controlling pass/fail expectations](ui.md#controlling-passfail-expectations). ## Running tests with different "compare modes" -UI tests may have different output depending on certain "modes" that the -compiler is in. For example, when using the Polonius mode, a test `foo.rs` will +UI tests may have different output depending on certain "modes" that the compiler is in. +For example, when using the Polonius mode, a test `foo.rs` will first look for expected output in `foo.polonius.stderr`, falling back to the -usual `foo.stderr` if not found. The following will run the UI test suite in -Polonius mode: +usual `foo.stderr` if not found. +The following will run the UI test suite in Polonius mode: ```text ./x test tests/ui --compare-mode=polonius @@ -229,8 +228,8 @@ See [Compare modes](compiletest.md#compare-modes) for more details. ## Running tests manually -Sometimes it's easier and faster to just run the test by hand. Most tests are -just `.rs` files, so after [creating a rustup +Sometimes it's easier and faster to just run the test by hand. +Most tests are just `.rs` files, so after [creating a rustup toolchain](../building/how-to-build-and-run.md#creating-a-rustup-toolchain), you can do something like: @@ -238,17 +237,19 @@ can do something like: rustc +stage1 tests/ui/issue-1234.rs ``` -This is much faster, but doesn't always work. For example, some tests include +This is much faster, but doesn't always work. +For example, some tests include directives that specify specific compiler flags, or which rely on other crates, and they may not run the same without those options. ## Running tests on a remote machine Tests may be run on a remote machine (e.g. to test builds for a different -architecture). This is done using `remote-test-client` on the build machine to +architecture). +This is done using `remote-test-client` on the build machine to send test programs to `remote-test-server` running on the remote machine. -`remote-test-server` executes the test programs and sends the results back to -the build machine. `remote-test-server` provides *unauthenticated remote code +`remote-test-server` executes the test programs and sends the results back to the build machine. +`remote-test-server` provides *unauthenticated remote code execution* so be careful where it is used. To do this, first build `remote-test-server` for the remote machine, e.g. for @@ -258,13 +259,12 @@ RISC-V ./x build src/tools/remote-test-server --target riscv64gc-unknown-linux-gnu ``` -The binary will be created at -`./build/host/stage2-tools/$TARGET_ARCH/release/remote-test-server`. Copy this -over to the remote machine. +The binary will be created at `./build/host/stage2-tools/$TARGET_ARCH/release/remote-test-server`. +Copy this over to the remote machine. On the remote machine, run the `remote-test-server` with the `--bind -0.0.0.0:12345` flag (and optionally `-v` for verbose output). Output should look -like this: +0.0.0.0:12345` flag (and optionally `-v` for verbose output). +Output should look like this: ```text $ ./remote-test-server -v --bind 0.0.0.0:12345 @@ -273,12 +273,13 @@ listening on 0.0.0.0:12345! ``` Note that binding the server to 0.0.0.0 will allow all hosts able to reach your -machine to execute arbitrary code on your machine. We strongly recommend either +machine to execute arbitrary code on your machine. +We strongly recommend either setting up a firewall to block external access to port 12345, or to use a more restrictive IP address when binding. -You can test if the `remote-test-server` is working by connecting to it and -sending `ping\n`. It should reply `pong`: +You can test if the `remote-test-server` is working by connecting to it and sending `ping\n`. +It should reply `pong`: ```text $ nc $REMOTE_IP 12345 @@ -287,8 +288,8 @@ pong ``` To run tests using the remote runner, set the `TEST_DEVICE_ADDR` environment -variable then use `x` as usual. For example, to run `ui` tests for a RISC-V -machine with the IP address `1.2.3.4` use +variable then use `x` as usual. +For example, to run `ui` tests for a RISC-V machine with the IP address `1.2.3.4` use ```text export TEST_DEVICE_ADDR="1.2.3.4:12345" @@ -315,34 +316,34 @@ run "/tmp/work/test1018/a" [...] ``` -Tests are built on the machine running `x` not on the remote machine. Tests -which fail to build unexpectedly (or `ui` tests producing incorrect build +Tests are built on the machine running `x` not on the remote machine. +Tests which fail to build unexpectedly (or `ui` tests producing incorrect build output) may fail without ever running on the remote machine. ## Testing on emulators -Some platforms are tested via an emulator for architectures that aren't readily -available. For architectures where the standard library is well supported and +Some platforms are tested via an emulator for architectures that aren't readily available. +For architectures where the standard library is well supported and the host operating system supports TCP/IP networking, see the above instructions for testing on a remote machine (in this case the remote machine is emulated). -There is also a set of tools for orchestrating running the tests within the -emulator. Platforms such as `arm-android` and `arm-unknown-linux-gnueabihf` are -set up to automatically run the tests under emulation on GitHub Actions. The -following will take a look at how a target's tests are run under emulation. +There is also a set of tools for orchestrating running the tests within the emulator. +Platforms such as `arm-android` and `arm-unknown-linux-gnueabihf` are +set up to automatically run the tests under emulation on GitHub Actions. +The following will take a look at how a target's tests are run under emulation. -The Docker image for [armhf-gnu] includes [QEMU] to emulate the ARM CPU -architecture. Included in the Rust tree are the tools [remote-test-client] and +The Docker image for [armhf-gnu] includes [QEMU] to emulate the ARM CPU architecture. +Included in the Rust tree are the tools [remote-test-client] and [remote-test-server] which are programs for sending test programs and libraries -to the emulator, and running the tests within the emulator, and reading the -results. The Docker image is set up to launch `remote-test-server` and the +to the emulator, and running the tests within the emulator, and reading the results. +The Docker image is set up to launch `remote-test-server` and the build tools use `remote-test-client` to communicate with the server to coordinate running tests (see [src/bootstrap/src/core/build_steps/test.rs]). -To run on the iOS/tvOS/watchOS/visionOS simulator, we can similarly treat it as -a "remote" machine. A curious detail here is that the network is shared between -the simulator instance and the host macOS, so we can use the local loopback -address `127.0.0.1`. Something like the following should work: +To run on the iOS/tvOS/watchOS/visionOS simulator, we can similarly treat it as a "remote" machine. +A curious detail here is that the network is shared between +the simulator instance and the host macOS, so we can use the local loopback address `127.0.0.1`. +Something like the following should work: ```sh # Build the test server for the iOS simulator: From b271341781450b45d09a3d4f6a4efcbf9a549ae1 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Tue, 11 Nov 2025 00:05:40 +0200 Subject: [PATCH 21/26] sembr: handle text inside div blocks The hackiness of this tool is getting more and more embarassing --- src/doc/rustc-dev-guide/ci/sembr/src/main.rs | 18 ++++++++++++++++++ src/doc/rustc-dev-guide/src/tests/running.md | 3 ++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/doc/rustc-dev-guide/ci/sembr/src/main.rs b/src/doc/rustc-dev-guide/ci/sembr/src/main.rs index 99e90e9081980..b50190c78b7b2 100644 --- a/src/doc/rustc-dev-guide/ci/sembr/src/main.rs +++ b/src/doc/rustc-dev-guide/ci/sembr/src/main.rs @@ -137,6 +137,7 @@ fn lengthen_lines(content: &str, limit: usize) -> String { let mut new_content = content.clone(); let mut new_n = 0; let mut in_code_block = false; + let mut in_html_div = false; let mut skip_next = false; for (n, line) in content.iter().enumerate() { if skip_next { @@ -150,6 +151,17 @@ fn lengthen_lines(content: &str, limit: usize) -> String { in_code_block = !in_code_block; continue; } + if line.trim_start().starts_with(" You almost never need to do this; +
+You almost never need to do this; CI will run these tests for you.
From 241e8909a0c75501444b1e8c80a98a18135894d8 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Tue, 11 Nov 2025 00:07:09 +0200 Subject: [PATCH 22/26] tests/running.md: minor improvements --- src/doc/rustc-dev-guide/src/tests/running.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/tests/running.md b/src/doc/rustc-dev-guide/src/tests/running.md index 58177342be1d9..e887bac0fc0ec 100644 --- a/src/doc/rustc-dev-guide/src/tests/running.md +++ b/src/doc/rustc-dev-guide/src/tests/running.md @@ -253,8 +253,8 @@ send test programs to `remote-test-server` running on the remote machine. `remote-test-server` provides *unauthenticated remote code execution* so be careful where it is used. -To do this, first build `remote-test-server` for the remote machine, e.g. for -RISC-V +To do this, first build `remote-test-server` for the remote machine +(using RISC-V as an example): ```text ./x build src/tools/remote-test-server --target riscv64gc-unknown-linux-gnu @@ -264,11 +264,11 @@ The binary will be created at `./build/host/stage2-tools/$TARGET_ARCH/release/re Copy this over to the remote machine. On the remote machine, run the `remote-test-server` with the `--bind -0.0.0.0:12345` flag (and optionally `-v` for verbose output). +0.0.0.0:12345` flag (and optionally `--verbose` flag). Output should look like this: ```text -$ ./remote-test-server -v --bind 0.0.0.0:12345 +$ ./remote-test-server --verbose --bind 0.0.0.0:12345 starting test server listening on 0.0.0.0:12345! ``` From 87c406469ff12a9d8f6ca2eca6470edfb8800ba7 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Tue, 11 Nov 2025 00:10:42 +0200 Subject: [PATCH 23/26] fix code block language marker --- src/doc/rustc-dev-guide/src/tests/running.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/doc/rustc-dev-guide/src/tests/running.md b/src/doc/rustc-dev-guide/src/tests/running.md index e887bac0fc0ec..07218e3662ff0 100644 --- a/src/doc/rustc-dev-guide/src/tests/running.md +++ b/src/doc/rustc-dev-guide/src/tests/running.md @@ -386,7 +386,7 @@ On the [wasm32-wasip1 target support page] a minimum version is specified that y Some cmake commands that take a while and give a lot of very concerning c++ warnings... Then, in `bootstrap.toml`, point to the sysroot like so: -``` +```toml [target.wasm32-wasip1] wasi-root = "/build/sysroot/install/share/wasi-sysroot" ``` From 6e14d2a7b1839cccb338776b879292c79f2fe1d1 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Tue, 11 Nov 2025 00:12:04 +0200 Subject: [PATCH 24/26] a more suitable marker --- src/doc/rustc-dev-guide/src/tests/running.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/tests/running.md b/src/doc/rustc-dev-guide/src/tests/running.md index 07218e3662ff0..48ce9d8ad185b 100644 --- a/src/doc/rustc-dev-guide/src/tests/running.md +++ b/src/doc/rustc-dev-guide/src/tests/running.md @@ -267,7 +267,7 @@ On the remote machine, run the `remote-test-server` with the `--bind 0.0.0.0:12345` flag (and optionally `--verbose` flag). Output should look like this: -```text +```console $ ./remote-test-server --verbose --bind 0.0.0.0:12345 starting test server listening on 0.0.0.0:12345! @@ -282,7 +282,7 @@ restrictive IP address when binding. You can test if the `remote-test-server` is working by connecting to it and sending `ping\n`. It should reply `pong`: -```text +```console $ nc $REMOTE_IP 12345 ping pong From 7675d7a1f001e5673ee4daf68688b6102a7f9a30 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Tue, 11 Nov 2025 00:27:59 +0200 Subject: [PATCH 25/26] sembr sanitizers.md --- src/doc/rustc-dev-guide/src/sanitizers.md | 41 ++++++++++++----------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/sanitizers.md b/src/doc/rustc-dev-guide/src/sanitizers.md index 53df27412f051..2bcab6fc0dba9 100644 --- a/src/doc/rustc-dev-guide/src/sanitizers.md +++ b/src/doc/rustc-dev-guide/src/sanitizers.md @@ -10,8 +10,7 @@ The rustc compiler contains support for following sanitizers: * [Hardware-assisted AddressSanitizer][clang-hwasan] a tool similar to AddressSanitizer but based on partial hardware assistance. * [KernelControlFlowIntegrity][clang-kcfi] LLVM Kernel Control Flow Integrity - (KCFI) provides forward-edge control flow protection for operating systems - kernels. + (KCFI) provides forward-edge control flow protection for operating systems kernels. * [LeakSanitizer][clang-lsan] a run-time memory leak detector. * [MemorySanitizer][clang-msan] a detector of uninitialized reads. * [ThreadSanitizer][clang-tsan] a fast data race detector. @@ -19,16 +18,16 @@ The rustc compiler contains support for following sanitizers: ## How to use the sanitizers? To enable a sanitizer compile with `-Z sanitizer=...` option, where value is one -of `address`, `cfi`, `hwaddress`, `kcfi`, `leak`, `memory` or `thread`. For more -details on how to use sanitizers please refer to the sanitizer flag in [the +of `address`, `cfi`, `hwaddress`, `kcfi`, `leak`, `memory` or `thread`. +For more details on how to use sanitizers please refer to the sanitizer flag in [the unstable book](https://doc.rust-lang.org/unstable-book/). ## How are sanitizers implemented in rustc? The implementation of sanitizers (except CFI) relies almost entirely on LLVM. The rustc is an integration point for LLVM compile time instrumentation passes -and runtime libraries. Highlight of the most important aspects of the -implementation: +and runtime libraries. +Highlight of the most important aspects of the implementation: * The sanitizer runtime libraries are part of the [compiler-rt] project, and [will be built][sanitizer-build] on [supported targets][sanitizer-targets] @@ -43,23 +42,26 @@ implementation: * During LLVM code generation, the functions intended for instrumentation are [marked][sanitizer-attribute] with appropriate LLVM attribute: - `SanitizeAddress`, `SanitizeHWAddress`, `SanitizeMemory`, or - `SanitizeThread`. By default all functions are instrumented, but this + `SanitizeAddress`, `SanitizeHWAddress`, `SanitizeMemory`, or `SanitizeThread`. + By default all functions are instrumented, but this behaviour can be changed with `#[sanitize(xyz = "on|off|")]`. * The decision whether to perform instrumentation or not is possible only at a - function granularity. In the cases were those decision differ between + function granularity. + In the cases were those decision differ between functions it might be necessary to inhibit inlining, both at [MIR level][inline-mir] and [LLVM level][inline-llvm]. * The LLVM IR generated by rustc is instrumented by [dedicated LLVM - passes][sanitizer-pass], different for each sanitizer. Instrumentation - passes are invoked after optimization passes. + passes][sanitizer-pass], different for each sanitizer. + Instrumentation passes are invoked after optimization passes. * When producing an executable, the sanitizer specific runtime library is - [linked in][sanitizer-link]. The libraries are searched for in the target - libdir. First relative to the overridden system root and subsequently - relative to the default system root. Fall-back to the default system root + [linked in][sanitizer-link]. + The libraries are searched for in the target libdir. + First relative to the overridden system root and subsequently + relative to the default system root. + Fall-back to the default system root ensures that sanitizer runtimes remain available when using sysroot overrides constructed by cargo `-Z build-std` or xargo. @@ -80,10 +82,9 @@ Sanitizers are validated by code generation tests in [`tests/ui/sanitizer/`][test-ui] directory. Testing sanitizer functionality requires the sanitizer runtimes (built when -`sanitizer = true` in `bootstrap.toml`) and target providing support for particular -sanitizer. When sanitizer is unsupported on given target, sanitizers tests will -be ignored. This behaviour is controlled by compiletest `needs-sanitizer-*` -directives. +`sanitizer = true` in `bootstrap.toml`) and target providing support for particular sanitizer. +When sanitizer is unsupported on given target, sanitizers tests will be ignored. +This behaviour is controlled by compiletest `needs-sanitizer-*` directives. [test-cg]: https://github.com/rust-lang/rust/tree/HEAD/tests/codegen-llvm [test-ui]: https://github.com/rust-lang/rust/tree/HEAD/tests/ui/sanitizer @@ -93,8 +94,8 @@ directives. To enable a sanitizer on a new target which is already supported by LLVM: 1. Include the sanitizer in the list of `supported_sanitizers` in [the target - definition][target-definition]. `rustc --target .. -Zsanitizer=..` should now - recognize sanitizer as supported. + definition][target-definition]. + `rustc --target .. -Zsanitizer=..` should now recognize sanitizer as supported. 2. [Build the runtime for the target and include it in the libdir.][sanitizer-targets] 3. [Teach compiletest that your target now supports the sanitizer.][compiletest-definition] Tests marked with `needs-sanitizer-*` should now run on the target. From de1b7754d15b093b3dd449e02b2287e163fd8318 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Tue, 11 Nov 2025 00:38:30 +0200 Subject: [PATCH 26/26] some text improvements --- src/doc/rustc-dev-guide/src/sanitizers.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/src/doc/rustc-dev-guide/src/sanitizers.md b/src/doc/rustc-dev-guide/src/sanitizers.md index 2bcab6fc0dba9..7b84335e3b906 100644 --- a/src/doc/rustc-dev-guide/src/sanitizers.md +++ b/src/doc/rustc-dev-guide/src/sanitizers.md @@ -19,8 +19,10 @@ The rustc compiler contains support for following sanitizers: To enable a sanitizer compile with `-Z sanitizer=...` option, where value is one of `address`, `cfi`, `hwaddress`, `kcfi`, `leak`, `memory` or `thread`. -For more details on how to use sanitizers please refer to the sanitizer flag in [the -unstable book](https://doc.rust-lang.org/unstable-book/). +For more details on how to use sanitizers, +please refer to the sanitizer flag in [The Unstable Book]. + +[The Unstable Book]: https://doc.rust-lang.org/unstable-book ## How are sanitizers implemented in rustc? @@ -43,13 +45,13 @@ Highlight of the most important aspects of the implementation: * During LLVM code generation, the functions intended for instrumentation are [marked][sanitizer-attribute] with appropriate LLVM attribute: `SanitizeAddress`, `SanitizeHWAddress`, `SanitizeMemory`, or `SanitizeThread`. - By default all functions are instrumented, but this + By default, all functions are instrumented, but this behaviour can be changed with `#[sanitize(xyz = "on|off|")]`. * The decision whether to perform instrumentation or not is possible only at a function granularity. In the cases were those decision differ between - functions it might be necessary to inhibit inlining, both at [MIR + functions, it might be necessary to inhibit inlining, both at [MIR level][inline-mir] and [LLVM level][inline-llvm]. * The LLVM IR generated by rustc is instrumented by [dedicated LLVM @@ -59,8 +61,8 @@ Highlight of the most important aspects of the implementation: * When producing an executable, the sanitizer specific runtime library is [linked in][sanitizer-link]. The libraries are searched for in the target libdir. - First relative to the overridden system root and subsequently - relative to the default system root. + First, the search is relative to the overridden system root, and subsequently, + it is relative to the default system root. Fall-back to the default system root ensures that sanitizer runtimes remain available when using sysroot overrides constructed by cargo `-Z build-std` or xargo. @@ -82,20 +84,20 @@ Sanitizers are validated by code generation tests in [`tests/ui/sanitizer/`][test-ui] directory. Testing sanitizer functionality requires the sanitizer runtimes (built when -`sanitizer = true` in `bootstrap.toml`) and target providing support for particular sanitizer. -When sanitizer is unsupported on given target, sanitizers tests will be ignored. +`sanitizer = true` in `bootstrap.toml`) and target providing support for particular a sanitizer. +When a sanitizer is unsupported on a given target, sanitizer tests will be ignored. This behaviour is controlled by compiletest `needs-sanitizer-*` directives. [test-cg]: https://github.com/rust-lang/rust/tree/HEAD/tests/codegen-llvm [test-ui]: https://github.com/rust-lang/rust/tree/HEAD/tests/ui/sanitizer -## Enabling sanitizer on a new target +## Enabling a sanitizer on a new target To enable a sanitizer on a new target which is already supported by LLVM: 1. Include the sanitizer in the list of `supported_sanitizers` in [the target definition][target-definition]. - `rustc --target .. -Zsanitizer=..` should now recognize sanitizer as supported. + `rustc --target .. -Zsanitizer=..` should now recognize the sanitizer as supported. 2. [Build the runtime for the target and include it in the libdir.][sanitizer-targets] 3. [Teach compiletest that your target now supports the sanitizer.][compiletest-definition] Tests marked with `needs-sanitizer-*` should now run on the target.