Skip to content

Commit

Permalink
docs: document new suggest-tests tool
Browse files Browse the repository at this point in the history
  • Loading branch information
Ezrashaw committed Apr 4, 2023
1 parent fca8af6 commit 7a8cade
Show file tree
Hide file tree
Showing 3 changed files with 73 additions and 0 deletions.
1 change: 1 addition & 0 deletions src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
- [Test headers](./tests/headers.md)
- [Performance testing](./tests/perf.md)
- [Crater](./tests/crater.md)
- [Suggest tests tool](./tests/suggest-tests.md)
- [Debugging the compiler](./compiler-debugging.md)
- [Using the tracing/logging instrumentation](./tracing.md)
- [Profiling the compiler](./profiling.md)
Expand Down
17 changes: 17 additions & 0 deletions src/building/suggested.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,23 @@ the problem. A nice side-effect of this style is that you are left
with a fairly fine-grained set of commits at the end, all of which
build and pass tests. This often helps reviewing.

## `x suggest`

The `x suggest` subcommand suggests (and runs) a subset of the extensive
`rust-lang/rust` tests based on files you have changed. This is especially useful
for new contributors who have not mastered the arcane `x` flags yet and more
experienced contributors as a shorthand for reducing mental effort. In all cases
it is useful not to run the full tests (which can take on the order of tens of
minutes) and just run a subset which are relevant to your changes. For example,
running `tidy` and `linkchecker` is useful when editing Markdown files, whereas UI
tests are much less likely to be helpful. While `x suggest` is a useful tool, it
does not guarantee perfect coverage (just as PR CI isn't a substitute for bors).
See the [dedicated chapter](../tests/suggest-tests.md) for more information and
contribution instructions.

Please note that `x suggest` is in a beta state currently and the tests that it
will suggest are limited.

## Configuring `rustup` to use nightly

Some parts of the bootstrap process uses pinned, nightly versions of tools like
Expand Down
55 changes: 55 additions & 0 deletions src/tests/suggest-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Suggest tests tool

This chapter is about the internals of and contribution instructions for the
`suggest-tests` tool. For a high-level overview of the tool, see
[this section](../building/suggested.md#x-suggest). This tool is currently in a
beta state and is tracked by [this](https://github.com/rust-lang/rust/issues/109933)
issue on Github. Currently the number of tests it will suggest are very limited
in scope, we are looking to expand this (contributions welcome!).

## Internals

The tool is defined in a separate crate ([`src/tools/suggest-tests`](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests))
which outputs suggestions which are parsed by a shim in bootstrap
([`src/bootstrap/suggest.rs`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/suggest.rs)).
The only notable thing the bootstrap shim does is (when invoked with the
`--run` flag) use bootstrap's internal mechanisms to create a new `Builder` and
uses it to invoke the suggested commands. The `suggest-tests` crate is where the
fun happens, two kinds of suggestions are defined: "static" and "dynamic"
suggestions.

### Static suggestions

Defined [here](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/static_suggestions.rs).
Static suggestions are simple: they are just [globs](https://crates.io/crates/glob)
which map to a `x` command. In `suggest-tests`, this is implemented with a
simple `macro_rules` macro.

### Dynamic suggestions

Defined [here](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/dynamic_suggestions.rs).
These are more complicated than static suggestions and are implemented as
functions with the following signature: `fn(&Path) -> Vec<Suggestion>`. In
other words, each suggestion takes a path to a modified file and (after running
arbitrary Rust code) can return any number of suggestions, or none. Dynamic
suggestions are useful for situations where fine-grained control over
suggestions is needed. For example, modifications to the `compiler/xyz/` path
should trigger the `x test compiler/xyz` suggestion. In the future, dynamic
suggestions might even read file contents to determine if (what) tests should
run.

## Adding a suggestion

The following steps should serve as a rough guide to add suggestions to
`suggest-tests` (very welcome!):

1. Determine the rules for your suggestion. Is it simple and operates only on
a single path or does it match globs? Does it need fine-grained control over
the resulting command or does "one size fit all"?
2. Based on the previous step, decide if your suggestion should be implemented
as either static or dynamic.
3. Implement the suggestion. If it is dynamic then a test is highly recommended,
to verify that your logic is correct and to give an example of the suggestion.
See the [tests.rs](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/static_suggestions.rs)
file.
4. Open a PR implementing your suggestion. **(TODO: add example PR)**

0 comments on commit 7a8cade

Please sign in to comment.