Skip to content

Commit

Permalink
Auto merge of rust-lang#3590 - RalfJung:miri-test-target, r=RalfJung
Browse files Browse the repository at this point in the history 
make MIRI_TEST_TARGET and RUSTC_BLESS entirely an internal thing

They are just used to communicate between `./miri test`/`./x.py test miri` and the test harness, but should not be used by users.
  • Loading branch information
@bors
bors committed May 9, 2024
2 parents 42d9b68 + cb44843 commit 3028864
Show file tree
Hide file tree
Showing 7 changed files with 179 additions and 146 deletions.
47 changes: 42 additions & 5 deletions src/tools/miri/CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,8 @@ You can (cross-)run the entire test suite using:
```

`./miri test FILTER` only runs those tests that contain `FILTER` in their filename (including the
base directory, e.g. `./miri test fail` will run all compile-fail tests). Multiple filters
are supported: `./miri test FILTER1 FILTER2`.
base directory, e.g. `./miri test fail` will run all compile-fail tests). Multiple filters are
supported: `./miri test FILTER1 FILTER2` runs all tests that contain either string.

#### Fine grained logging

Expand Down Expand Up @@ -139,9 +139,8 @@ and then you can use it as if it was installed by `rustup` as a component of the
in the `miri` toolchain's sysroot to prevent conflicts with other toolchains.
The Miri binaries in the `cargo` bin directory (usually `~/.cargo/bin`) are managed by rustup.

There's a test for the cargo wrapper in the `test-cargo-miri` directory; run
`./run-test.py` in there to execute it. Like `./miri test`, this respects the
`MIRI_TEST_TARGET` environment variable to execute the test for another target.
There's a test for the cargo wrapper in the `test-cargo-miri` directory; run `./run-test.py` in
there to execute it. You can pass `--target` to execute the test for another target.

### Using a modified standard library

Expand Down Expand Up @@ -287,3 +286,41 @@ https. Add the following to your `.gitconfig`:
[url "git@github.com:"]
pushInsteadOf = https://github.com/
```

## Internal environment variables

The following environment variables are *internal* and must not be used by
anyone but Miri itself. They are used to communicate between different Miri
binaries, and as such worth documenting:

* `CARGO_EXTRA_FLAGS` is understood by `./miri` and passed to all host cargo invocations.
* `MIRI_BE_RUSTC` can be set to `host` or `target`. It tells the Miri driver to
actually not interpret the code but compile it like rustc would. With `target`, Miri sets
some compiler flags to prepare the code for interpretation; with `host`, this is not done.
This environment variable is useful to be sure that the compiled `rlib`s are compatible
with Miri.
* `MIRI_CALLED_FROM_SETUP` is set during the Miri sysroot build,
which will re-invoke `cargo-miri` as the `rustc` to use for this build.
* `MIRI_CALLED_FROM_RUSTDOC` when set to any value tells `cargo-miri` that it is
running as a child process of `rustdoc`, which invokes it twice for each doc-test
and requires special treatment, most notably a check-only build before interpretation.
This is set by `cargo-miri` itself when running as a `rustdoc`-wrapper.
* `MIRI_CWD` when set to any value tells the Miri driver to change to the given
directory after loading all the source files, but before commencing
interpretation. This is useful if the interpreted program wants a different
working directory at run-time than at build-time.
* `MIRI_LOCAL_CRATES` is set by `cargo-miri` to tell the Miri driver which
crates should be given special treatment in diagnostics, in addition to the
crate currently being compiled.
* `MIRI_ORIG_RUSTDOC` is set and read by different phases of `cargo-miri` to remember the
value of `RUSTDOC` from before it was overwritten.
* `MIRI_REPLACE_LIBRS_IF_NOT_TEST` when set to any value enables a hack that helps bootstrap
run the standard library tests in Miri.
* `MIRI_TEST_TARGET` is set by `./miri test` (and `./x.py test miri`) to tell the test harness about
the chosen target.
* `MIRI_VERBOSE` when set to any value tells the various `cargo-miri` phases to
perform verbose logging.
* `MIRI_HOST_SYSROOT` is set by bootstrap to tell `cargo-miri` which sysroot to use for *host*
operations.
* `RUSTC_BLESS` is set by `./miri test` (and `./x.py test miri`) to indicate bless-mode to the test
harness.
36 changes: 0 additions & 36 deletions src/tools/miri/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -463,50 +463,14 @@ by all intended entry points, i.e. `cargo miri` and `./miri {test,run}`):
* `MIRI_SYSROOT` indicates the sysroot to use. When using `cargo miri`, this skips the automatic
setup -- only set this if you do not want to use the automatically created sysroot. When invoking
`cargo miri setup`, this indicates where the sysroot will be put.
* `MIRI_TEST_TARGET` (recognized by `./miri {test,run}`) indicates which target
architecture to test against. The `--target` flag may be used for the same
purpose.
* `MIRI_TEST_THREADS` (recognized by `./miri test`): set the number of threads to use for running tests.
By default, the number of cores is used.
* `MIRI_NO_STD` makes sure that the target's sysroot is built without libstd. This allows testing
and running no_std programs. (Miri has a heuristic to detect no-std targets based on the target
name; this environment variable is only needed when that heuristic fails.)
* `RUSTC_BLESS` (recognized by `./miri test` and `cargo-miri-test/run-test.py`): overwrite all
`stderr` and `stdout` files instead of checking whether the output matches.
* `MIRI_SKIP_UI_CHECKS` (recognized by `./miri test`): don't check whether the
`stderr` or `stdout` files match the actual output.

The following environment variables are *internal* and must not be used by
anyone but Miri itself. They are used to communicate between different Miri
binaries, and as such worth documenting:

* `MIRI_BE_RUSTC` can be set to `host` or `target`. It tells the Miri driver to
actually not interpret the code but compile it like rustc would. With `target`, Miri sets
some compiler flags to prepare the code for interpretation; with `host`, this is not done.
This environment variable is useful to be sure that the compiled `rlib`s are compatible
with Miri.
* `MIRI_CALLED_FROM_SETUP` is set during the Miri sysroot build,
which will re-invoke `cargo-miri` as the `rustc` to use for this build.
* `MIRI_CALLED_FROM_RUSTDOC` when set to any value tells `cargo-miri` that it is
running as a child process of `rustdoc`, which invokes it twice for each doc-test
and requires special treatment, most notably a check-only build before interpretation.
This is set by `cargo-miri` itself when running as a `rustdoc`-wrapper.
* `MIRI_CWD` when set to any value tells the Miri driver to change to the given
directory after loading all the source files, but before commencing
interpretation. This is useful if the interpreted program wants a different
working directory at run-time than at build-time.
* `MIRI_LOCAL_CRATES` is set by `cargo-miri` to tell the Miri driver which
crates should be given special treatment in diagnostics, in addition to the
crate currently being compiled.
* `MIRI_ORIG_RUSTDOC` is set and read by different phases of `cargo-miri` to remember the
value of `RUSTDOC` from before it was overwritten.
* `MIRI_REPLACE_LIBRS_IF_NOT_TEST` when set to any value enables a hack that helps bootstrap
run the standard library tests in Miri.
* `MIRI_VERBOSE` when set to any value tells the various `cargo-miri` phases to
perform verbose logging.
* `MIRI_HOST_SYSROOT` is set by bootstrap to tell `cargo-miri` which sysroot to use for *host*
operations.

[testing-miri]: CONTRIBUTING.md#testing-the-miri-driver

## Miri `extern` functions
Expand Down
67 changes: 35 additions & 32 deletions src/tools/miri/ci/ci.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,24 +31,26 @@ time ./miri build --all-targets # the build that all the `./miri test` below wil
endgroup

# Run tests. Recognizes these variables:
# - MIRI_TEST_TARGET: the target to test. Empty for host target.
# - TEST_TARGET: the target to test. Empty for host target.
# - GC_STRESS: if non-empty, run the GC stress test for the main test suite.
# - MIR_OPT: if non-empty, re-run test `pass` tests with mir-opt-level=4
# - MANY_SEEDS: if set to N, run the "many-seeds" tests N times
# - TEST_BENCH: if non-empty, check that the benchmarks all build
# - CARGO_MIRI_ENV: if non-empty, set some env vars and config to potentially confuse cargo-miri
function run_tests {
if [ -n "${MIRI_TEST_TARGET-}" ]; then
begingroup "Testing foreign architecture $MIRI_TEST_TARGET"
if [ -n "${TEST_TARGET-}" ]; then
begingroup "Testing foreign architecture $TEST_TARGET"
TARGET_FLAG="--target $TEST_TARGET"
else
begingroup "Testing host architecture"
TARGET_FLAG=""
fi

## ui test suite
if [ -n "${GC_STRESS-}" ]; then
time MIRIFLAGS="${MIRIFLAGS-} -Zmiri-provenance-gc=1" ./miri test
time MIRIFLAGS="${MIRIFLAGS-} -Zmiri-provenance-gc=1" ./miri test $TARGET_FLAG
else
time ./miri test
time ./miri test $TARGET_FLAG
fi

## advanced tests
Expand All @@ -59,17 +61,17 @@ function run_tests {
# them. Also error locations change so we don't run the failing tests.
# We explicitly enable debug-assertions here, they are disabled by -O but we have tests
# which exist to check that we panic on debug assertion failures.
time MIRIFLAGS="${MIRIFLAGS-} -O -Zmir-opt-level=4 -Cdebug-assertions=yes" MIRI_SKIP_UI_CHECKS=1 ./miri test tests/{pass,panic}
time MIRIFLAGS="${MIRIFLAGS-} -O -Zmir-opt-level=4 -Cdebug-assertions=yes" MIRI_SKIP_UI_CHECKS=1 ./miri test $TARGET_FLAG tests/{pass,panic}
fi
if [ -n "${MANY_SEEDS-}" ]; then
# Also run some many-seeds tests.
time for FILE in tests/many-seeds/*.rs; do
./miri run "--many-seeds=0..$MANY_SEEDS" "$FILE"
./miri run "--many-seeds=0..$MANY_SEEDS" $TARGET_FLAG "$FILE"
done
fi
if [ -n "${TEST_BENCH-}" ]; then
# Check that the benchmarks build and run, but only once.
time HYPERFINE="hyperfine -w0 -r1" ./miri bench
time HYPERFINE="hyperfine -w0 -r1" ./miri bench $TARGET_FLAG
fi

## test-cargo-miri
Expand All @@ -91,7 +93,7 @@ function run_tests {
echo 'build.rustc-wrapper = "thisdoesnotexist"' > .cargo/config.toml
fi
# Run the actual test
time ${PYTHON} test-cargo-miri/run-test.py
time ${PYTHON} test-cargo-miri/run-test.py $TARGET_FLAG
# Clean up
unset RUSTC MIRI
rm -rf .cargo
Expand All @@ -100,17 +102,18 @@ function run_tests {
}

function run_tests_minimal {
if [ -n "${MIRI_TEST_TARGET-}" ]; then
begingroup "Testing MINIMAL foreign architecture $MIRI_TEST_TARGET: only testing $@"
if [ -n "${TEST_TARGET-}" ]; then
begingroup "Testing MINIMAL foreign architecture $TEST_TARGET: only testing $@"
TARGET_FLAG="--target $TEST_TARGET"
else
echo "run_tests_minimal requires MIRI_TEST_TARGET to be set"
echo "run_tests_minimal requires TEST_TARGET to be set"
exit 1
fi

time ./miri test "$@"
time ./miri test $TARGET_FLAG "$@"

# Ensure that a small smoke test of cargo-miri works.
time cargo miri run --manifest-path test-cargo-miri/no-std-smoke/Cargo.toml --target ${MIRI_TEST_TARGET-$HOST_TARGET}
time cargo miri run --manifest-path test-cargo-miri/no-std-smoke/Cargo.toml $TARGET_FLAG

endgroup
}
Expand All @@ -126,33 +129,33 @@ case $HOST_TARGET in
# Extra tier 1
# With reduced many-seed count to avoid spending too much time on that.
# (All OSes and ABIs are run with 64 seeds at least once though via the macOS runner.)
MANY_SEEDS=16 MIRI_TEST_TARGET=i686-unknown-linux-gnu run_tests
MANY_SEEDS=16 MIRI_TEST_TARGET=aarch64-unknown-linux-gnu run_tests
MANY_SEEDS=16 MIRI_TEST_TARGET=x86_64-apple-darwin run_tests
MANY_SEEDS=16 MIRI_TEST_TARGET=x86_64-pc-windows-gnu run_tests
MANY_SEEDS=16 TEST_TARGET=i686-unknown-linux-gnu run_tests
MANY_SEEDS=16 TEST_TARGET=aarch64-unknown-linux-gnu run_tests
MANY_SEEDS=16 TEST_TARGET=x86_64-apple-darwin run_tests
MANY_SEEDS=16 TEST_TARGET=x86_64-pc-windows-gnu run_tests
;;
aarch64-apple-darwin)
# Host (tier 2)
GC_STRESS=1 MIR_OPT=1 MANY_SEEDS=64 TEST_BENCH=1 CARGO_MIRI_ENV=1 run_tests
# Extra tier 1
MANY_SEEDS=64 MIRI_TEST_TARGET=i686-pc-windows-gnu run_tests
MANY_SEEDS=64 MIRI_TEST_TARGET=x86_64-pc-windows-msvc CARGO_MIRI_ENV=1 run_tests
MANY_SEEDS=64 TEST_TARGET=i686-pc-windows-gnu run_tests
MANY_SEEDS=64 TEST_TARGET=x86_64-pc-windows-msvc CARGO_MIRI_ENV=1 run_tests
# Extra tier 2
MIRI_TEST_TARGET=arm-unknown-linux-gnueabi run_tests
MIRI_TEST_TARGET=s390x-unknown-linux-gnu run_tests # big-endian architecture of choice
TEST_TARGET=arm-unknown-linux-gnueabi run_tests
TEST_TARGET=s390x-unknown-linux-gnu run_tests # big-endian architecture of choice
# Partially supported targets (tier 2)
VERY_BASIC="integer vec string btreemap" # common things we test on all of them (if they have std), requires no target-specific shims
BASIC="$VERY_BASIC hello hashmap alloc align" # ensures we have the shims for stdout and basic data structures
MIRI_TEST_TARGET=x86_64-unknown-freebsd run_tests_minimal $BASIC panic/panic concurrency/simple atomic threadname libc-mem libc-misc libc-random libc-time fs env num_cpus
MIRI_TEST_TARGET=i686-unknown-freebsd run_tests_minimal $BASIC panic/panic concurrency/simple atomic threadname libc-mem libc-misc libc-random libc-time fs env num_cpus
MIRI_TEST_TARGET=x86_64-unknown-illumos run_tests_minimal $VERY_BASIC hello panic/panic concurrency/simple pthread-sync libc-mem libc-misc libc-random
MIRI_TEST_TARGET=x86_64-pc-solaris run_tests_minimal $VERY_BASIC hello panic/panic concurrency/simple pthread-sync libc-mem libc-misc libc-random
MIRI_TEST_TARGET=aarch64-linux-android run_tests_minimal $VERY_BASIC hello panic/panic
MIRI_TEST_TARGET=wasm32-wasi run_tests_minimal $VERY_BASIC wasm
MIRI_TEST_TARGET=wasm32-unknown-unknown run_tests_minimal $VERY_BASIC wasm
MIRI_TEST_TARGET=thumbv7em-none-eabihf run_tests_minimal no_std
TEST_TARGET=x86_64-unknown-freebsd run_tests_minimal $BASIC panic/panic concurrency/simple atomic threadname libc-mem libc-misc libc-random libc-time fs env num_cpus
TEST_TARGET=i686-unknown-freebsd run_tests_minimal $BASIC panic/panic concurrency/simple atomic threadname libc-mem libc-misc libc-random libc-time fs env num_cpus
TEST_TARGET=x86_64-unknown-illumos run_tests_minimal $VERY_BASIC hello panic/panic concurrency/simple pthread-sync libc-mem libc-misc libc-random
TEST_TARGET=x86_64-pc-solaris run_tests_minimal $VERY_BASIC hello panic/panic concurrency/simple pthread-sync libc-mem libc-misc libc-random
TEST_TARGET=aarch64-linux-android run_tests_minimal $VERY_BASIC hello panic/panic
TEST_TARGET=wasm32-wasi run_tests_minimal $VERY_BASIC wasm
TEST_TARGET=wasm32-unknown-unknown run_tests_minimal $VERY_BASIC wasm
TEST_TARGET=thumbv7em-none-eabihf run_tests_minimal no_std
# Custom target JSON file
MIRI_TEST_TARGET=tests/avr.json MIRI_NO_STD=1 run_tests_minimal no_std
TEST_TARGET=tests/avr.json MIRI_NO_STD=1 run_tests_minimal no_std
;;
i686-pc-windows-msvc)
# Host
Expand All @@ -162,7 +165,7 @@ case $HOST_TARGET in
# Extra tier 1
# We really want to ensure a Linux target works on a Windows host,
# and a 64bit target works on a 32bit host.
MIRI_TEST_TARGET=x86_64-unknown-linux-gnu run_tests
TEST_TARGET=x86_64-unknown-linux-gnu run_tests
;;
*)
echo "FATAL: unknown host target: $HOST_TARGET"
Expand Down
Loading

0 comments on commit 3028864

Please sign in to comment.