debug-assertions are implicitly disabled when running cargo test with opt-level=3 set from rustflags

[benny-n]

Problem

As the title states, it looks like having rustflags = ["-C", "opt-level=3"] in your config.toml implicitly disables debug assertions, even when running cargo test, which should have them enabled by default.

Some background:

I am currently working on a project where I had a config.toml looking something like this:

[target.x86_64-unknown-linux-gnu]
rustflags = [
    # ...
    "-C",
    "opt-level=3",
    # ...
]

My task was to create a lightweight profile without optimizations to enjoy faster compilation times. However, I had to ensure that tests were still compiled with opt-level=3 because it heavily affected their total run time.
So the first thing I did was create a [profile.test] entry and move opt-level=3 from rustflags to it:

[target.x86_64-unknown-linux-gnu]
rustflags = [
    # ...
    # "-C",
    # "opt-level=3",
    # ...
]

[profile.test]
opt-level = 3

Surprisingly, after moving it, I observed some tests failing on debug assertions. It took a while to figure out, but after some digging, I found out that the code was never compiled with debug assertions to begin with, even though "-C debug-assertions=false" was NOT part of the rustflags array.

Since debug assertions should be enabled by default for tests, and since passing opt-level=3 from a profile or from rustflags should be pretty much the same, this behavior seemed somewhat confusing to me.

Please do let me know if I'm missing something.

Steps

1. cargo init and edit main.rs with:

fn main() {}

#[test]
fn debug_assert_false() {
    debug_assert!(false)
}

2. create a .cargo/config.toml and edit with:

[target.x86_64-unknown-linux-gnu] # also reproduces for aarch64-apple-darwin
rustflags = ["-C", "opt-level=3"]

3. run cargo test and observe the test passing.
4. edit .cargo/config.toml with:

# [target.x86_64-unknown-linux-gnu] # also reproduces for aarch64-apple-darwin
# rustflags = ["-C", "opt-level=3"]

[profile.test]
opt-level = 3

5. run cargo test again and observe the test failing.

Version

cargo 1.78.0 (54d8815d0 2024-03-26)
release: 1.78.0
commit-hash: 54d8815d04fa3816edc207bbc4dd36bf18014dbc
commit-date: 2024-03-26
host: x86_64-unknown-linux-gnu
libgit2: 1.7.2 (sys:0.18.2 vendored)
libcurl: 8.6.0-DEV (sys:0.4.72+curl-8.6.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Ubuntu 22.04 (jammy) [64-bit]

[epage]

You don't need to comment out rustflags; just the presence of absence of profile.test makes this pass/fail

Looking at verbose output

$ cargo clean && cargo test -vvv
     Removed 45 files, 12.2MiB total
   Compiling cargo-14033 v0.1.0 (/home/epage/src/personal/dump/cargo-14033)
     Running `CARGO=/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_BIN_NAME=cargo-1403
3 CARGO_CRATE_NAME=cargo_14033 CARGO_MANIFEST_DIR=/home/epage/src/personal/dump/cargo-14033 CARGO_PKG_AUTHORS='' CARGO
_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=cargo-14033 CA
RGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0
CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='
/home/epage/src/personal/dump/cargo-14033/target/debug/deps:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux
-gnu/lib' /home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name cargo_14033 --edition=
2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=118
--emit=dep-info,link -C opt-level=3 -C embed-bitcode=no -C debuginfo=2 -C debug-assertions=on --test -C metadata=b4ef5
f5c9b16634d -C extra-filename=-b4ef5f5c9b16634d --out-dir /home/epage/src/personal/dump/cargo-14033/target/debug/deps
-C incremental=/home/epage/src/personal/dump/cargo-14033/target/debug/incremental -L dependency=/home/epage/src/person
al/dump/cargo-14033/target/debug/deps -C opt-level=3`
    Finished `test` profile [optimized + debuginfo] target(s) in 0.13s
     Running `CARGO=/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_MANIFEST_DIR=/home/
epage/src/personal/dump/cargo-14033 CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICE
NSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=cargo-14033 CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST
_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARG
O_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/epage/src/personal/dump/cargo-14033/target/debug/deps:/home/epage/src/pers
onal/dump/cargo-14033/target/debug:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-u
nknown-linux-gnu/lib:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib' /home/epage/src/personal/dump
/cargo-14033/target/debug/deps/cargo_14033-b4ef5f5c9b16634d`

running 1 test
test debug_assert_false ... FAILED

failures:

---- debug_assert_false stdout ----
thread 'debug_assert_false' panicked at src/main.rs:7:5:
assertion failed: false
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    debug_assert_false

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

error: test failed, to rerun pass `--bin cargo-14033`

$ cargo clean && cargo test -vvv
     Removed 24 files, 6.1MiB total
   Compiling cargo-14033 v0.1.0 (/home/epage/src/personal/dump/cargo-14033)
     Running `CARGO=/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_BIN_NAME=cargo-1403
3 CARGO_CRATE_NAME=cargo_14033 CARGO_MANIFEST_DIR=/home/epage/src/personal/dump/cargo-14033 CARGO_PKG_AUTHORS='' CARGO
_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=cargo-14033 CA
RGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0
CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='
/home/epage/src/personal/dump/cargo-14033/target/debug/deps:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux
-gnu/lib' /home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name cargo_14033 --edition=
2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=118
--emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=1ab2997d3cc74d9b -C extra-filename=-1ab2997
d3cc74d9b --out-dir /home/epage/src/personal/dump/cargo-14033/target/debug/deps -C incremental=/home/epage/src/persona
l/dump/cargo-14033/target/debug/incremental -L dependency=/home/epage/src/personal/dump/cargo-14033/target/debug/deps
-C opt-level=3`
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.13s
     Running `CARGO=/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_MANIFEST_DIR=/home/
epage/src/personal/dump/cargo-14033 CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICE
NSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=cargo-14033 CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST
_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARG
O_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/epage/src/personal/dump/cargo-14033/target/debug/deps:/home/epage/src/pers
onal/dump/cargo-14033/target/debug:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-u
nknown-linux-gnu/lib:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib' /home/epage/src/personal/dump
/cargo-14033/target/debug/deps/cargo_14033-1ab2997d3cc74d9b`

running 1 test
test debug_assert_false ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

The difference is -C debug-assertions

The code for us setting that is:

cargo/src/cargo/core/compiler/mod.rs

Lines 1089 to 1108 in e52e0de

	// `-C overflow-checks` is implied by the setting of `-C debug-assertions`,
	// so we only need to provide `-C overflow-checks` if it differs from
	// the value of `-C debug-assertions` we would provide.
	if opt_level.as_str() != "0" {
	if debug_assertions {
	cmd.args(&["-C", "debug-assertions=on"]);
	if !overflow_checks {
	cmd.args(&["-C", "overflow-checks=off"]);
	}
	} else if overflow_checks {
	cmd.args(&["-C", "overflow-checks=on"]);
	}
	} else if !debug_assertions {
	cmd.args(&["-C", "debug-assertions=off"]);
	if overflow_checks {
	cmd.args(&["-C", "overflow-checks=on"]);
	}
	} else if !overflow_checks {
	cmd.args(&["-C", "overflow-checks=off"]);
	}

We seem to be going out of our way to rely on the default value

If not specified, debug assertions are automatically enabled only if the opt-level is 0.

Not too sure why we are doing this (except to reduce noise?). However, RUSTFLAGS is considered a lower level mechanism that bypasses Cargo and can break Cargo's behavior, so I'm a bit unsure if this is something we'd deem need "fixing".

[benny-n]

Hi, thanks for the quick reply.

We seem to be going out of our way to rely on the default value

If not specified, debug assertions are automatically enabled only if the opt-level is 0.

Given that changing rustc's default behavior is out of the question, I was about to suggest that cargo would behave the same and not enable debug assertions automatically unless the opt-level is 0, but after giving it a second thought, it would probably contradict the whole concept of cargo profiles (changing a specific option in a profile should not implicitly change the default value of another option).

I think that what bugged me the most here was not the behavior itself (the assertions being implicitly disabled) but rather the inconsistency between the two ways to set the opt-level, which from a naive point of view seems like they should behave exactly the same.

But then again, if you say that RUSTFLAGS is considered a mechanism that can break stuff, I guess that should be expected.

[weihanglo]

But then again, if you say that RUSTFLAGS is considered a mechanism that can break stuff, that should be expected.

While we don't intend to use this as an excuse to break stuff, the expectation is documented:

Caution: Due to the low-level nature of passing flags directly to the compiler, this may cause a conflict with future versions of Cargo which may issue the same or similar flags on its own which may interfere with the flags you specify. This is an area where Cargo may not always be backwards compatible.

There are other rustflags like --target cause even weirder behavior when being passed via build.rustflags. I am not sure if we should document every odd bit in The Cargo Book. That might be too much noise 😞. Maybe we could add a similar notice on the top of the Profiles chapter?