Deprecate the "bench" and "test" profiles

[briansmith]

AFAICT, any time the "bench" or "test" profile is different from "release", it is a misconfiguration. Cargo should warn whenever a "bench" or "test" profile is defined, and it should always use the "dev" or "release" profile, based on whether or not "--release" was passed.

From #cargo IRC chat today:

I also noticed something strange about profiles: why would one want the "test" profile or "bench" profile to ever be different from "release"?
If they were different, then you wouldn't be testing or measuring what you release.
that's a good point, I've never seen a difference in practice at least
I had mine different for a while because I hadn't thought about it. I'd only changed "release"

[jonas-schievink]

Test should be different from release because I don't want to spend ten minutes waiting for optimizations every time I run tests.

[briansmith]

Test should be different from release because I don't want to spend ten minutes waiting for optimizations every time I run tests.

cargo test should use the dev profile and cargo test --release should use the release profile. cargo bench should always build a release build.

[jonas-schievink]

Ah, yes. That makes sense.

[weihanglo]

Cargo named profile (RFC 2678) has been landed since 1.57 (2021-12-02). That should (hopefully) make everyone happy as test/bench profiles now inherits settings from dev/release, but still open to customizations.

Going to close this now. If you found this is a fault, please comment, and we'll consider re-open.