Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for named profiles (RFC 2678) #6989

Open
wants to merge 42 commits into
base: master
from

Conversation

@da-x
Copy link
Member

commented May 27, 2019

Tracking issue: #6988

Implementation according to the RFC.

Steps:

  • A working implementation
    • Add feature gate
    • Profile name validation
  • Testing
  • Documentation
  • Review
Support for named Cargo profiles
This allows creating custom profiles that inherit from other profiles.

For example, one can have a release-lto profile that looks like this:

    [profile.release-lto]
    inherits = "release"
    lto = true

The profile name will also carry itself into the output directory name
so that the different build outputs can be cached independently from
one another.

So in effect, at the `target` directory, a name will be created for
the new profile, in addition to the 'debug' and 'release' builds:

```
    $ cargo build --profile release-lto
    $ ls -l target
    debug release release-lto
```
@rust-highfive

This comment has been minimized.

Copy link

commented May 27, 2019

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @ehuss (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@ehuss

This comment has been minimized.

Copy link
Contributor

commented May 31, 2019

Nice! I'm looking forward to landing this.

Did you have any questions or have anything you want me to look at now? I've only skimmed this so far, and the only major thing that jumped out is that all new behavior will need to be behind a feature gate (documented in features.rs).

@da-x

This comment has been minimized.

Copy link
Member Author

commented Jun 1, 2019

Should the feature gate only affect the use of the [profile.<custom-name>] clauses, or should it also make other behavior changes conditional? (There are other changes, such as the added consistency for smoothing out of the behavior of specifying --profile= combined with --debug and --release across command).

da-x added 4 commits Jun 1, 2019
custom-named-profiles: Bug fix - no need for 'inherits' in root profiles
When resolving profile parameters for custom named profiles, we can skip
'dev' and 'release' because they are the root profiles from which all
others inherit. Otherwise, for example, we fail the tests where a simple
`[profile.dev]` clause is specified.
@da-x

This comment has been minimized.

Copy link
Member Author

commented Jun 1, 2019

Pushed changes: fixed a bug ; fixed the build of Cargo's tests ; fixed two tests, there are 25 tests still failing which need fixing.

da-x added 4 commits Jun 1, 2019
custom-named-profiles: fix merges of predefined profiles
The predefined 'test' and 'bench' profiles should exist even if
the user did not define anything in the Cargo.toml. This also
rewrites how the merging is done.
custom-named-profiles: fix profile for custom build
The profile returned for custom build should have the proper 'root'
field value.

This fixes test build_script::profile_and_opt_level_set_correctly.
custom-named-profiles: predefined check profile
Since we also for 'cargo rustc' to received '--profile check', let
it be consistent with the other commands and use a predefined 'check'
profile for it that inherits from "dev".

Fixes tests check::rustc_check and check::rustc_check_err.
@da-x

This comment has been minimized.

Copy link
Member Author

commented Jun 1, 2019

More fixes made, 12 failures remaining.

@da-x

This comment has been minimized.

Copy link
Member Author

commented Jun 1, 2019

@ehuss The profile_targets::profile_selection* test failures are mainly the ones that may benefit from your observation. The implementation changed which profile is being picked for which target. This is some of the confusion we wanted to clear up regarding how Cargo picks from the various profiles under various commands.

The results from the current PR changes are that at least for profile_selection_bench(), only codegen-units=4 is being seen in verbose, which means only [profile.bench] gets picked up, which I assume is what we wanted. The effects mainly stem from the removal of the reliance over CompileMode:: in src/cargo/core/profiles.rs.

@ehuss

This comment has been minimized.

Copy link
Contributor

commented Jun 3, 2019

I think the feature gate should cover the new [profile] tables and the --profile flag. Generally anything that changes the UI. The --profile flag can just be gated on -Zunstable-options (just check config.cli_unstable().unstable_options before reading it).

As for the change in consistency for test/bench, I'm on the fence. I think it would be OK to change it now as long as it inherits the settings from dev/release. Assuming that, then I think you should go ahead and update the profile_selection tests.

da-x added 6 commits Jun 7, 2019
custom-named-profiles: fix handling of 'clean' command
Before the RFC, the 'clean' command would have cleaned entirety of
target/ unless `--release` is specified - in that case it would have
cleaned only `target/release`.

Initial commit broke that.

Now, passing `--profile=NAME` will clean the directory related to
profile `NAME` via the dir-name setting. `--release` is supported for
backward compatibility.
tests: adjust profiles::profile_panic_test_bench for sorted output
New code relies on a BTreeMap, rather than handed coded calls.
@ehuss

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

Hey @da-x I just wanted to check in and see if you're waiting for me to review. Are there any known things unfinished?

da-x added 2 commits Jul 5, 2019
Revert "tests: profile_doc_deprecated should have 'inherits'"
This reverts commit 07d084532346d45bb1404a53de6af294fff480fa.
custom-profiles: for backward compatibility, allow 'doc' with a defau…
…lt 'inherits'

Some packages still have old `Cargo.toml` with a `profile.doc` section
and obviously without an 'inherits' keyword. We should not trip Cargo on
processing their Cargo.toml files.
@da-x

This comment has been minimized.

Copy link
Member Author

commented Jul 5, 2019

Yes, we still have some stuff unfinished.

Here's the current status:

  1. I've started to use this feature for an internal project of mine, to see whether there are issues.
    One issue was discovered when I tried using a custom profile with profile overrides. The errors I got was: "profile overrides may only be specified for dev or release profile, not release-lto". The restriction itself is easy to fix, but besides it, the profile overrides need to be inherited between the different profiles, per the RFC. I need to return to the code to see how to manage this.
  2. Missing new tests - I intend to add some tests to make sure 'inherits' does what it's suppose to be doing under various scenarios, plus usage of --profile and dir-name.
  3. Profile name validation, per the RFC.
  4. Documentation changes.
@bors

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

☔️ The latest upstream changes (presumably #7137) made this pull request unmergeable. Please resolve the merge conflicts.

@ehuss

This comment has been minimized.

Copy link
Contributor

commented Jul 31, 2019

Just wanted to check in again and see if you had any questions or need any help.

@da-x

This comment has been minimized.

Copy link
Member Author

commented Aug 1, 2019

@ehuss, I have made some unpushed changes for making sure that 1) profile overrides are inherited between profiles using inherits. 2) have a higher priority than profile options provided by custom profiles. 3) I need your okay for lifting the restriction over specifying profiles overrides for profiles other than dev and release.

To illustrate, the codegen-units values in this Cargo.toml:

[profile.dev]
codegen-units = 7

[profile.dev.overrides.xxx]
codegen-units = 5
[profile.dev.overrides.yyy]
codegen-units = 3

[profile.other]
inherits = "dev"
codegen-units = 2

[profile.other.overrides.yyy]
codegen-units = 6

Will yield:

with cargo build: root=7, xxx=5, yyy=3.
with cargo build --profile=other: root=2, xxx=5, yyy=6.

@bors

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2019

☔️ The latest upstream changes (presumably #7214) made this pull request unmergeable. Please resolve the merge conflicts.

da-x added 2 commits Aug 9, 2019
profiles: evaluate overrides separately from profiles
Before this change, settings from custom profiles had precedence
over overrides from inherited profiles:

    [profile.dev.overrides.pkg]
    codegen-units = 5

    [profile.dev-new]
    inherits = "dev"
    codegen-units = 2

Here with --profile=dev-new, `pkg` would have gotten codegen-units=2.

The fix, this, when going over the list of TomlProfiles for the purpose
of merging them, we should first merge the profiles themselves and only
later apply settings from the profile overrides (and in the same order).
@bors

This comment has been minimized.

Copy link
Contributor

commented Sep 3, 2019

☔️ The latest upstream changes (presumably #7216) made this pull request unmergeable. Please resolve the merge conflicts.

@ehuss

This comment has been minimized.

Copy link
Contributor

commented Sep 3, 2019

I need your okay for lifting the restriction over specifying profiles overrides for profiles other than dev and release.

Sorry, I missed your question. I think that makes sense and seems fine to me. The example you gave seems to be what I would expect.

da-x added 11 commits Sep 8, 2019
Reinstate warning regarding 'debug' profile
The previous warning was detected at the decoding level, with the test
removed in an earlier commit. Here it is brought back, in the custom
profile processing level.

Keeping this warning will serve to prevent confusion, when people expect
to affect the 'debug' directory via the 'debug' profile to no effect,
where in fact the 'dev' profile is the profile that they opted to
change.
testsuite: introduce profile_custom
This suite of tests verifies various cases around the 'inherits'
keyword, and also verifies the relationship between profile overrides
and custom profiles.
@da-x

This comment has been minimized.

Copy link
Member Author

commented Sep 12, 2019

Added tests.

Remaining changes involve the documentation.

@da-x da-x changed the title Support for named profiles (RFC 2689) Support for named profiles (RFC 2678) Sep 13, 2019

@da-x

This comment has been minimized.

Copy link
Member Author

commented Sep 13, 2019

I added documentation in unstable.md. @ehuss, all finished and waiting for review pending CI green check-mark completion.

@bors

This comment has been minimized.

Copy link
Contributor

commented Sep 17, 2019

☔️ The latest upstream changes (presumably #7311) made this pull request unmergeable. Please resolve the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.