New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cargo profile dependencies #2282

Merged
merged 10 commits into from Mar 2, 2018

Conversation

Projects
None yet
@Manishearth
Member

Manishearth commented Jan 8, 2018

Tracking issue: rust-lang/rust#48683

Rendered

@Manishearth

This comment has been minimized.

Show comment
Hide comment

@Centril Centril added the T-cargo label Jan 8, 2018

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 8, 2018

Member

Custom profiles can be listed in a .cargo/config, however the user is responsible for clearing up build directories if the profile changes. That is, it is undefined behavior to run cargo build --profile foo if foo has been defined in .cargo/config and the profile has been edited since the last time you ran cargo build --profile foo.

Hm, I would think Cargo should just rebuild everything in this case? And I even think this will happen naturally with the current implementation: IIRC, compiler flags are hashed into the fingerprint for determining if the crate should be rebuild.

What is the purpose of profiles .cargo/config? Is it something for the "big build system" use case? One interesting idea I have is that we can allow to specify profile as json on the command line. That would allow the build system to control Cargo without writing any files.

Member

matklad commented Jan 8, 2018

Custom profiles can be listed in a .cargo/config, however the user is responsible for clearing up build directories if the profile changes. That is, it is undefined behavior to run cargo build --profile foo if foo has been defined in .cargo/config and the profile has been edited since the last time you ran cargo build --profile foo.

Hm, I would think Cargo should just rebuild everything in this case? And I even think this will happen naturally with the current implementation: IIRC, compiler flags are hashed into the fingerprint for determining if the crate should be rebuild.

What is the purpose of profiles .cargo/config? Is it something for the "big build system" use case? One interesting idea I have is that we can allow to specify profile as json on the command line. That would allow the build system to control Cargo without writing any files.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jan 8, 2018

Member

Hm, I would think Cargo should just rebuild everything in this case?

I think tracking changes to config files might be out of scope for cargo

And I even think this will happen naturally with the current implementation: IIRC, compiler flags are hashed into the fingerprint for determining if the crate should be rebuild.

If this is true then that works great!

What is the purpose of profiles .cargo/config? Is it something for the "big build system" use case?

Well, two purposes. Lets people specify global useful profiles like cargo build --debugopt or something.

Also, it fixes the big build system case -- Having a custom profile for each combination of flags quickly leads to blowup. Firefox lets you individually flip debug symbols, debug-assertions, and optimization levels. There are more involved, though. (There are use cases; folks hacking on Rust code may want different settings from folks hacking on C++ code)

Profiles aren't part of the package anyway, they're build info, and that really should go in config anyway.

Member

Manishearth commented Jan 8, 2018

Hm, I would think Cargo should just rebuild everything in this case?

I think tracking changes to config files might be out of scope for cargo

And I even think this will happen naturally with the current implementation: IIRC, compiler flags are hashed into the fingerprint for determining if the crate should be rebuild.

If this is true then that works great!

What is the purpose of profiles .cargo/config? Is it something for the "big build system" use case?

Well, two purposes. Lets people specify global useful profiles like cargo build --debugopt or something.

Also, it fixes the big build system case -- Having a custom profile for each combination of flags quickly leads to blowup. Firefox lets you individually flip debug symbols, debug-assertions, and optimization levels. There are more involved, though. (There are use cases; folks hacking on Rust code may want different settings from folks hacking on C++ code)

Profiles aren't part of the package anyway, they're build info, and that really should go in config anyway.

@est31

This comment has been minimized.

Show comment
Hide comment
@est31

est31 Jan 9, 2018

Contributor

Is this compatible with the profiles 2.0 proposal by @matklad ? IIRC there was an embargo for new features for profiles until they get revamped? Generally I like the idea of this RFC though.

Contributor

est31 commented Jan 9, 2018

Is this compatible with the profiles 2.0 proposal by @matklad ? IIRC there was an embargo for new features for profiles until they get revamped? Generally I like the idea of this RFC though.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jan 10, 2018

Member

@est31 I think this addresses the "some kind of DSL to map crates to flags" from profiles 2.0

This RFC does not attempt to handle the issue with test/bench/doc but I think that can be addressed independently. One solution for that would be closer to the "includeable profiles" proposal in the pre-RFC, and have something like [profile.dev] testing = otherprofilename

Member

Manishearth commented Jan 10, 2018

@est31 I think this addresses the "some kind of DSL to map crates to flags" from profiles 2.0

This RFC does not attempt to handle the issue with test/bench/doc but I think that can be addressed independently. One solution for that would be closer to the "includeable profiles" proposal in the pre-RFC, and have something like [profile.dev] testing = otherprofilename

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 10, 2018

Member

@est31 I think of this RFC as basically profiles 2.0, and I really love it :) It doesn't solve the "which build in profile is run when" problem, but I think it can be addressed separately. Although I would prefer that instead of [profile.dev] testing = otherprofilename we'd just killed all build-in profiles other then dev or release.

Member

matklad commented Jan 10, 2018

@est31 I think of this RFC as basically profiles 2.0, and I really love it :) It doesn't solve the "which build in profile is run when" problem, but I think it can be addressed separately. Although I would prefer that instead of [profile.dev] testing = otherprofilename we'd just killed all build-in profiles other then dev or release.

@ben0x539

This comment has been minimized.

Show comment
Hide comment
@ben0x539

ben0x539 Jan 14, 2018

Hash the profile data into the output path and you don't need to worry about undefined behavior on profile changes, I guess.

ben0x539 commented Jan 14, 2018

Hash the profile data into the output path and you don't need to worry about undefined behavior on profile changes, I guess.

@jonhoo

This comment has been minimized.

Show comment
Hide comment
@jonhoo

jonhoo Jan 18, 2018

Contributor

One question I have with this is how this applies transitively. For example, consider crate c which depends on b which depends on a. Say b knows that a should always be compiled with optimizations. If c is compiled in debug mode, is it necessary for c to also declare that b.a should be compiled with optimizations? Or can c somehow be informed by b that a should be compiled with optimizations?

Contributor

jonhoo commented Jan 18, 2018

One question I have with this is how this applies transitively. For example, consider crate c which depends on b which depends on a. Say b knows that a should always be compiled with optimizations. If c is compiled in debug mode, is it necessary for c to also declare that b.a should be compiled with optimizations? Or can c somehow be informed by b that a should be compiled with optimizations?

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 19, 2018

Member

@jonhoo currently, only the leaf crate's profiles are accounted for (it can even be argued on this basis that profiles should not live in Cargo.toml and should go into .cargo/config instead), and, for workspaces, you can only specify profiles in the root of the workspace. I believe that this RFC does not propose to change this mechanics.

I can imagine an orthogonal RFC though, which allows upstream crate to advice some profile settings to the downstream crates. For example, the image crate could advise "compile me with -O 3 in the dev profile", and this setting would be used for crates which depend on image, unless they explicitly override it.

The possible interaction between this RFC and hypothetical advise RFC is that it's not clear how upstream crates can advice on custom profiles (they don't even know their names!). I think it's reasonable to assume though that this is not a super important problem which we need to solve now: either of "advises do not work with custom profiles" or "custom profile can opt into advises from a particular build-in profile" seems like an ok solution.

Member

matklad commented Jan 19, 2018

@jonhoo currently, only the leaf crate's profiles are accounted for (it can even be argued on this basis that profiles should not live in Cargo.toml and should go into .cargo/config instead), and, for workspaces, you can only specify profiles in the root of the workspace. I believe that this RFC does not propose to change this mechanics.

I can imagine an orthogonal RFC though, which allows upstream crate to advice some profile settings to the downstream crates. For example, the image crate could advise "compile me with -O 3 in the dev profile", and this setting would be used for crates which depend on image, unless they explicitly override it.

The possible interaction between this RFC and hypothetical advise RFC is that it's not clear how upstream crates can advice on custom profiles (they don't even know their names!). I think it's reasonable to assume though that this is not a super important problem which we need to solve now: either of "advises do not work with custom profiles" or "custom profile can opt into advises from a particular build-in profile" seems like an ok solution.

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 19, 2018

Member

Also, cc @wycats for the whole discussion :)

Member

matklad commented Jan 19, 2018

Also, cc @wycats for the whole discussion :)

@jonhoo

This comment has been minimized.

Show comment
Hide comment
@jonhoo

jonhoo Jan 19, 2018

Contributor

@matklad you're right that this could totally be addressed in a separate RFC. In fact, I wonder whether, if we had that RFC, that would alleviate many of the pains that this RFC tries to counter. The reason I bring this up as a concern here is that I worry that we'll see these extra profiles added all over the place as a result of this particular solution to the problem. Basically every crate that depends on image would add this extra profile (probably with different names too!), and crates that depend on those crates would also add them, and so on. Going forward with this RFC (without the other one first) will likely lead to lots of people sticking extra duplicated overrides in their Cargo.toml, which bothers me. An "upstream-down" approach strikes me as much nicer for the ecosystem as a whole.

On a somewhat separate tangent: how often do we expect users to actually add separate profiles, rather than just adding overrides for the debug profile? I personally think (though I may be wrong) that the primary use-case for this is compiling specific dependencies with opts in debug mode, in which case it's unclear we even need custom profiles, and not just "being able to specify overrides for existing profiles". Do we have a compelling use-case for where an entirely new profile is warranted?

Contributor

jonhoo commented Jan 19, 2018

@matklad you're right that this could totally be addressed in a separate RFC. In fact, I wonder whether, if we had that RFC, that would alleviate many of the pains that this RFC tries to counter. The reason I bring this up as a concern here is that I worry that we'll see these extra profiles added all over the place as a result of this particular solution to the problem. Basically every crate that depends on image would add this extra profile (probably with different names too!), and crates that depend on those crates would also add them, and so on. Going forward with this RFC (without the other one first) will likely lead to lots of people sticking extra duplicated overrides in their Cargo.toml, which bothers me. An "upstream-down" approach strikes me as much nicer for the ecosystem as a whole.

On a somewhat separate tangent: how often do we expect users to actually add separate profiles, rather than just adding overrides for the debug profile? I personally think (though I may be wrong) that the primary use-case for this is compiling specific dependencies with opts in debug mode, in which case it's unclear we even need custom profiles, and not just "being able to specify overrides for existing profiles". Do we have a compelling use-case for where an entirely new profile is warranted?

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Jan 19, 2018

Member

@jonhoo FWIW this RFC does propose allowing profiles in .cargo/config, but it also allows them in Cargo.toml for consistency

Member

Manishearth commented Jan 19, 2018

@jonhoo FWIW this RFC does propose allowing profiles in .cargo/config, but it also allows them in Cargo.toml for consistency

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 19, 2018

Member

@jonhoo note that the RFC supports overrides for build-in profiles, so I don't believe that "every crate that depends on image would add this extra profile". Rather, every crate which depends on image would add [profiles.dev.overrides.image] opt-level = 3. This indeed will be duplicated, but the amount of duplication is small, and gives you extra flexibility as each crate can decide precisely on the settings for image.

In general, I think the need to customize profiles should be rare, so I'd go for the increased flexibility rather than for increased convenience (but the syntax proposed in this RFC looks pretty and convenient).

I personally think (though I may be wrong) that the primary use-case for this is compiling specific dependencies with opts in debug mode, in which case it's unclear we even need custom profiles, and not just "being able to specify overrides for existing profiles".

Hm, interesting! I believe that it is indeed possible to split this RFC further into "overrides" and "custom profiles" parts, and I agree that "overrides" part has potentially many more users, because fully custom profiles a required mainly by large users like Servo. My personal preference would be to nevertheless pursue the custom profiles part. I really like how it creates a very simple mental model for profiles: "if you pass --profile foo, then everything uses the foo profile". This model is pretty different for the one for build-in profiles, where different parts of the packages use different profiles. And having such simple model implemented in Cargo would be very useful, because it would make "fixing" build-in profiles to use this model simpler.

Member

matklad commented Jan 19, 2018

@jonhoo note that the RFC supports overrides for build-in profiles, so I don't believe that "every crate that depends on image would add this extra profile". Rather, every crate which depends on image would add [profiles.dev.overrides.image] opt-level = 3. This indeed will be duplicated, but the amount of duplication is small, and gives you extra flexibility as each crate can decide precisely on the settings for image.

In general, I think the need to customize profiles should be rare, so I'd go for the increased flexibility rather than for increased convenience (but the syntax proposed in this RFC looks pretty and convenient).

I personally think (though I may be wrong) that the primary use-case for this is compiling specific dependencies with opts in debug mode, in which case it's unclear we even need custom profiles, and not just "being able to specify overrides for existing profiles".

Hm, interesting! I believe that it is indeed possible to split this RFC further into "overrides" and "custom profiles" parts, and I agree that "overrides" part has potentially many more users, because fully custom profiles a required mainly by large users like Servo. My personal preference would be to nevertheless pursue the custom profiles part. I really like how it creates a very simple mental model for profiles: "if you pass --profile foo, then everything uses the foo profile". This model is pretty different for the one for build-in profiles, where different parts of the packages use different profiles. And having such simple model implemented in Cargo would be very useful, because it would make "fixing" build-in profiles to use this model simpler.

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 29, 2018

Member

@rfcbot fcp merge

Member

matklad commented Jan 29, 2018

@rfcbot fcp merge

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Jan 29, 2018

Team member @matklad has proposed to merge this. The next step is review by the rest of the tagged teams:

Concerns:

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

rfcbot commented Jan 29, 2018

Team member @matklad has proposed to merge this. The next step is review by the rest of the tagged teams:

Concerns:

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 29, 2018

Member

@rust-lang/cargo discussed this RFC at today's meeting, and it looks really great!

There are couple of implementation-level concerns/questions though.

Member

matklad commented Jan 29, 2018

@rust-lang/cargo discussed this RFC at today's meeting, and it looks really great!

There are couple of implementation-level concerns/questions though.

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 29, 2018

Member

@rfcbot concern workspaces

Cargo supports building several packages together as a part of the workspace. In workspaces, only the profile section from the root of the workspace (virtual or not) is taking an effect. Currently this works as expected, because profiles apply to all crates in the DAG of dependencies equally.

However with this RFC there's an observable difference between the "current" crate and all other crates. How this interacts with multi-package workspaces.

For example, if workspace consists of foo and bar, and foo depends on bar, and profiles section contains

[profile.dev]
opt-level = 0

[profile.dev.overrides."*"]
opt-level = 2

what rule applies to bar?

Member

matklad commented Jan 29, 2018

@rfcbot concern workspaces

Cargo supports building several packages together as a part of the workspace. In workspaces, only the profile section from the root of the workspace (virtual or not) is taking an effect. Currently this works as expected, because profiles apply to all crates in the DAG of dependencies equally.

However with this RFC there's an observable difference between the "current" crate and all other crates. How this interacts with multi-package workspaces.

For example, if workspace consists of foo and bar, and foo depends on bar, and profiles section contains

[profile.dev]
opt-level = 0

[profile.dev.overrides."*"]
opt-level = 2

what rule applies to bar?

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 29, 2018

Member

@rfcbot concern build_override

build_override definitely feels like a special case. Would it be feasible/worthwhile to make a general feature which allows to tweak each target? The staraman syntax would be [profile.dev.overrides.foo.build], [profile.dev.overrides.foo.bin."*"], [profile.dev.overrides.foo.test.test_bar]?

I personally also think that maybe we should just always compile build.rs files with optimizations? You compile build.rs only once, so it seems like a win, and is somewhat similar to cargo install in spirit. There's also a question of what to do with crates like LALRPOP: if you use it, you definitely want to compile build.rs in release mode, and, with current proposal, every reverse dependecy of LALRPOP would need to specify this explicitly.

It would be great if someone could measure the impact of always-release build.rs. How frequent are the cases when it significantly worsens compile times? How frequent are the cases where compile times are improved?

Member

matklad commented Jan 29, 2018

@rfcbot concern build_override

build_override definitely feels like a special case. Would it be feasible/worthwhile to make a general feature which allows to tweak each target? The staraman syntax would be [profile.dev.overrides.foo.build], [profile.dev.overrides.foo.bin."*"], [profile.dev.overrides.foo.test.test_bar]?

I personally also think that maybe we should just always compile build.rs files with optimizations? You compile build.rs only once, so it seems like a win, and is somewhat similar to cargo install in spirit. There's also a question of what to do with crates like LALRPOP: if you use it, you definitely want to compile build.rs in release mode, and, with current proposal, every reverse dependecy of LALRPOP would need to specify this explicitly.

It would be great if someone could measure the impact of always-release build.rs. How frequent are the cases when it significantly worsens compile times? How frequent are the cases where compile times are improved?

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Jan 29, 2018

Member

Implementation-wise, it would probably make sense to file all of

  • custom profile
  • profile overrides
  • profiles in .cargo/config

under separate feature-flags.

We can also stabilize this piece-wise. For example, overrides."image=0.2 piston>5.0" looks more controversial than overrides.image, so probably could stabilize the second form first, and than make a follow-up.

Member

matklad commented Jan 29, 2018

Implementation-wise, it would probably make sense to file all of

  • custom profile
  • profile overrides
  • profiles in .cargo/config

under separate feature-flags.

We can also stabilize this piece-wise. For example, overrides."image=0.2 piston>5.0" looks more controversial than overrides.image, so probably could stabilize the second form first, and than make a follow-up.

@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Jan 29, 2018

Member

@rfcbot concern Document how to set panic

This isn't a showstopper, just something I'd like to see documented in the RFC.

You can't currently pass custom options to rustc, so I don't think this applies to anything else. But right now, you might write a crate that only makes sense with panic=unwind, or only makes sense with panic=abort. You might set those in your build profiles. If someone overrides those profiles, they'll have problems.

I've seen multiple proposals for other ways to specify panic behavior, such as module attributes within Rust code. I don't know the current status of those proposals.

I'd like to ask that the RFC suggest how to handle this situation. Or, if the current state is that there's no other stable way to specify this, I'd like to suggest that an RFC get kicked off to provide a mechanism for this. (That RFC would not need to complete before this one gets accepted, but I'd like to see something proposed.)

The analogous situation for C: running make CFLAGS='-O0 -g' should override the Makefile specifying -O3, but should not override the Makefile specifying -Wall or -fno-strict-aliasing or -std=c11. People have rough conventions and understandings for which flags a command-line-specified CFLAGS should override and which it shouldn't (though this still remains the realm of undocumented tribal knowledge). Similarly, overriding a build profile should not cause a crate to fail to build.

Member

joshtriplett commented Jan 29, 2018

@rfcbot concern Document how to set panic

This isn't a showstopper, just something I'd like to see documented in the RFC.

You can't currently pass custom options to rustc, so I don't think this applies to anything else. But right now, you might write a crate that only makes sense with panic=unwind, or only makes sense with panic=abort. You might set those in your build profiles. If someone overrides those profiles, they'll have problems.

I've seen multiple proposals for other ways to specify panic behavior, such as module attributes within Rust code. I don't know the current status of those proposals.

I'd like to ask that the RFC suggest how to handle this situation. Or, if the current state is that there's no other stable way to specify this, I'd like to suggest that an RFC get kicked off to provide a mechanism for this. (That RFC would not need to complete before this one gets accepted, but I'd like to see something proposed.)

The analogous situation for C: running make CFLAGS='-O0 -g' should override the Makefile specifying -O3, but should not override the Makefile specifying -Wall or -fno-strict-aliasing or -std=c11. People have rough conventions and understandings for which flags a command-line-specified CFLAGS should override and which it shouldn't (though this still remains the realm of undocumented tribal knowledge). Similarly, overriding a build profile should not cause a crate to fail to build.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Feb 16, 2018

Member

Updated. I removed the notion of custom profiles, however I kept around the ability to define profiles in .cargo/config. That can be made into a separate RFC if desired, it seems to be a small feature.

Member

Manishearth commented Feb 16, 2018

Updated. I removed the notion of custom profiles, however I kept around the ability to define profiles in .cargo/config. That can be made into a separate RFC if desired, it seems to be a small feature.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Feb 16, 2018

Member

@Manishearth Ah sorry, yes the .cargo/config seems fine.

Member

aturon commented Feb 16, 2018

@Manishearth Ah sorry, yes the .cargo/config seems fine.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Feb 16, 2018

Member

Thanks @Manishearth, looks great! I've checked off my review box; just waiting on @joshtriplett and @withoutboats now.

Member

aturon commented Feb 16, 2018

Thanks @Manishearth, looks great! I've checked off my review box; just waiting on @joshtriplett and @withoutboats now.

@Manishearth Manishearth changed the title from Custom cargo profiles to Cargo profile dependencies Feb 16, 2018

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Feb 16, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

rfcbot commented Feb 16, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

3 similar comments
@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Feb 16, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

rfcbot commented Feb 16, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Feb 16, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

rfcbot commented Feb 16, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Feb 16, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

rfcbot commented Feb 16, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Feb 16, 2018

Member

uh oh it's the robot rebellion

Member

Manishearth commented Feb 16, 2018

uh oh it's the robot rebellion

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Feb 26, 2018

The final comment period is now complete.

rfcbot commented Feb 26, 2018

The final comment period is now complete.

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Feb 26, 2018

Member

We've discussed the broader issue of workflows at today's meeting, here's the summary.

Today's profiles represent two different things. The first one is the workflow, or operation, which Cargo currently is doing, like testing, documenting or benchmarking. The other, more narrow, is the set of "optimization flags" applied when building the code.

It makes sense to split the current profile concept into two separate ones, by introducing workflow as a first-class thing for describing a particular set of operations cargo does, and scaling down profile concept to just mean the set of optimization flags.

One can imagine that there will be roughly one workflow for each major Cargo command (testing workflow, documenting workflow, benchmarking workflow). Ideally, it should be possible to add custom workflows like a fuzzing workflow.

In contrast, there should be relatively fewer profiles: dev and release seem to cover most use-case, but obvious possible extensions include release-with-debuginfo and a "slightly optimized" dev.

Each workflow should be linked to some default profile, but the profile could be overridden via command line flag. For example, testing workflow would use dev profile by default, and benchmarking workflow would use the release profile, however, test --profile=release would also work.

Now, the interesting question is, which flags belong to profiles, and which flags belong to workflow? The interesting dimension for comparison here is "can I apply this flag to only part of the packages?". It is interesting because workflows are global for the whole project, while profiles, with this RFC, can apply only to the specific packages. However, certain optimization flags, like lto apply to the artifact as a whole: i.e, their granularity is larger than a crate, but smaller than project. So, this gives us three bins to sort flags into:

  • workflow flags (per project)
  • profile global (per artifact)
  • profile local (per package == per dependency == per library crate)

Sorting the current flags gives this picture:

  • workflow:

    • rpath
    • panic
  • profile global:

    • lto
    • codegen-units
  • profile local:

    • opt-level
    • debug
    • debug-assertions
Member

matklad commented Feb 26, 2018

We've discussed the broader issue of workflows at today's meeting, here's the summary.

Today's profiles represent two different things. The first one is the workflow, or operation, which Cargo currently is doing, like testing, documenting or benchmarking. The other, more narrow, is the set of "optimization flags" applied when building the code.

It makes sense to split the current profile concept into two separate ones, by introducing workflow as a first-class thing for describing a particular set of operations cargo does, and scaling down profile concept to just mean the set of optimization flags.

One can imagine that there will be roughly one workflow for each major Cargo command (testing workflow, documenting workflow, benchmarking workflow). Ideally, it should be possible to add custom workflows like a fuzzing workflow.

In contrast, there should be relatively fewer profiles: dev and release seem to cover most use-case, but obvious possible extensions include release-with-debuginfo and a "slightly optimized" dev.

Each workflow should be linked to some default profile, but the profile could be overridden via command line flag. For example, testing workflow would use dev profile by default, and benchmarking workflow would use the release profile, however, test --profile=release would also work.

Now, the interesting question is, which flags belong to profiles, and which flags belong to workflow? The interesting dimension for comparison here is "can I apply this flag to only part of the packages?". It is interesting because workflows are global for the whole project, while profiles, with this RFC, can apply only to the specific packages. However, certain optimization flags, like lto apply to the artifact as a whole: i.e, their granularity is larger than a crate, but smaller than project. So, this gives us three bins to sort flags into:

  • workflow flags (per project)
  • profile global (per artifact)
  • profile local (per package == per dependency == per library crate)

Sorting the current flags gives this picture:

  • workflow:

    • rpath
    • panic
  • profile global:

    • lto
    • codegen-units
  • profile local:

    • opt-level
    • debug
    • debug-assertions
@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Feb 27, 2018

Member

Why is codegen-units profile global?

I think LTO, rpath, panic are all global, of which rpath/LTO are workflow-related ones, and panic is weird.

We can amend this RFC so that LTO, rpath, panic can only be specified at the top level

(we already do for panics)

Member

Manishearth commented Feb 27, 2018

Why is codegen-units profile global?

I think LTO, rpath, panic are all global, of which rpath/LTO are workflow-related ones, and panic is weird.

We can amend this RFC so that LTO, rpath, panic can only be specified at the top level

(we already do for panics)

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Mar 2, 2018

Member

@matklad I'm confused as to what the status of this RFC is. Was your comment an indicator of the overall plans for profiles, or a request for amendments to this one (seems like they would be minor). The FCP has finished already.

Member

Manishearth commented Mar 2, 2018

@matklad I'm confused as to what the status of this RFC is. Was your comment an indicator of the overall plans for profiles, or a request for amendments to this one (seems like they would be minor). The FCP has finished already.

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Mar 2, 2018

Member

@Manishearth this was a comment about the general future direction of profiles, the RFC is great as it stands! I think it should be merged now, but I am not sure about the precise process to do this :)

Member

matklad commented Mar 2, 2018

@Manishearth this was a comment about the general future direction of profiles, the RFC is great as it stands! I think it should be merged now, but I am not sure about the precise process to do this :)

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Mar 2, 2018

Member

Cool. I think we're just supposed to rename the file and fill in details and then merge it. I'll let @nrc merge

Member

Manishearth commented Mar 2, 2018

Cool. I think we're just supposed to rename the file and fill in details and then merge it. I'll let @nrc merge

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Mar 2, 2018

Member

Updated

Member

Manishearth commented Mar 2, 2018

Updated

@Manishearth

This comment has been minimized.

Show comment
Hide comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment