Make Cargo aware of standard library dependencies #1133

Closed
wants to merge 66 commits into
from

Conversation

@Ericson2314
Contributor

Ericson2314 commented May 26, 2015

Currently, all packages implicitly depend on libstd. This makes Cargo unsuitable for packages that
need a custom-built libstd, or otherwise depend on crates with the same names as libstd and the
crates behind the facade. The proposed fixes also open the door to a future where libstd can be
Cargoized.

I've been busy with school until very recently, so apologies in advance if it turns out I am not taking into account some recent change.

Rendered

Ericson2314 added some commits May 25, 2015

First version
I clarified and cleaned up some things, but force-pushed as
 - The ideas are the same, just the wording is changed
 - At the time of commit, I had not yet submitted the PR
@posix4e

This comment has been minimized.

Show comment
Hide comment

posix4e commented May 26, 2015

Wow neat

text/0000-cargo-libstd-awareness.md
+
+Currently, all packages implicitly depend on libstd. This makes Cargo unsuitable for packages that
+need a custom-built libstd, or otherwise depend on crates with the same names as libstd and the
+crates behind the facade. The proposed fixes also open the door to a future were libstd can be

This comment has been minimized.

@tshepang

tshepang May 26, 2015

Contributor

s/were/where

@tshepang

tshepang May 26, 2015

Contributor

s/were/where

This comment has been minimized.

@Ericson2314

Ericson2314 May 26, 2015

Contributor

Thanks!

@Ericson2314

Ericson2314 May 26, 2015

Contributor

Thanks!

@llogiq

This comment has been minimized.

Show comment
Hide comment
@llogiq

llogiq May 26, 2015

Contributor

A funny 'coin'cidence is that the Java folks at Oracle are currently testing a similar concept (and they needed 9 major versions to get there). 👍

Contributor

llogiq commented May 26, 2015

A funny 'coin'cidence is that the Java folks at Oracle are currently testing a similar concept (and they needed 9 major versions to get there). 👍

text/0000-cargo-libstd-awareness.md
+The only new interface proposed is a boolean field to the package meta telling Cargo that the
+package does not depend on libstd by default. This need not imply Rust's `no_std`, as one might want
+to `use` their own build of libstd by default. To disambiguate, this field is called
+q`no-implicit-deps`; please, go ahead and bikeshead the name. `no-implicit-deps` is false by

This comment has been minimized.

@Valloric

Valloric May 26, 2015

The specific name doesn't concern me too much, but do make it boolean-positive instead of negative. So instead of having no-implicit-deps and setting it to false by default, make it use-implicit-deps and make it true by default.

Whatever name is bikeshedded should be boolean-positive.

@Valloric

Valloric May 26, 2015

The specific name doesn't concern me too much, but do make it boolean-positive instead of negative. So instead of having no-implicit-deps and setting it to false by default, make it use-implicit-deps and make it true by default.

Whatever name is bikeshedded should be boolean-positive.

text/0000-cargo-libstd-awareness.md
+
+## Phase 1
+
+Add a `--no-sysroot` flag to `rustc`, and pass that to `rustc` is the case that `no-implicit-deps`

This comment has been minimized.

@Valloric

Valloric May 26, 2015

Similar here; instead of --no-sysroot, make it --use-sysroot=false (with the default being true).

@Valloric

Valloric May 26, 2015

Similar here; instead of --no-sysroot, make it --use-sysroot=false (with the default being true).

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 May 26, 2015

Contributor

@Valloric in case you don't get a notification from the commit message, I did what you suggested with the exception that the field is just called implicit-deps. I did that as I didn't want to make anyone think of Rust's use, because then the fields sounds more like #![no_std].

Contributor

Ericson2314 commented May 26, 2015

@Valloric in case you don't get a notification from the commit message, I did what you suggested with the exception that the field is just called implicit-deps. I did that as I didn't want to make anyone think of Rust's use, because then the fields sounds more like #![no_std].

@Valloric

This comment has been minimized.

Show comment
Hide comment
@Valloric

Valloric May 26, 2015

@Ericson2314 That's fine; like I said, I don't personally care much about the name (you'll get more than enough bikeshedding from others), only that a double negative is avoided.

@Ericson2314 That's fine; like I said, I don't personally care much about the name (you'll get more than enough bikeshedding from others), only that a double negative is avoided.

@llogiq

This comment has been minimized.

Show comment
Hide comment
@llogiq

llogiq May 28, 2015

That should probably read "...is to be downloaded...".

That should probably read "...is to be downloaded...".

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 May 28, 2015

Owner

I was flip/floping on what tense to use, so I think should be just "...is downloaded", but thanks for the catch!

Owner

Ericson2314 replied May 28, 2015

I was flip/floping on what tense to use, so I think should be just "...is downloaded", but thanks for the catch!

@llogiq

This comment has been minimized.

Show comment
Hide comment
@llogiq

llogiq May 28, 2015

Contributor

Also the text states that "std should be on crates.io" – the rust distribution should still contain all of std anyway, right? Otherwise people who download rust to try it during their commute are going to have a hard time. 😄 (Case in point: I recently had to download a doc snapshot from servo so I could continue to hack on rust-clippy when offline, because the standard docs no longer contain the rustc… crates).

Contributor

llogiq commented May 28, 2015

Also the text states that "std should be on crates.io" – the rust distribution should still contain all of std anyway, right? Otherwise people who download rust to try it during their commute are going to have a hard time. 😄 (Case in point: I recently had to download a doc snapshot from servo so I could continue to hack on rust-clippy when offline, because the standard docs no longer contain the rustc… crates).

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton May 28, 2015

Member

Thanks for the RFC @Ericson2314! I'm always quite eager to improve the cross-compilation experience of Rust, and I think that this will help out embedded targets and such quite a bit!

At a high level, could you clarify the motivation for this RFC a bit more in the text? After reading it it's not clear to me what the RFC is trying to achieve. For example:

  • Is the point of this RFC to enable easy cross compilation of the standard library?
  • Is this enabling swapping in an alternate standard library?
  • Is this just disabling the ability to extern crate std if a crate opts-out?

I think that refining the motivation will help me understand more of the detailed design as well, but I'll make some specific comments below about this.

However, it also passes in --sysroot /path/to/some/libdir, the directory being were libstd is.

I'm curious where you came up with this? Cargo doesn't pass --sysroot at all.

This need not imply Rust's no_std, as one might want to use their own build of libstd by default.

This worries me a bit, I don't think we want the ability to swap in an alternate standard library in a super easy fashion. If you're depending on your own "standard library", then I think it should be just that, a dependency in Cargo.toml which goes through the normal crates.io + Cargo process. In general I think it's pretty brittle to swap out the std that an upstream crate is using, praying the APIs match up and the compile still succeeds.

This hotfix is enough to allow us bare-metal devs to use Cargo for our own projects, but doesn't suffice for creating an ecosystem of packages that depend on crates behind the facade but not libstd itself.

Can you clarify why you can't use Cargo for these kinds of projects today? If you use #![no_std] and don't use extern crate std, you can basically do anything with Cargo you can do with the compiler. For example I have written my own kernel which is built with Cargo and depends on rlibc from crates.io, and it all works out just fine (no standard library exists).

But users can use config files to extend the namespace so their own modded libstds can be used instead. Even if they don't want to change libstd but just cross-compile it, this is frankly the easiest way as Cargo will seemliest cross compile both their project and it's transitive dependencies.

I also unfortunately don't fully understand what's going on here. Can you elaborate why you want to swap out a separate libstd than the one the compiler may already find elsewhere? If crates can be tracked in Cargo, then I definitely think they should, so I don't follow the motivation to not use #![no_std] and instead try to name a crate std somewhere.

In this way we can put packages on crates.io that depend on the crates behind the facade.

I don't quite understand this comment because this can already be done? I can upload a crate which says #![no_std] and doesn't depend on the standard library (only libcore). Can you elaborate on how this is different and what you're thinking here?


Overall I think this RFC feels like it's "bolting on" features after the fact instead of creating a well-integrated and smooth system with Cargo. I believe this is also (depending on the motivation) highly inter-related with #1005, and it may wish to be considered here. I think some refinement of the motivation, however, will help guide this RFC, though!

Member

alexcrichton commented May 28, 2015

Thanks for the RFC @Ericson2314! I'm always quite eager to improve the cross-compilation experience of Rust, and I think that this will help out embedded targets and such quite a bit!

At a high level, could you clarify the motivation for this RFC a bit more in the text? After reading it it's not clear to me what the RFC is trying to achieve. For example:

  • Is the point of this RFC to enable easy cross compilation of the standard library?
  • Is this enabling swapping in an alternate standard library?
  • Is this just disabling the ability to extern crate std if a crate opts-out?

I think that refining the motivation will help me understand more of the detailed design as well, but I'll make some specific comments below about this.

However, it also passes in --sysroot /path/to/some/libdir, the directory being were libstd is.

I'm curious where you came up with this? Cargo doesn't pass --sysroot at all.

This need not imply Rust's no_std, as one might want to use their own build of libstd by default.

This worries me a bit, I don't think we want the ability to swap in an alternate standard library in a super easy fashion. If you're depending on your own "standard library", then I think it should be just that, a dependency in Cargo.toml which goes through the normal crates.io + Cargo process. In general I think it's pretty brittle to swap out the std that an upstream crate is using, praying the APIs match up and the compile still succeeds.

This hotfix is enough to allow us bare-metal devs to use Cargo for our own projects, but doesn't suffice for creating an ecosystem of packages that depend on crates behind the facade but not libstd itself.

Can you clarify why you can't use Cargo for these kinds of projects today? If you use #![no_std] and don't use extern crate std, you can basically do anything with Cargo you can do with the compiler. For example I have written my own kernel which is built with Cargo and depends on rlibc from crates.io, and it all works out just fine (no standard library exists).

But users can use config files to extend the namespace so their own modded libstds can be used instead. Even if they don't want to change libstd but just cross-compile it, this is frankly the easiest way as Cargo will seemliest cross compile both their project and it's transitive dependencies.

I also unfortunately don't fully understand what's going on here. Can you elaborate why you want to swap out a separate libstd than the one the compiler may already find elsewhere? If crates can be tracked in Cargo, then I definitely think they should, so I don't follow the motivation to not use #![no_std] and instead try to name a crate std somewhere.

In this way we can put packages on crates.io that depend on the crates behind the facade.

I don't quite understand this comment because this can already be done? I can upload a crate which says #![no_std] and doesn't depend on the standard library (only libcore). Can you elaborate on how this is different and what you're thinking here?


Overall I think this RFC feels like it's "bolting on" features after the fact instead of creating a well-integrated and smooth system with Cargo. I believe this is also (depending on the motivation) highly inter-related with #1005, and it may wish to be considered here. I think some refinement of the motivation, however, will help guide this RFC, though!

Ericson2314 added some commits May 28, 2015

Rewrite after talking to @alexcrichton on IRC.
tl;dr:
 - Motivation is significantly expanded
 - Phase 1 is cut for being too half-assed
@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 May 29, 2015

Contributor

Ok, I talked to @alexcrichton for a long while on IRC, and then rewrite the RFC. In short, I massively elaborated the motivation, and cut phase 1. I'm respond to some his points, which will also offer some summery of what we talked about, though go to the IRC logs if you're really curious.

At a high level, could you clarify the motivation for this RFC a bit more in the text? After reading it it's not clear to me what the RFC is trying to achieve. For example:

  • Is the point of this RFC to enable easy cross compilation of the standard library?
  • Is this enabling swapping in an alternate standard library?
  • Is this just disabling the ability to extern crate std if a crate opts-out?

I think that refining the motivation will help me understand more of the detailed design as well, but I'll make some specific comments below about this.

All three of those are valid motivation for this.

I'm curious where you came up with this? Cargo doesn't pass --sysroot at all.

I extrapolated incorrectly from his comments in the issue I opened long ago. It's fixed in the current RFC. The larger picture is the same either way however: Cargo allows rustc to look in the sysroot for libs.

This worries me a bit, I don't think we want the ability to swap in an alternate standard library in a super easy fashion.

I'm OK with having implicit-deps = false imply #![no_std], I just wanted to point out that technically the two features are orthogonal. The RFC now as some examples of how they can be used orthogonally, but again I don't care very much about this bit.

If you're depending on your own "standard library", then I think it should be just that, a dependency in Cargo.toml which goes through the normal crates.io + Cargo process.

I agree wholeheartedly, but this is currently not possible if crate names + target platform overlap with anything that is in the sysroot.

In general I think it's pretty brittle to swap out the std that an upstream crate is using, praying the APIs match up and the compile still succeeds.

This this can only be accomplished with a Cargo config override, which I consider sufficiently hacky. It's possible phase 1 would have allowed this to happen more accidentally, but that's outta here.

Can you clarify why you can't use Cargo for these kinds of projects today? If you use #![no_std] and don't use extern crate std, you can basically do anything with Cargo you can do with the compiler. For example I have written my own kernel which is built with Cargo and depends on rlibc from crates.io, and it all works out just fine (no standard library exists).

I wanted libcore to be cross compiled automatically, but if on some machines the host and target match (i.e. no cross compile), the build will fail. See the expanded motivation for more details.

I don't quite understand this comment because this can already be done? I can upload a crate which says #![no_std] and doesn't depend on the standard library (only libcore). Can you elaborate on how this is different and what you're thinking here?

I meant do this in a such that all deps can be (cross-)compiled automatically. See rewritten RFC and below.

Overall I think this RFC feels like it's "bolting on" features after the fact instead of creating a well-integrated and smooth system with Cargo. I believe this is also (depending on the motivation) highly inter-related with #1005, and it may wish to be considered here. I think some refinement of the motivation, however, will help guide this RFC, though!

I think I was able to explain my case on IRC. Basically @alexcrichton and I (and presumably you, dear reader, too) would like easy no-brainer cross compiling. If we don't make packages state which of libstd or its backing crates packages they need, future Cargo needs to download/build everything to play it safe, and needs to do that for ever.

By treating these things as normal deps, Cargo can do something adhoc like querying a Mozilla build farm, but can seamlessly transition to just building them like any other package if they get Cargoized. Then all that ad hoc functionality can be thrown out the window.

Finally, if you use a unstable compiler, you can cross compile your own libstd today, and it is nice to do so. While it would be great to standardize libcore, using it effectively requires a fair amount of lang items, and if those are standardized, perhaps all of libcore's implementation could to[1]. Thus, for the time being, freestanding development is probably going to require a compiler that is capable of building libcore, and it just so happens that those people benefit from that the most, so we might as well give them a way to cross compile their whole project---libcore up---today.

[1] @alexcrichton pointed out that with standardization there might be less overlap with the lang needed items to use libcore and the lang items it uses in its implementation, quite possible.

Contributor

Ericson2314 commented May 29, 2015

Ok, I talked to @alexcrichton for a long while on IRC, and then rewrite the RFC. In short, I massively elaborated the motivation, and cut phase 1. I'm respond to some his points, which will also offer some summery of what we talked about, though go to the IRC logs if you're really curious.

At a high level, could you clarify the motivation for this RFC a bit more in the text? After reading it it's not clear to me what the RFC is trying to achieve. For example:

  • Is the point of this RFC to enable easy cross compilation of the standard library?
  • Is this enabling swapping in an alternate standard library?
  • Is this just disabling the ability to extern crate std if a crate opts-out?

I think that refining the motivation will help me understand more of the detailed design as well, but I'll make some specific comments below about this.

All three of those are valid motivation for this.

I'm curious where you came up with this? Cargo doesn't pass --sysroot at all.

I extrapolated incorrectly from his comments in the issue I opened long ago. It's fixed in the current RFC. The larger picture is the same either way however: Cargo allows rustc to look in the sysroot for libs.

This worries me a bit, I don't think we want the ability to swap in an alternate standard library in a super easy fashion.

I'm OK with having implicit-deps = false imply #![no_std], I just wanted to point out that technically the two features are orthogonal. The RFC now as some examples of how they can be used orthogonally, but again I don't care very much about this bit.

If you're depending on your own "standard library", then I think it should be just that, a dependency in Cargo.toml which goes through the normal crates.io + Cargo process.

I agree wholeheartedly, but this is currently not possible if crate names + target platform overlap with anything that is in the sysroot.

In general I think it's pretty brittle to swap out the std that an upstream crate is using, praying the APIs match up and the compile still succeeds.

This this can only be accomplished with a Cargo config override, which I consider sufficiently hacky. It's possible phase 1 would have allowed this to happen more accidentally, but that's outta here.

Can you clarify why you can't use Cargo for these kinds of projects today? If you use #![no_std] and don't use extern crate std, you can basically do anything with Cargo you can do with the compiler. For example I have written my own kernel which is built with Cargo and depends on rlibc from crates.io, and it all works out just fine (no standard library exists).

I wanted libcore to be cross compiled automatically, but if on some machines the host and target match (i.e. no cross compile), the build will fail. See the expanded motivation for more details.

I don't quite understand this comment because this can already be done? I can upload a crate which says #![no_std] and doesn't depend on the standard library (only libcore). Can you elaborate on how this is different and what you're thinking here?

I meant do this in a such that all deps can be (cross-)compiled automatically. See rewritten RFC and below.

Overall I think this RFC feels like it's "bolting on" features after the fact instead of creating a well-integrated and smooth system with Cargo. I believe this is also (depending on the motivation) highly inter-related with #1005, and it may wish to be considered here. I think some refinement of the motivation, however, will help guide this RFC, though!

I think I was able to explain my case on IRC. Basically @alexcrichton and I (and presumably you, dear reader, too) would like easy no-brainer cross compiling. If we don't make packages state which of libstd or its backing crates packages they need, future Cargo needs to download/build everything to play it safe, and needs to do that for ever.

By treating these things as normal deps, Cargo can do something adhoc like querying a Mozilla build farm, but can seamlessly transition to just building them like any other package if they get Cargoized. Then all that ad hoc functionality can be thrown out the window.

Finally, if you use a unstable compiler, you can cross compile your own libstd today, and it is nice to do so. While it would be great to standardize libcore, using it effectively requires a fair amount of lang items, and if those are standardized, perhaps all of libcore's implementation could to[1]. Thus, for the time being, freestanding development is probably going to require a compiler that is capable of building libcore, and it just so happens that those people benefit from that the most, so we might as well give them a way to cross compile their whole project---libcore up---today.

[1] @alexcrichton pointed out that with standardization there might be less overlap with the lang needed items to use libcore and the lang items it uses in its implementation, quite possible.

text/0000-cargo-libstd-awareness.md
+
+
+However rather than passing in libstd and its deps, Cargo lets the compiler look for them as need in
+the compiler's sysroot [specifically `<sysroot>/lib/<target>`]. This is quite coarse in comparison,

This comment has been minimized.

@alexcrichton

alexcrichton May 30, 2015

Member

I think it's important to spell out here that it's far from standard practice to dump libraries in the sysroot, and the only stable library in the sysroot today is libstd. We have been very hesitant to stabilize any more than precisely one library for many of these reasons.

@alexcrichton

alexcrichton May 30, 2015

Member

I think it's important to spell out here that it's far from standard practice to dump libraries in the sysroot, and the only stable library in the sysroot today is libstd. We have been very hesitant to stabilize any more than precisely one library for many of these reasons.

This comment has been minimized.

@Ericson2314

Ericson2314 May 30, 2015

Contributor

I'm a bit confused what you'd like me to elaborate on. I already wanted to emphasize we just do this in the case of libstd and its dependencies---in other words that we are so close---just 1 library away!---from not linking with the sysroot at all.

@Ericson2314

Ericson2314 May 30, 2015

Contributor

I'm a bit confused what you'd like me to elaborate on. I already wanted to emphasize we just do this in the case of libstd and its dependencies---in other words that we are so close---just 1 library away!---from not linking with the sysroot at all.

This comment has been minimized.

@alexcrichton

alexcrichton Jun 1, 2015

Member

Hm re-reading I'm not quite sure what I was thinking... It may have been from the aspect that "and its deps" isn't so relevant in stable Rust today as libstd is the only library that can be implicitly linked to.

@alexcrichton

alexcrichton Jun 1, 2015

Member

Hm re-reading I'm not quite sure what I was thinking... It may have been from the aspect that "and its deps" isn't so relevant in stable Rust today as libstd is the only library that can be implicitly linked to.

This comment has been minimized.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

Ah OK. I'll clarify the situation for stable Rust.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

Ah OK. I'll clarify the situation for stable Rust.

text/0000-cargo-libstd-awareness.md
+ - Packages may link or not link against libs in that directory as they please, with Cargo being
+ none the wiser.
+
+ - Cargo-built crates with the same name as those in there will collide, as the sysroot libs don't

This comment has been minimized.

@alexcrichton

alexcrichton May 30, 2015

Member

This isn't quite true, for example liblibc exists in both the sysroot and on crates.io. Cargo will pass --extern libc=... which overrides everything (including the sysroot).

@alexcrichton

alexcrichton May 30, 2015

Member

This isn't quite true, for example liblibc exists in both the sysroot and on crates.io. Cargo will pass --extern libc=... which overrides everything (including the sysroot).

This comment has been minimized.

@Ericson2314

Ericson2314 May 30, 2015

Contributor

Hmm, I'll need to update my code so I can see why the build is failing, or whether it does today.

@Ericson2314

Ericson2314 May 30, 2015

Contributor

Hmm, I'll need to update my code so I can see why the build is failing, or whether it does today.

text/0000-cargo-libstd-awareness.md
+compile libstd for me. Since I needed to use an unstable compiler anyways, it was possible in
+principle to build absolutely everything I needed with the same `rustc` version. Because of some
+trouble with Cargo and target JSONs, I didn't use a custom target specification, and just used
+`x86_64-gnu-linux`, meaning that depending on platform I was compiling on, I may or may have been

This comment has been minimized.

@alexcrichton

alexcrichton May 30, 2015

Member

s/x86_64-gnu-linux/x86_64-unknown-linux-gnu/

@alexcrichton

alexcrichton May 30, 2015

Member

s/x86_64-gnu-linux/x86_64-unknown-linux-gnu/

This comment has been minimized.

@Ericson2314

Ericson2314 May 30, 2015

Contributor

Oops, thanks!

@Ericson2314

Ericson2314 May 30, 2015

Contributor

Oops, thanks!

text/0000-cargo-libstd-awareness.md
+bikeshead the name. `implicit-deps` is true by default to maintain compatibility with existing
+packages. When true, "std" will be implicitly appended to the list of dependencies.
+
+When Cargo sees a package name it cannot resolve, it will query `rustc` for the default sysroot, and

This comment has been minimized.

@alexcrichton

alexcrichton May 30, 2015

Member

Could you clarify what this means for "a package name it cannot resolve"? For example Cargo does not attempt to resolve the name "std" in any way today, so I'm not sure where this sort of resolution failure will start from.

@alexcrichton

alexcrichton May 30, 2015

Member

Could you clarify what this means for "a package name it cannot resolve"? For example Cargo does not attempt to resolve the name "std" in any way today, so I'm not sure where this sort of resolution failure will start from.

This comment has been minimized.

@alexcrichton

alexcrichton May 30, 2015

Member

Reading a little more, my interpretation is that you're proposing that a crate explicitly declares that it depends on core and std (if the boolean field above is specified), is that right? If so, can you go into some more detail about what the syntax for doing so might be?

@alexcrichton

alexcrichton May 30, 2015

Member

Reading a little more, my interpretation is that you're proposing that a crate explicitly declares that it depends on core and std (if the boolean field above is specified), is that right? If so, can you go into some more detail about what the syntax for doing so might be?

This comment has been minimized.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

Basically, I want it so

implicit-deps = false;

[dependencies]
core = "*" # Or some more appropriate version specifier
alloc = "*"
# ...other crates behind the facade....
std = "*"

and

implicit-deps = true;

and

# implicit-deps is true by default

all mean the same thing.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

Basically, I want it so

implicit-deps = false;

[dependencies]
core = "*" # Or some more appropriate version specifier
alloc = "*"
# ...other crates behind the facade....
std = "*"

and

implicit-deps = true;

and

# implicit-deps is true by default

all mean the same thing.

This comment has been minimized.

@alexcrichton

alexcrichton Jun 15, 2015

Member

Hm ok, your first snippet has a bit of a different interpretation because it means the dependencies like std come from crates.io, which probably isn't going to happen any time soon. Put another way there's no way to express a dependency on std in Cargo.toml today because we distribute it in binary form instead of on crates.io.

I think the reason I'm somewhat uneasy to add implicit-deps is that if you specify implicit-deps there's no way for you to actually link to the standard library. Did you have something in mind for doing that?

@alexcrichton

alexcrichton Jun 15, 2015

Member

Hm ok, your first snippet has a bit of a different interpretation because it means the dependencies like std come from crates.io, which probably isn't going to happen any time soon. Put another way there's no way to express a dependency on std in Cargo.toml today because we distribute it in binary form instead of on crates.io.

I think the reason I'm somewhat uneasy to add implicit-deps is that if you specify implicit-deps there's no way for you to actually link to the standard library. Did you have something in mind for doing that?

This comment has been minimized.

@Ericson2314

Ericson2314 Jun 16, 2015

Contributor

Mmm, this RFC changes the way cargo works so that depended-on crates not on crates.io are looked for in the syroot instead, precisely so we can continue distributing those crates the same way for the time being. This allows:

  • A clear migration path to a future where those crates are defined on crates.io
  • A finer way to distinguish what functionality libraries intended for kernel use need
  • Unstable rust users to locally define those crates so cargo cross-compiles them with the rest of their program.
@Ericson2314

Ericson2314 Jun 16, 2015

Contributor

Mmm, this RFC changes the way cargo works so that depended-on crates not on crates.io are looked for in the syroot instead, precisely so we can continue distributing those crates the same way for the time being. This allows:

  • A clear migration path to a future where those crates are defined on crates.io
  • A finer way to distinguish what functionality libraries intended for kernel use need
  • Unstable rust users to locally define those crates so cargo cross-compiles them with the rest of their program.

This comment has been minimized.

@alexcrichton

alexcrichton Jun 17, 2015

Member

Ah I see what this is saying now, although unfortunately I feel like that's a little too much magic going on under the hood. Cargo understands the "source" for any particular package, and it needs to understand if that source is crates.io or the sysroot ahead of time. Along those lines I think that this needs to have some new source syntax, such as:

[dependencies]
std = { rustc-sysroot = true }
@alexcrichton

alexcrichton Jun 17, 2015

Member

Ah I see what this is saying now, although unfortunately I feel like that's a little too much magic going on under the hood. Cargo understands the "source" for any particular package, and it needs to understand if that source is crates.io or the sysroot ahead of time. Along those lines I think that this needs to have some new source syntax, such as:

[dependencies]
std = { rustc-sysroot = true }

This comment has been minimized.

@Ericson2314

Ericson2314 Jun 17, 2015

Contributor

While yes, that does make Cargo's life easier, does this information really belong in the package metadata? A package just cares what version std is, not how cargo obtained it. Also, if we switch to deploying these crates via crates.io before the end of 1.0, we wouldn't want packages to break because they mandated that std must come from the sysroot.

That said, I'd still rather have that than the status quo. IIRC local packages (with cargo config) override no matter the version so this doesn't prohibit my third bullet point in my previous post.

@Ericson2314

Ericson2314 Jun 17, 2015

Contributor

While yes, that does make Cargo's life easier, does this information really belong in the package metadata? A package just cares what version std is, not how cargo obtained it. Also, if we switch to deploying these crates via crates.io before the end of 1.0, we wouldn't want packages to break because they mandated that std must come from the sysroot.

That said, I'd still rather have that than the status quo. IIRC local packages (with cargo config) override no matter the version so this doesn't prohibit my third bullet point in my previous post.

text/0000-cargo-libstd-awareness.md
+The only new interface proposed is a boolean field in `Cargo.toml` specifying that the package does
+not depend on libstd by default. Note that this is technically orthogonal to Rust's `no_std`, as one
+might want to `use` their own build of libstd by default, or implicitly depend on it but not
+glob-import the prelude. To disambiguate, this field is called `implicit-deps`; please, go ahead and

This comment has been minimized.

@alexcrichton

alexcrichton May 30, 2015

Member

Looking at this from another angle, the only implicitly available crate that is available in stable Rust is std. This means that we have quite a bit of freedom when considering the other crates distributed with Rust itself. For example this field could in theory just be implicit-std = false which passes a flag to the compiler disabling the implicit usage of std, and then the compiler will implicitly deny access to all other crates by default (e.g. even libcore).

Just a note that we have very few constraints today (just the name "std"), and we can do whatever we like with the other deps.

@alexcrichton

alexcrichton May 30, 2015

Member

Looking at this from another angle, the only implicitly available crate that is available in stable Rust is std. This means that we have quite a bit of freedom when considering the other crates distributed with Rust itself. For example this field could in theory just be implicit-std = false which passes a flag to the compiler disabling the implicit usage of std, and then the compiler will implicitly deny access to all other crates by default (e.g. even libcore).

Just a note that we have very few constraints today (just the name "std"), and we can do whatever we like with the other deps.

This comment has been minimized.

@Ericson2314

Ericson2314 May 30, 2015

Contributor

I'm confused, I think the semantics you are describing is exactly what I proposed. implicit-deps controls access std and it's dependencies---whatever those may be.

I went with -deps and not -std because of unstable Rust. But if we want to gear the name around stable Rust, sure.

@Ericson2314

Ericson2314 May 30, 2015

Contributor

I'm confused, I think the semantics you are describing is exactly what I proposed. implicit-deps controls access std and it's dependencies---whatever those may be.

I went with -deps and not -std because of unstable Rust. But if we want to gear the name around stable Rust, sure.

This comment has been minimized.

@alexcrichton

alexcrichton Jun 1, 2015

Member

Oh yes we're definitely thinking of the same thing, I was just wondering if the name implicit-std was better. For example the name implicit-deps seems kinda scary that any crate could be an implicit dependency, when in fact there is only one crate in the stable world that can be an implicit dependency -- libstd.

I agree there are more crates that can be implicitly depended upon, but none of them are stable today, so we may not need to consider them. I think I was just somewhat startled at how this may imply that implicit dependencies are allowed from anywhere (when it's in fact just the sysroot)

@alexcrichton

alexcrichton Jun 1, 2015

Member

Oh yes we're definitely thinking of the same thing, I was just wondering if the name implicit-std was better. For example the name implicit-deps seems kinda scary that any crate could be an implicit dependency, when in fact there is only one crate in the stable world that can be an implicit dependency -- libstd.

I agree there are more crates that can be implicitly depended upon, but none of them are stable today, so we may not need to consider them. I think I was just somewhat startled at how this may imply that implicit dependencies are allowed from anywhere (when it's in fact just the sysroot)

This comment has been minimized.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

I see. Somebody beginning Rust, or beginning just even unstablized Rust, has no idea what dependencies would normally be implicit. One the other hand implicit-std sounds like one is just getting access to std, since Cargo normally doesn't allow one to extern transitive dependencies lest a direct dependency changes its dependencies. A name like implicit-std-and-its-deps would be the most clear, but my is it wordy.

Another option is to change it so that implicit-std = true only gives access to std itself, and dependencies on the crates behind the facade must always be explicit. This would also help on the off-chance that we want to version unstable std independently from its backing crates. Unfortunately, it also breaks backwards compatibility with existing packages, but only those using unstable Rust.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

I see. Somebody beginning Rust, or beginning just even unstablized Rust, has no idea what dependencies would normally be implicit. One the other hand implicit-std sounds like one is just getting access to std, since Cargo normally doesn't allow one to extern transitive dependencies lest a direct dependency changes its dependencies. A name like implicit-std-and-its-deps would be the most clear, but my is it wordy.

Another option is to change it so that implicit-std = true only gives access to std itself, and dependencies on the crates behind the facade must always be explicit. This would also help on the off-chance that we want to version unstable std independently from its backing crates. Unfortunately, it also breaks backwards compatibility with existing packages, but only those using unstable Rust.

This comment has been minimized.

@eternaleye

eternaleye Jun 15, 2015

Another, more concise phrasing might be implicit-stdlibs = true - the 's' punches well above its weight class.

@eternaleye

eternaleye Jun 15, 2015

Another, more concise phrasing might be implicit-stdlibs = true - the 's' punches well above its weight class.

This comment has been minimized.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

Not bad! I tried to think of a short name to capture all that, but came up short.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

Not bad! I tried to think of a short name to capture all that, but came up short.

text/0000-cargo-libstd-awareness.md
+## Future Compatibility
+
+In the future, rather than giving up if libraries are missing Cargo could attempt to download them
+from some build cache. In the farther future, the stdlib libraries may be Cargoized, and Cargo able

This comment has been minimized.

@alexcrichton

alexcrichton May 30, 2015

Member

Ah one point I forgot about previously, which is probably pretty relevant to this, is: the compiler can only link against libraries it previously built. This means that we would need a build cache per-revision of the compiler, which unfortunately makes this much more infeasible :(

@alexcrichton

alexcrichton May 30, 2015

Member

Ah one point I forgot about previously, which is probably pretty relevant to this, is: the compiler can only link against libraries it previously built. This means that we would need a build cache per-revision of the compiler, which unfortunately makes this much more infeasible :(

This comment has been minimized.

@Ericson2314

Ericson2314 May 30, 2015

Contributor

Hmm, we over at NixOS maintain a build cache for Haskell and it works. If you only focus the slower release channels, and prioritize popular packages, it can still be useful.

The idea of a stable ABI scares me, but if/when it happens, that problem goes away too.

@Ericson2314

Ericson2314 May 30, 2015

Contributor

Hmm, we over at NixOS maintain a build cache for Haskell and it works. If you only focus the slower release channels, and prioritize popular packages, it can still be useful.

The idea of a stable ABI scares me, but if/when it happens, that problem goes away too.

This comment has been minimized.

@alexcrichton

alexcrichton Jun 1, 2015

Member

Ah yeah we could definitely pre-cache builds of std for each official release of the compiler, for example, but it may want to be mentioned here as a potential downside. For example all custom builds of the compiler (e.g. nightlies) will not have access to pre-built archives.

@alexcrichton

alexcrichton Jun 1, 2015

Member

Ah yeah we could definitely pre-cache builds of std for each official release of the compiler, for example, but it may want to be mentioned here as a potential downside. For example all custom builds of the compiler (e.g. nightlies) will not have access to pre-built archives.

This comment has been minimized.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

I still don't see the downside. Today, the compiler and library are built together, so we are already building and storing std for each prebuilt compiler. Whether or not std is downloaded with a prebuilt compiler or separately from a crates.io build cache, the build time and storage requirements are the same.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

I still don't see the downside. Today, the compiler and library are built together, so we are already building and storing std for each prebuilt compiler. Whether or not std is downloaded with a prebuilt compiler or separately from a crates.io build cache, the build time and storage requirements are the same.

This comment has been minimized.

@llogiq

llogiq Jun 15, 2015

Contributor

Downloading std separately will stop those who have installed rust to look at it during their commute (case in point: me, a few weeks ago). Not everyone has a fast internet connection everywhere, so there may be other hidden downsides.

@llogiq

llogiq Jun 15, 2015

Contributor

Downloading std separately will stop those who have installed rust to look at it during their commute (case in point: me, a few weeks ago). Not everyone has a fast internet connection everywhere, so there may be other hidden downsides.

This comment has been minimized.

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

Mmm, once std is downloaded once, it doesn't in principle need to be downloaded again. Assuming a local build cache shared between projects is implemented at this point in the future, the compiler's install script could set it up and pre-populate it with std. That way nobody forgets std on their commute :).

@Ericson2314

Ericson2314 Jun 15, 2015

Contributor

Mmm, once std is downloaded once, it doesn't in principle need to be downloaded again. Assuming a local build cache shared between projects is implemented at this point in the future, the compiler's install script could set it up and pre-populate it with std. That way nobody forgets std on their commute :).

@alexcrichton alexcrichton self-assigned this Jun 1, 2015

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jun 17, 2015

Member

@Ericson2314 so I'm still a little uneasy about this RFC, so I had another read through the motivation section to see if I could get myself up to speed. First I'll mention that my frame of mind is basically only geared towards the standard library today. Using any other dependency in the distribution is not stable and we're free to break if necessary. Along these lines the, I think that this RFC is proposing is that the following two manifests are the same:

implicit-std = true # the default setting
implicit-std = false
[dependencies]
std = { rustc-sysroot = true }

So given that, I think one part I'm uneasy about is that I don't understand what practical impact this will have if implemented today. The possible benefits I see are:

  • Cargo might be able to give a somewhat better error message, but this seems like quite a lot of effort just for a better error message.
  • One day, Cargo might be able to not download as much for a target it's missing libraries for. For example just libcore could be downloaded if it knows that libstd isn't depended on.

Other than that though, I can't think of what benefits this may bring (although I may be forgetting some). Does that accurately sum up what you're thinking about this RFC? Or have I forgotten some aspect?

Member

alexcrichton commented Jun 17, 2015

@Ericson2314 so I'm still a little uneasy about this RFC, so I had another read through the motivation section to see if I could get myself up to speed. First I'll mention that my frame of mind is basically only geared towards the standard library today. Using any other dependency in the distribution is not stable and we're free to break if necessary. Along these lines the, I think that this RFC is proposing is that the following two manifests are the same:

implicit-std = true # the default setting
implicit-std = false
[dependencies]
std = { rustc-sysroot = true }

So given that, I think one part I'm uneasy about is that I don't understand what practical impact this will have if implemented today. The possible benefits I see are:

  • Cargo might be able to give a somewhat better error message, but this seems like quite a lot of effort just for a better error message.
  • One day, Cargo might be able to not download as much for a target it's missing libraries for. For example just libcore could be downloaded if it knows that libstd isn't depended on.

Other than that though, I can't think of what benefits this may bring (although I may be forgetting some). Does that accurately sum up what you're thinking about this RFC? Or have I forgotten some aspect?

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Jul 3, 2015

Contributor

The big practical benefit right now is being able to cross compile for unstable freestanding code. https://github.com/RustOS-Fork-Holding-Ground is now mostly up to date, so you can see how that works, and try it yourself. I put this cargo config in a directory with those repos to make things more convenient:

paths = [ "rust/cargo/core"
        , "rust/cargo/alloc"
        , "rust/cargo/rand"
        , "rust/cargo/collections"

        , "bitflags"
        , "libcpu"
        , "lazy-static-spin"
        , "spinlock-rs"

        , "RustOS"
        , "RustOS/bump-pointer"
        ]

I can do this today, but I have to fork everything to add the requisite cargo dependencies to make it work---really annoying. With this RFC, simply having that config file to tell cargo to get these packages from those directories instead of from the sysroot will be adequate.

Contributor

Ericson2314 commented Jul 3, 2015

The big practical benefit right now is being able to cross compile for unstable freestanding code. https://github.com/RustOS-Fork-Holding-Ground is now mostly up to date, so you can see how that works, and try it yourself. I put this cargo config in a directory with those repos to make things more convenient:

paths = [ "rust/cargo/core"
        , "rust/cargo/alloc"
        , "rust/cargo/rand"
        , "rust/cargo/collections"

        , "bitflags"
        , "libcpu"
        , "lazy-static-spin"
        , "spinlock-rs"

        , "RustOS"
        , "RustOS/bump-pointer"
        ]

I can do this today, but I have to fork everything to add the requisite cargo dependencies to make it work---really annoying. With this RFC, simply having that config file to tell cargo to get these packages from those directories instead of from the sysroot will be adequate.

@jdub

This comment has been minimized.

Show comment
Hide comment
@jdub

jdub Jul 4, 2015

I hit this today, wanting to share my target-specific core across a bunch of dependencies. Without this, I'd have to copypasta every dependency to add:

[dependencies]
core = { path = "../../../appropriate/relative/location/core" }

I like the idea of ensuring std dependencies have to be explicitly found and built by telling cargo/rustc to ignore the sysroot.

jdub commented Jul 4, 2015

I hit this today, wanting to share my target-specific core across a bunch of dependencies. Without this, I'd have to copypasta every dependency to add:

[dependencies]
core = { path = "../../../appropriate/relative/location/core" }

I like the idea of ensuring std dependencies have to be explicitly found and built by telling cargo/rustc to ignore the sysroot.

text/0000-cargo-libstd-awareness.md
+
+However rather than passing in libstd and its deps, Cargo lets the compiler look for them as need in
+the compiler's sysroot [specifically `<sysroot>/lib/<target>`]. This is quite coarse in comparison,
+and we loose all the advantages of the previous method:

This comment has been minimized.

@jdub

jdub Jul 4, 2015

Nit: s/loose/lose/

@jdub

jdub Jul 4, 2015

Nit: s/loose/lose/

@jdub

This comment has been minimized.

Show comment
Hide comment
@jdub

jdub Jul 4, 2015

How might this affect the implicit dependencies of dependencies? e.g.

foo

  • no implicit dependencies
  • depends on vendor/core
  • depends on bar
  • has #![no_std] and extern crate core

bar

  • unmodified, so has implicit dependencies
  • implicit dependency on core
  • has #![no_std] and extern crate core

Right now, for a custom target build, bar can't find core, and config path overrides don't override implicit dependencies.

jdub commented Jul 4, 2015

How might this affect the implicit dependencies of dependencies? e.g.

foo

  • no implicit dependencies
  • depends on vendor/core
  • depends on bar
  • has #![no_std] and extern crate core

bar

  • unmodified, so has implicit dependencies
  • implicit dependency on core
  • has #![no_std] and extern crate core

Right now, for a custom target build, bar can't find core, and config path overrides don't override implicit dependencies.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Jul 4, 2015

Contributor

@jdub So in the RFC, I don't let the package specify whether the dependency is supposed to come from the sysroot. That would mean the only way to change the way the package is resolved is with a config path. And since the implicit form is just an alternative syntax for the same thing, the core is overridden in foo and bar.

If we allowed one to explicitly state whether the dependency came from the sysroot, I think your situation is no different from having:

# in Foo
[dependencies.core]
path = "/a/b/c"
# in Bar
[dependencies.core]
path = "/x/y/z"

I'm not sure whether cargo treats this is as two different packages with the same name, or a conflict. Certainly, when two packages have different version constraints on a mutual dependency, Cargo combines the constraints. But version constraints are different than paths/URLs in that they are explicitly designed to be combined.

It's in part because of this that I don't want to let packages specify whether the package is gotten. I feel that is a implementation detail of the way rustc and cargo interact today, and so I don't want to bake it into the semantics of package manifests now and forever.

Contributor

Ericson2314 commented Jul 4, 2015

@jdub So in the RFC, I don't let the package specify whether the dependency is supposed to come from the sysroot. That would mean the only way to change the way the package is resolved is with a config path. And since the implicit form is just an alternative syntax for the same thing, the core is overridden in foo and bar.

If we allowed one to explicitly state whether the dependency came from the sysroot, I think your situation is no different from having:

# in Foo
[dependencies.core]
path = "/a/b/c"
# in Bar
[dependencies.core]
path = "/x/y/z"

I'm not sure whether cargo treats this is as two different packages with the same name, or a conflict. Certainly, when two packages have different version constraints on a mutual dependency, Cargo combines the constraints. But version constraints are different than paths/URLs in that they are explicitly designed to be combined.

It's in part because of this that I don't want to let packages specify whether the package is gotten. I feel that is a implementation detail of the way rustc and cargo interact today, and so I don't want to bake it into the semantics of package manifests now and forever.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jul 6, 2015

Member

@Ericson2314, @jdub, I don't quite follow the logic of how this brings about a benefit right now. If we were to remove the implicit dependency on the standard library, then you'll have to go through all your crates and add std as a dependency (or core). There's nothing stopping you from doing this today, however, and if you do it today the normal std/core libs will not be used.

In other words, I feel like the end state of this RFC (we're explicitly depending on std where possible) is already possible today. I'm having a tough time seeing what state this puts us in that can't be achieved today.

Member

alexcrichton commented Jul 6, 2015

@Ericson2314, @jdub, I don't quite follow the logic of how this brings about a benefit right now. If we were to remove the implicit dependency on the standard library, then you'll have to go through all your crates and add std as a dependency (or core). There's nothing stopping you from doing this today, however, and if you do it today the normal std/core libs will not be used.

In other words, I feel like the end state of this RFC (we're explicitly depending on std where possible) is already possible today. I'm having a tough time seeing what state this puts us in that can't be achieved today.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Jul 6, 2015

Contributor

@alexcrichton Right now, I need to fork all those libraries to make the dependencies explicit. Conversely, nobody using the libraries the "normal way" can use my forks. With this proposal, there is a single way to write the package manifests so that they work for everybody: by default cargo will get the libraries from the sysroot, but with overrides it will build them.

Contributor

Ericson2314 commented Jul 6, 2015

@alexcrichton Right now, I need to fork all those libraries to make the dependencies explicit. Conversely, nobody using the libraries the "normal way" can use my forks. With this proposal, there is a single way to write the package manifests so that they work for everybody: by default cargo will get the libraries from the sysroot, but with overrides it will build them.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jul 7, 2015

Member

@Ericson2314 whoa! That's a use case I haven't seen mentioned in this RFC yet! This RFC to me seems pretty heavyweight to get overrides working, for example cargo could just have a special case about having an override with the package name std to just pass --extern std=... to all compilations.

Member

alexcrichton commented Jul 7, 2015

@Ericson2314 whoa! That's a use case I haven't seen mentioned in this RFC yet! This RFC to me seems pretty heavyweight to get overrides working, for example cargo could just have a special case about having an override with the package name std to just pass --extern std=... to all compilations.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Jul 7, 2015

Contributor

I swore it was in the RFC, but looking back it is indeed missing. Whoops! Now that I have my free standing stuff mostly building, I'll go clean up the RFC---it could certainly use it.

I think that special case is less simple that it sounds. For example, this is much more useful with the crates behind the facade than std itself, but the names of those crates vary between compiler versions. My proposal might seem complex up front, but it explicitly tries to remove special cases now and in the future, which I hope would make it easier to implement and maintain.

Contributor

Ericson2314 commented Jul 7, 2015

I swore it was in the RFC, but looking back it is indeed missing. Whoops! Now that I have my free standing stuff mostly building, I'll go clean up the RFC---it could certainly use it.

I think that special case is less simple that it sounds. For example, this is much more useful with the crates behind the facade than std itself, but the names of those crates vary between compiler versions. My proposal might seem complex up front, but it explicitly tries to remove special cases now and in the future, which I hope would make it easier to implement and maintain.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jul 16, 2015

Member

@Ericson2314 were you going to update this RFC? If not the tools team was going to put this into its final comment period, and I would personally think that while we may want the functionality to be able to override std for the entire compilation it may not want to manifest itself in this form. I also think we need more of a track record with no_std before we can take much more action in this direction.

Member

alexcrichton commented Jul 16, 2015

@Ericson2314 were you going to update this RFC? If not the tools team was going to put this into its final comment period, and I would personally think that while we may want the functionality to be able to override std for the entire compilation it may not want to manifest itself in this form. I also think we need more of a track record with no_std before we can take much more action in this direction.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Jul 17, 2015

Contributor

Sorry, I've been busy with work and other things. I do intend to reword it, and also change it if it cannot accepted in its current form. I'll try to do this by the end of the weekend.

Contributor

Ericson2314 commented Jul 17, 2015

Sorry, I've been busy with work and other things. I do intend to reword it, and also change it if it cannot accepted in its current form. I'll try to do this by the end of the weekend.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jul 17, 2015

Member

Ok, no worries! In that case we'll hold off on FCP for this until it's updated.

Member

alexcrichton commented Jul 17, 2015

Ok, no worries! In that case we'll hold off on FCP for this until it's updated.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Sep 6, 2017

Contributor

Great! I like the eRFC route, in which case I'd probably drop a lot here about sysroots: in the short term, experimenters will be building the stdlib from scratch anyways, and in the long term I hope Cargo won't need/use the sysroot at all.

Contributor

Ericson2314 commented Sep 6, 2017

Great! I like the eRFC route, in which case I'd probably drop a lot here about sysroots: in the short term, experimenters will be building the stdlib from scratch anyways, and in the long term I hope Cargo won't need/use the sysroot at all.

@tamird tamird referenced this pull request in rust-lang/rust Sep 12, 2017

Merged

{compiler-builtins,libc} shim cleanup #44515

@RalfJung

This comment has been minimized.

Show comment
Hide comment
@RalfJung

RalfJung Sep 13, 2017

Member

The strawman proposal would be to add [cargo-features] field to Cargo.toml, which is only available on nightly Cargo's, and don't tie it to version numbers in any way.

Except cargo doesn't complain about known sections, does it? I accidentally used an older cargo with a Cargo.toml that already uses [patch], and old cargo just ignored that section and went on.

So, old stable cargo would just ignore unstable features, while stable rust generally errors out on unstable features.

Unstable features have landed.

For the record, I suppose that's rust-lang/cargo#4433.

Member

RalfJung commented Sep 13, 2017

The strawman proposal would be to add [cargo-features] field to Cargo.toml, which is only available on nightly Cargo's, and don't tie it to version numbers in any way.

Except cargo doesn't complain about known sections, does it? I accidentally used an older cargo with a Cargo.toml that already uses [patch], and old cargo just ignored that section and went on.

So, old stable cargo would just ignore unstable features, while stable rust generally errors out on unstable features.

Unstable features have landed.

For the record, I suppose that's rust-lang/cargo#4433.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Sep 13, 2017

Member

Except cargo doesn't complain about known sections, does it? I accidentally used an older cargo with a Cargo.toml that already uses [patch], and old cargo just ignored that section and went on.

C:\Users\steve\tmp\foo [master +3 ~0 -0 !]> cat .\Cargo.toml
[package]
name = "foo"
version = "0.1.0"
authors = ["steveklabnik <steve@steveklabnik.com>"]

[dependencies]

[unknown]
C:\Users\steve\tmp\foo [master +3 ~0 -0 !]> cargo build
warning: unused manifest key: unknown
   Compiling foo v0.1.0 (file:///C:/Users/steve/tmp/foo)
    Finished dev [unoptimized + debuginfo] target(s) in 0.51 secs
Member

steveklabnik commented Sep 13, 2017

Except cargo doesn't complain about known sections, does it? I accidentally used an older cargo with a Cargo.toml that already uses [patch], and old cargo just ignored that section and went on.

C:\Users\steve\tmp\foo [master +3 ~0 -0 !]> cat .\Cargo.toml
[package]
name = "foo"
version = "0.1.0"
authors = ["steveklabnik <steve@steveklabnik.com>"]

[dependencies]

[unknown]
C:\Users\steve\tmp\foo [master +3 ~0 -0 !]> cargo build
warning: unused manifest key: unknown
   Compiling foo v0.1.0 (file:///C:/Users/steve/tmp/foo)
    Finished dev [unoptimized + debuginfo] target(s) in 0.51 secs
@RalfJung

This comment has been minimized.

Show comment
Hide comment
@RalfJung

RalfJung Sep 14, 2017

Member

@steveklabnik interesting. I tried this (in the src folder of rustc master), and I still do not get a warning:

$ cat Cargo.toml | tail -n 2
[patch_the_things.'https://github.com/rust-lang/cargo']
cargo = { path = "tools/cargo" }
$ cargo +stable update -p gcc
    Updating registry `https://github.com/rust-lang/crates.io-index`
    Updating git repository `https://github.com/rust-lang/cargo`
      Adding cargo v0.23.0 (https://github.com/rust-lang/cargo#bf0b81a3)
      Adding crates-io v0.12.0 (https://github.com/rust-lang/cargo#bf0b81a3)
    Updating gcc v0.3.53 -> v0.3.54

As you can see, no warning. Nightly cargo shows the same behavior. Same for changing the section to [unknown].

If I do the same not in rustc but elsewhere, I do get the expected warning. Very strange.

EDIT: It seems to have something to do with being a pure workspace toml file vs. containing a [package].

Member

RalfJung commented Sep 14, 2017

@steveklabnik interesting. I tried this (in the src folder of rustc master), and I still do not get a warning:

$ cat Cargo.toml | tail -n 2
[patch_the_things.'https://github.com/rust-lang/cargo']
cargo = { path = "tools/cargo" }
$ cargo +stable update -p gcc
    Updating registry `https://github.com/rust-lang/crates.io-index`
    Updating git repository `https://github.com/rust-lang/cargo`
      Adding cargo v0.23.0 (https://github.com/rust-lang/cargo#bf0b81a3)
      Adding crates-io v0.12.0 (https://github.com/rust-lang/cargo#bf0b81a3)
    Updating gcc v0.3.53 -> v0.3.54

As you can see, no warning. Nightly cargo shows the same behavior. Same for changing the section to [unknown].

If I do the same not in rustc but elsewhere, I do get the expected warning. Very strange.

EDIT: It seems to have something to do with being a pure workspace toml file vs. containing a [package].

@nrc

This comment has been minimized.

Show comment
Hide comment
@nrc

nrc Feb 1, 2018

Member

rust-lang/cargo#4959 is a partial (and more minimal) alternative to this RFC. I propose we close this RFC in favour of that proposal.

@rfcbot fcp close

Member

nrc commented Feb 1, 2018

rust-lang/cargo#4959 is a partial (and more minimal) alternative to this RFC. I propose we close this RFC in favour of that proposal.

@rfcbot fcp close

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Feb 1, 2018

Team member @nrc has proposed to close this. The next step is review by the rest of the tagged teams:

No concerns currently listed.

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

rfcbot commented Feb 1, 2018

Team member @nrc has proposed to close this. The next step is review by the rest of the tagged teams:

No concerns currently listed.

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@jonathandturner

This comment has been minimized.

Show comment
Hide comment
@jonathandturner

jonathandturner Feb 1, 2018

Contributor

Go ahead and make me a noop vote

Contributor

jonathandturner commented Feb 1, 2018

Go ahead and make me a noop vote

@mark-i-m

This comment has been minimized.

Show comment
Hide comment
@mark-i-m

mark-i-m Feb 1, 2018

Contributor

I think we should keep this RFC open simply as a tradition 😛

Contributor

mark-i-m commented Feb 1, 2018

I think we should keep this RFC open simply as a tradition 😛

@killercup

This comment has been minimized.

Show comment
Hide comment
@killercup

killercup Feb 1, 2018

Member

@rfcbot reviewed

Member

killercup commented Feb 1, 2018

@rfcbot reviewed

@japaric

This comment has been minimized.

Show comment
Hide comment
@japaric

japaric Feb 1, 2018

Member

@rfcbot reviewed

Member

japaric commented Feb 1, 2018

@rfcbot reviewed

@fitzgen

This comment has been minimized.

Show comment
Hide comment
@fitzgen

fitzgen Feb 1, 2018

Member

@rfcbot reviewed

Member

fitzgen commented Feb 1, 2018

@rfcbot reviewed

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Feb 1, 2018

Contributor

I do agree we should have a more minimal first step, but I don't think it should be (just) rust-lang/cargo#4959. The most important part of this is the interface for specifying stdlilb deps, so we can cleanly bypass all the crufty sysroot and boostrapping implementation details needed today. Much of the discussion thread of this PR also mistakenly focused on uninteresting implementation minutia.

I'm happy to rewrite this or open a new one to that end.

Contributor

Ericson2314 commented Feb 1, 2018

I do agree we should have a more minimal first step, but I don't think it should be (just) rust-lang/cargo#4959. The most important part of this is the interface for specifying stdlilb deps, so we can cleanly bypass all the crufty sysroot and boostrapping implementation details needed today. Much of the discussion thread of this PR also mistakenly focused on uninteresting implementation minutia.

I'm happy to rewrite this or open a new one to that end.

@matklad

This comment has been minimized.

Show comment
Hide comment
@matklad

matklad Feb 1, 2018

Member

+1 to the idea that sysroot should die and that stdlib crates should not be much different from crates from crates.io. We still, after several years, would like std to be present in cargo metadata, for example, to simplify some stuff in IntelliJ :)

Member

matklad commented Feb 1, 2018

+1 to the idea that sysroot should die and that stdlib crates should not be much different from crates from crates.io. We still, after several years, would like std to be present in cargo metadata, for example, to simplify some stuff in IntelliJ :)

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Feb 1, 2018

Contributor

@matklad Yeah that's a great point. When I started preaching death to the sysroot, it was a pretty academic question of avoiding the complexity of needless distinctions. But now with all the new tooling work, be it incremental compilation, IDE support, etc, there's real tangible benefits visible to just about every regular user (far beyond the niche of embedded and OS devs).

Contributor

Ericson2314 commented Feb 1, 2018

@matklad Yeah that's a great point. When I started preaching death to the sysroot, it was a pretty academic question of avoiding the complexity of needless distinctions. But now with all the new tooling work, be it incremental compilation, IDE support, etc, there's real tangible benefits visible to just about every regular user (far beyond the niche of embedded and OS devs).

@nrc

This comment has been minimized.

Show comment
Hide comment
@nrc

nrc Feb 1, 2018

Member

@Ericson2314 I'd like to not lose track of the other issues. I don't think a new RFC is the best way to do that though. Perhaps opening an issue or two on the Cargo repo would be better? Where there are UI things, I'd be keen to experiment on top of the sysroot stuff as a plugin if possible, otherwise as an unstable feature, then RFC later. For other things it would be good to identify small chunks of work which could be isolated and have their own RFCs or implementation.

Member

nrc commented Feb 1, 2018

@Ericson2314 I'd like to not lose track of the other issues. I don't think a new RFC is the best way to do that though. Perhaps opening an issue or two on the Cargo repo would be better? Where there are UI things, I'd be keen to experiment on top of the sysroot stuff as a plugin if possible, otherwise as an unstable feature, then RFC later. For other things it would be good to identify small chunks of work which could be isolated and have their own RFCs or implementation.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Feb 2, 2018

Contributor

@nrc unstable feature -> RFC is fine with me. The stdlib dep is pretty orthogonal to @japaric's sysroot stuff, however.

Contributor

Ericson2314 commented Feb 2, 2018

@nrc unstable feature -> RFC is fine with me. The stdlib dep is pretty orthogonal to @japaric's sysroot stuff, however.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Feb 2, 2018

Contributor

@nrc OK made 2 (and found 1). Those basically correspond to what I implemented in my Cargo PR way back when, and can all be handled "decently independently".

Contributor

Ericson2314 commented Feb 2, 2018

@nrc OK made 2 (and found 1). Those basically correspond to what I implemented in my Cargo PR way back when, and can all be handled "decently independently".

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Mar 8, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

rfcbot commented Mar 8, 2018

🔔 This is now entering its final comment period, as per the review above. 🔔

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Mar 18, 2018

The final comment period is now complete.

rfcbot commented Mar 18, 2018

The final comment period is now complete.

@Centril

This comment has been minimized.

Show comment
Hide comment
@Centril

Centril Mar 18, 2018

Contributor

Closing the RFC per complete FCP #1133 (comment) with a disposition to close.
Thanks @Ericson2314 for the RFC!

Contributor

Centril commented Mar 18, 2018

Closing the RFC per complete FCP #1133 (comment) with a disposition to close.
Thanks @Ericson2314 for the RFC!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment