New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for libcore + no_std stabilization #27701

Closed
alexcrichton opened this Issue Aug 12, 2015 · 133 comments

Comments

Projects
None yet
@alexcrichton
Member

alexcrichton commented Aug 12, 2015

This issue is intended to represent the outstanding issues for stabilizing libcore and allowing its usage on stable Rust. There are a number of features currently associated with libcore:

  • core
  • core_char_ext
  • core_prelude
  • core_slice_ext
  • core_str_ext

(note that core_float will be handled in a separate issue)

The design of libcore largely mirrors that of the standard library (good) but there are a few deviations:

  • core::atomic differs from std::sync::atomic
  • Modules like nonzero, panicking, and array are public

Overall there are a number of tasks that probably need to be done before stabilizing these items:

  • A full audit should be done to ensure that the structure of libcore is the same as the structure of libstd
  • The name core needs to be agreed upon as the stable name for the library
  • The set of extension traits for primitives needs to be decided upon. This strategy of trait-in-core and inherent-above-core should be agreed upon as the best path forward.
  • The set of items in the prelude should be audited to ensure it's a subset of the standard library's

bors added a commit that referenced this issue Aug 22, 2015

Auto merge of #27871 - alexcrichton:stabilize-libcore, r=aturon
These commits move libcore into a state so that it's ready for stabilization, performing some minor cleanup:

* The primitive modules for integers in the standard library were all removed from the source tree as they were just straight reexports of the libcore variants.
* The `core::atomic` module now lives in `core::sync::atomic`. The `core::sync` module is otherwise empty, but ripe for expansion!
* The `core::prelude::v1` module was stabilized after auditing that it is a subset of the standard library's prelude plus some primitive extension traits (char, str, and slice)
* Some unstable-hacks for float parsing errors were shifted around to not use the same unstable hacks (e.g. the `flt2dec` module is now used for "privacy").


After this commit, the remaining large unstable functionality specific to libcore is:

* `raw`, `intrinsics`, `nonzero`, `array`, `panicking`, `simd` -- these modules are all unstable or not reexported in the standard library, so they're just remaining in the same status quo as before
* `num::Float` - this extension trait for floats needs to be audited for functionality (much of that is happening in #27823)  and may also want to be renamed to `FloatExt` or `F32Ext`/`F64Ext`.
* Should the extension traits for primitives be stabilized in libcore?

I believe other unstable pieces are not isolated to just libcore but also affect the standard library.

cc #27701
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 17, 2015

Member

I think that all the major issues here have been resolved, so I'm nominating this for stabilization in 1.5

Member

alexcrichton commented Sep 17, 2015

I think that all the major issues here have been resolved, so I'm nominating this for stabilization in 1.5

@bluss

This comment has been minimized.

Show comment
Hide comment
@bluss

bluss Sep 17, 2015

Contributor

Is it possible to support multiple impl primitive blocks, so that core doesn't have to use as many extension traits?

Contributor

bluss commented Sep 17, 2015

Is it possible to support multiple impl primitive blocks, so that core doesn't have to use as many extension traits?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 18, 2015

Member

Not currently that I'm aware of at least, and that would definitely be part of the stabilization aspect of this issue.

Member

alexcrichton commented Sep 18, 2015

Not currently that I'm aware of at least, and that would definitely be part of the stabilization aspect of this issue.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Sep 23, 2015

Member

Idea: mark the methods on the extension traits as stable, but leave the traits themselves unstable. Works because they're in the prelude. (We did this for SliceConcatExt.)

Not a great permanent story, but it's a place to start.

Member

aturon commented Sep 23, 2015

Idea: mark the methods on the extension traits as stable, but leave the traits themselves unstable. Works because they're in the prelude. (We did this for SliceConcatExt.)

Not a great permanent story, but it's a place to start.

@brson

This comment has been minimized.

Show comment
Hide comment
@brson

brson Sep 23, 2015

Contributor

I want to understand the story for undefined symbols before making this stable. There's a real possibility of people defining these themselves in ways that aren't forward compatible.

Contributor

brson commented Sep 23, 2015

I want to understand the story for undefined symbols before making this stable. There's a real possibility of people defining these themselves in ways that aren't forward compatible.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 24, 2015

Member

This issue/feature is now entering its cycle-long FCP for stabilization in 1.5


@brson, we currently have very few undefined symbols actually:

$ nm -u libcore*

core-10cbabc2.0.o:
                 U fmod
                 U fmodf
                 U memcmp
                 U memcpy
                 U memset
                 U __powidf2
                 U __powisf2
                 U rust_begin_unwind
                 U rust_eh_personality

Of these, rust_begin_unwind and rust_eh_personality are verified by the compiler to exist in the form of a lang item (e.g. it's unstable to define them) whenever you're producing a staticlib, executable, or dylib, so we're covered on that front. The memcmp, memcpy, and memset functions are lowered to by LLVM and I think it's safe to say that our reliance on their meaning will never cause breakage. The __powi{d,s}f2 function dependencies may disappear but otherwise they're provided by compiler-rt so I don't think we have to worry about them.

Otherwise, the only symbols we rely on are fmod and fmodf and that in turn is only because Div and Rem must be implemented in libcore instead of being able to define them in libstd for floats. I'm personally comfortable with this because --gc-sections makes the dependency go away and the symbols also have very well known meanings.

Do you have further concerns in light of this information?

Member

alexcrichton commented Sep 24, 2015

This issue/feature is now entering its cycle-long FCP for stabilization in 1.5


@brson, we currently have very few undefined symbols actually:

$ nm -u libcore*

core-10cbabc2.0.o:
                 U fmod
                 U fmodf
                 U memcmp
                 U memcpy
                 U memset
                 U __powidf2
                 U __powisf2
                 U rust_begin_unwind
                 U rust_eh_personality

Of these, rust_begin_unwind and rust_eh_personality are verified by the compiler to exist in the form of a lang item (e.g. it's unstable to define them) whenever you're producing a staticlib, executable, or dylib, so we're covered on that front. The memcmp, memcpy, and memset functions are lowered to by LLVM and I think it's safe to say that our reliance on their meaning will never cause breakage. The __powi{d,s}f2 function dependencies may disappear but otherwise they're provided by compiler-rt so I don't think we have to worry about them.

Otherwise, the only symbols we rely on are fmod and fmodf and that in turn is only because Div and Rem must be implemented in libcore instead of being able to define them in libstd for floats. I'm personally comfortable with this because --gc-sections makes the dependency go away and the symbols also have very well known meanings.

Do you have further concerns in light of this information?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 24, 2015

Member

Oh and as a further clarification that I forgot to point out, this would also involve stabilizing #![no_std]. As a result I'm tagging this T-lang and renominating to make sure it comes up in triage over there. Triage in all the teams!

Member

alexcrichton commented Sep 24, 2015

Oh and as a further clarification that I forgot to point out, this would also involve stabilizing #![no_std]. As a result I'm tagging this T-lang and renominating to make sure it comes up in triage over there. Triage in all the teams!

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 24, 2015

Member

And to further refine my comment, I remember now that we have more undefined symbols on 32-bit because some 64-bit integer operations frequently require support from compiler-rt. Taking a look at the undefined symbols from i686-unknown-linux-gnu:

         U __divdi3
         U fmod
         U fmodf
         U _GLOBAL_OFFSET_TABLE_
         U memcmp
         U memcpy
         U memset
         U __moddi3
         U __mulodi4
         U __powidf2
         U __powisf2
         U rust_begin_unwind
         U rust_eh_personality
         U __udivdi3
         U __umoddi3

It's basically the same story here, though, nothing unexpected and I don't think we should worry about people relying on their own incorrect versions of compiler-rt intrinsics.

Member

alexcrichton commented Sep 24, 2015

And to further refine my comment, I remember now that we have more undefined symbols on 32-bit because some 64-bit integer operations frequently require support from compiler-rt. Taking a look at the undefined symbols from i686-unknown-linux-gnu:

         U __divdi3
         U fmod
         U fmodf
         U _GLOBAL_OFFSET_TABLE_
         U memcmp
         U memcpy
         U memset
         U __moddi3
         U __mulodi4
         U __powidf2
         U __powisf2
         U rust_begin_unwind
         U rust_eh_personality
         U __udivdi3
         U __umoddi3

It's basically the same story here, though, nothing unexpected and I don't think we should worry about people relying on their own incorrect versions of compiler-rt intrinsics.

@briansmith

This comment has been minimized.

Show comment
Hide comment
@briansmith

briansmith Sep 25, 2015

Some things I noticed when I worked on adapting ring to work with #![no_std] and libcore:

I don't get why I have to use core::foo instead of std::foo. I ended up having to write things like this:

// Re-export `{mem, ptr}` from either `std` or `core` as appropriate. This
// keeps the conditional logic on one place.
#[cfg(feature = "lib_no_std")]
pub use core::{mem, ptr};
#[cfg(not(feature = "lib_no_std"))]
pub use std::{mem, ptr};

Then I refer to, e.g. ring::ffi::mem::uninitialized() and ring::ffi::ptr::null() from all my other modules. But, it doesn't really make sense. I feel like it would be much better to just be able to use std::ffi::ptr::null() in #![no_std].

Also, it is confusing to me about whether #![no_std] applies to the entire crate or only to individual modules. Especially when building using Cargo, it isn't clear to me how it is useful to have a mix of #![no_std] modules within a single crate.

Finally, the documentation should be clearer about how to write a library that automatically works in both #![no_std]mode and non-#![no_std] mode depending on whether the executable is built with or without #![no_std]. Right now I have my library crate define a feature that controls whether or not I enable #![no_std] in the library, but it would be nice if it were automatic.

Some things I noticed when I worked on adapting ring to work with #![no_std] and libcore:

I don't get why I have to use core::foo instead of std::foo. I ended up having to write things like this:

// Re-export `{mem, ptr}` from either `std` or `core` as appropriate. This
// keeps the conditional logic on one place.
#[cfg(feature = "lib_no_std")]
pub use core::{mem, ptr};
#[cfg(not(feature = "lib_no_std"))]
pub use std::{mem, ptr};

Then I refer to, e.g. ring::ffi::mem::uninitialized() and ring::ffi::ptr::null() from all my other modules. But, it doesn't really make sense. I feel like it would be much better to just be able to use std::ffi::ptr::null() in #![no_std].

Also, it is confusing to me about whether #![no_std] applies to the entire crate or only to individual modules. Especially when building using Cargo, it isn't clear to me how it is useful to have a mix of #![no_std] modules within a single crate.

Finally, the documentation should be clearer about how to write a library that automatically works in both #![no_std]mode and non-#![no_std] mode depending on whether the executable is built with or without #![no_std]. Right now I have my library crate define a feature that controls whether or not I enable #![no_std] in the library, but it would be nice if it were automatic.

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Sep 25, 2015

Contributor

@briansmith Why not only use no_std rather than have two modes? If you have optional features that require std, can’t you still unconditionally use use core::… for most of your imports?

Contributor

SimonSapin commented Sep 25, 2015

@briansmith Why not only use no_std rather than have two modes? If you have optional features that require std, can’t you still unconditionally use use core::… for most of your imports?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 25, 2015

Member

@briansmith I totally agree that we can bolster our docs in this respect! As @SimonSapin mentioned, there's actually no need to have a "dual mode" where sometimes you use std and the rest of the time you use core, the intention is for the crate to permanently mention #![no_std] and then everything "just works". In that sense, it sounds like a number of concerns you have would be addressed?

Part of the pain of using #![no_std] today is that if you want to work on both stable and nightly Rust you've got this whole import and duality problem, but that's why I'd like to stabilize #![no_std] :).

Additionally, #![no_std] is only a crate level attribute (which the docs should explain), so there's actually no notion of a "no_std module" vs another module as a whole crate should behave the same way.

Does that help explain things? I can try to bolster up our docs on this subject before we push this over the finish line!

Member

alexcrichton commented Sep 25, 2015

@briansmith I totally agree that we can bolster our docs in this respect! As @SimonSapin mentioned, there's actually no need to have a "dual mode" where sometimes you use std and the rest of the time you use core, the intention is for the crate to permanently mention #![no_std] and then everything "just works". In that sense, it sounds like a number of concerns you have would be addressed?

Part of the pain of using #![no_std] today is that if you want to work on both stable and nightly Rust you've got this whole import and duality problem, but that's why I'd like to stabilize #![no_std] :).

Additionally, #![no_std] is only a crate level attribute (which the docs should explain), so there's actually no notion of a "no_std module" vs another module as a whole crate should behave the same way.

Does that help explain things? I can try to bolster up our docs on this subject before we push this over the finish line!

@briansmith

This comment has been minimized.

Show comment
Hide comment
@briansmith

briansmith Sep 25, 2015

@briansmith I totally agree that we can bolster our docs in this respect! As @SimonSapin mentioned, there's actually no need to have a "dual mode" where sometimes you use std and the rest of the time you use core, the intention is for the crate to permanently mention #![no_std] and then everything "just works". In that sense, it sounds like a number of concerns you have would be addressed?

When I add this to my lib.rs:

#![feature(no_std)]
#![no_std]

I get:

src\digest.rs:28:5: 28:8 error: unresolved import `std::mem`. Maybe a missing `e
xtern crate std`? [E0432]
src\digest.rs:28 use std::mem;

src\aead.rs:27:5: 27:8 error: unresolved import `std`. There is no `std` in `???
` [E0432]
src\aead.rs:27 use std;

What am I doing wrong? The repo is at https://github.com/briansmith/ring, branch "no_std".

@briansmith I totally agree that we can bolster our docs in this respect! As @SimonSapin mentioned, there's actually no need to have a "dual mode" where sometimes you use std and the rest of the time you use core, the intention is for the crate to permanently mention #![no_std] and then everything "just works". In that sense, it sounds like a number of concerns you have would be addressed?

When I add this to my lib.rs:

#![feature(no_std)]
#![no_std]

I get:

src\digest.rs:28:5: 28:8 error: unresolved import `std::mem`. Maybe a missing `e
xtern crate std`? [E0432]
src\digest.rs:28 use std::mem;

src\aead.rs:27:5: 27:8 error: unresolved import `std`. There is no `std` in `???
` [E0432]
src\aead.rs:27 use std;

What am I doing wrong? The repo is at https://github.com/briansmith/ring, branch "no_std".

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 25, 2015

Member

Wow we really do need some documentation on this badly... The detailed design of #![no_std] can currently be found in an RFC, but the gist of it is that #![no_std] doesn't import std at the crate root or the std prelude in every module, but rather core at the crate root and the core prelude in every module.

If you're core-compatible you can basically rewrite all imports to be from core instead of std and you should be good to go!

Member

alexcrichton commented Sep 25, 2015

Wow we really do need some documentation on this badly... The detailed design of #![no_std] can currently be found in an RFC, but the gist of it is that #![no_std] doesn't import std at the crate root or the std prelude in every module, but rather core at the crate root and the core prelude in every module.

If you're core-compatible you can basically rewrite all imports to be from core instead of std and you should be good to go!

@briansmith

This comment has been minimized.

Show comment
Hide comment
@briansmith

briansmith Sep 25, 2015

OK, if I use core:: instead of std:: it works. But if I remove #![no_std] then it stops working:

src\aead.rs:27:5: 27:9 error: unresolved import `core`. There is no `core` in `?
??` [E0432]
src\aead.rs:27 use core;
                   ^~~~
src\aead.rs:27:5: 27:9 help: run `rustc --explain E0432` to see a detailed expla
nation
src\digest.rs:28:5: 28:9 error: unresolved import `core::mem`. Maybe a missing `
extern crate core`? [E0432]/
src\digest.rs:28 use core::mem;

Why is core only implicitly imported when we use #![no_std]? Why not always implicitly import core like std is implicitly imported?

Also, all the documentation uses std::foo instead of core::foo. Are you planning to update the documentation to use core:: ubiquitously? It is confusing to have two sets of names for the same things.

OK, if I use core:: instead of std:: it works. But if I remove #![no_std] then it stops working:

src\aead.rs:27:5: 27:9 error: unresolved import `core`. There is no `core` in `?
??` [E0432]
src\aead.rs:27 use core;
                   ^~~~
src\aead.rs:27:5: 27:9 help: run `rustc --explain E0432` to see a detailed expla
nation
src\digest.rs:28:5: 28:9 error: unresolved import `core::mem`. Maybe a missing `
extern crate core`? [E0432]/
src\digest.rs:28 use core::mem;

Why is core only implicitly imported when we use #![no_std]? Why not always implicitly import core like std is implicitly imported?

Also, all the documentation uses std::foo instead of core::foo. Are you planning to update the documentation to use core:: ubiquitously? It is confusing to have two sets of names for the same things.

@briansmith

This comment has been minimized.

Show comment
Hide comment
@briansmith

briansmith Sep 25, 2015

Also, let's say my module uses core:: everywhere, and it is imported into a program that uses std:: everywhere. Do I have to worry that there will be code duplication between the core:: and :std:: variants of features?

Also, let's say my module uses core:: everywhere, and it is imported into a program that uses std:: everywhere. Do I have to worry that there will be code duplication between the core:: and :std:: variants of features?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 25, 2015

Member

Why is core only implicitly imported when we use #![no_std]?

At 1.0 we didn't inject the core library, and it would be a breaking change to do so now. Additionally I would personally not want it injected by default as std is "The Standard Library" which should be ubiquitously used instead of a mixture of core/std in my opinion. I would see it as a failure mode if a style guideline existed where in normal Rust code one should use core::foo for imports from core and std::foo for std-only imports.

Are you planning to update the documentation to use core:: ubiquitously?

Not currently, the documentation for #![no_std] would indicate that core is simply a subset of the standard library, so if a code example will work with only libcore it'll work by just rewriting std to core imports.

It is confusing to have two sets of names for the same things.

I think this confusion primarily stems from a lack of documentation. I think it's crucial to maintain the distinction of whether code is in a "core only" or "std only" context. It's important to know what abilities you have access to (e.g. can you do I/O?). Beyond that the structures are exactly the same, so once this piece of information is known I don't think it'll be too confusing (but it definitely needs to be explained!)

Do I have to worry that there will be code duplication between the core:: and :std:: variants of features?

That's actually the great part! Because the standard library simply reexports everything from core, a #![no_std] crate can still be used with any other crate seamlessly, there won't be any duplication of anything.

Member

alexcrichton commented Sep 25, 2015

Why is core only implicitly imported when we use #![no_std]?

At 1.0 we didn't inject the core library, and it would be a breaking change to do so now. Additionally I would personally not want it injected by default as std is "The Standard Library" which should be ubiquitously used instead of a mixture of core/std in my opinion. I would see it as a failure mode if a style guideline existed where in normal Rust code one should use core::foo for imports from core and std::foo for std-only imports.

Are you planning to update the documentation to use core:: ubiquitously?

Not currently, the documentation for #![no_std] would indicate that core is simply a subset of the standard library, so if a code example will work with only libcore it'll work by just rewriting std to core imports.

It is confusing to have two sets of names for the same things.

I think this confusion primarily stems from a lack of documentation. I think it's crucial to maintain the distinction of whether code is in a "core only" or "std only" context. It's important to know what abilities you have access to (e.g. can you do I/O?). Beyond that the structures are exactly the same, so once this piece of information is known I don't think it'll be too confusing (but it definitely needs to be explained!)

Do I have to worry that there will be code duplication between the core:: and :std:: variants of features?

That's actually the great part! Because the standard library simply reexports everything from core, a #![no_std] crate can still be used with any other crate seamlessly, there won't be any duplication of anything.

@briansmith

This comment has been minimized.

Show comment
Hide comment
@briansmith

briansmith Sep 25, 2015

I am surprised that it is so difficult to convey how confusing this is.

  • Why should the Rust programmer have to wonder whether core::foo is the same as std::foo or not?
  • Why should the default be to write code that is incompatible with #![no_std]? In particular, let's say a library only uses features of std that are also in core. Why should that library be fail to work in a #![no_std] executable?
  • Why does the documentation recommend coding patterns (using std:: instead of core::) that don't work ubiquitously?
  • Why does Rust have an elaborate style guide about how to format and organize code, but not have canonical names for the most core library features, thus making it impossible to write code in a canonical form that works ubiquitously?

IMO, it would make a lot more sense to NOT have #![no_std] and instead have #![feature(no_std)] that disables everything in std:: that requires runtime support, and then deprecate core::. This would be more consistent with how other sets of features within std:: are enabled/disabled and would allow every library feature to have one, canonical, ubiquitous, name.

I am surprised that it is so difficult to convey how confusing this is.

  • Why should the Rust programmer have to wonder whether core::foo is the same as std::foo or not?
  • Why should the default be to write code that is incompatible with #![no_std]? In particular, let's say a library only uses features of std that are also in core. Why should that library be fail to work in a #![no_std] executable?
  • Why does the documentation recommend coding patterns (using std:: instead of core::) that don't work ubiquitously?
  • Why does Rust have an elaborate style guide about how to format and organize code, but not have canonical names for the most core library features, thus making it impossible to write code in a canonical form that works ubiquitously?

IMO, it would make a lot more sense to NOT have #![no_std] and instead have #![feature(no_std)] that disables everything in std:: that requires runtime support, and then deprecate core::. This would be more consistent with how other sets of features within std:: are enabled/disabled and would allow every library feature to have one, canonical, ubiquitous, name.

@briansmith

This comment has been minimized.

Show comment
Hide comment
@briansmith

briansmith Sep 25, 2015

Here is the development mode that I see for libcore compatible libraries on crates.io:

  1. Somebody makes a library that isn't libcore-compatible.
  2. Somebody else that needs the library to be libcore-compatible submits pull requests to make it libcore-compatible so they can use it in their no_std projects.

Step #2 will likely involve making some features of the third-party library conditional on whether std is being used or whether core is being used. Note that it isn't clear how how this conditional enablement of features should work. In particular, which #![cfg(feature = "???")] should be used? In my suggestion above, this would be #![cfg(feature = "no_std")].

Step #2 will also likely involve changing some std:: references to core:: references. According to the Drawbacks section of the RFC, this is likely to be an source-copmatible-breaking change in the library. For example, if the library exports macros that reference std:: and those changes are changed to core::. This would put the third-party library developers in a Catch-22: they either has to choose between backward-compatibility or libcore compatibility. In my suggestion above, this would not be an issue because there would be no separate core::.

Here is the development mode that I see for libcore compatible libraries on crates.io:

  1. Somebody makes a library that isn't libcore-compatible.
  2. Somebody else that needs the library to be libcore-compatible submits pull requests to make it libcore-compatible so they can use it in their no_std projects.

Step #2 will likely involve making some features of the third-party library conditional on whether std is being used or whether core is being used. Note that it isn't clear how how this conditional enablement of features should work. In particular, which #![cfg(feature = "???")] should be used? In my suggestion above, this would be #![cfg(feature = "no_std")].

Step #2 will also likely involve changing some std:: references to core:: references. According to the Drawbacks section of the RFC, this is likely to be an source-copmatible-breaking change in the library. For example, if the library exports macros that reference std:: and those changes are changed to core::. This would put the third-party library developers in a Catch-22: they either has to choose between backward-compatibility or libcore compatibility. In my suggestion above, this would not be an issue because there would be no separate core::.

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Sep 25, 2015

Contributor

core and std are crates that can be used from other crates with extern crate core; or extern crate std;, just like ring is a crate that can be used with extern crate ring. The only thing that makes them special is that, by default, the compiler implicitly injects extern crate std; at the top of every crate and use std::prelude::v1::*; at the top of every module and every crate. #![no_std] inhibits this injection. The plan is that extern crate core; and use core::prelude::v1::*; are injected instead, but I don’t know if that part is implemented yet.

So #![no_std] only influences which crates are used. It can not influence what’s in these crates, that would require re-compiling these crates. We could have two different crates that are both called std, one of which only includes the core functionality, and use rustc --extern std=some/thing.rlib to disambiguate… I don’t have a opinion on whether this would be a good idea.

There is no such thing as a #![no_std] executable, only #![no_std] crates. Each crate has a number of dependencies, which may or may not (implicitly) include std. An executable links the union of all recursive dependencies. It can mix crates that use #![no_std] and crates that don’t. In that case std will be linked.

std has a number of pub use statements that “reexport” items defined in its own dependencies. These items are not duplicated, just given an (additional) other name. An executable that links std also links std’s own dependencies.

Contributor

SimonSapin commented Sep 25, 2015

core and std are crates that can be used from other crates with extern crate core; or extern crate std;, just like ring is a crate that can be used with extern crate ring. The only thing that makes them special is that, by default, the compiler implicitly injects extern crate std; at the top of every crate and use std::prelude::v1::*; at the top of every module and every crate. #![no_std] inhibits this injection. The plan is that extern crate core; and use core::prelude::v1::*; are injected instead, but I don’t know if that part is implemented yet.

So #![no_std] only influences which crates are used. It can not influence what’s in these crates, that would require re-compiling these crates. We could have two different crates that are both called std, one of which only includes the core functionality, and use rustc --extern std=some/thing.rlib to disambiguate… I don’t have a opinion on whether this would be a good idea.

There is no such thing as a #![no_std] executable, only #![no_std] crates. Each crate has a number of dependencies, which may or may not (implicitly) include std. An executable links the union of all recursive dependencies. It can mix crates that use #![no_std] and crates that don’t. In that case std will be linked.

std has a number of pub use statements that “reexport” items defined in its own dependencies. These items are not duplicated, just given an (additional) other name. An executable that links std also links std’s own dependencies.

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Sep 25, 2015

Contributor

It’s true that our story is not great for crates that want to optionally use #![no_std], but they can still use extern crate core; even if they also depend on std.

Contributor

SimonSapin commented Sep 25, 2015

It’s true that our story is not great for crates that want to optionally use #![no_std], but they can still use extern crate core; even if they also depend on std.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Sep 25, 2015

Member

@briansmith

These are all good questions.

I think a lot of the friction here is due to the std facade, which proposed making the standard library largely a shim over a bunch of smaller crates. (Over time, we've scaled this design back significantly.)

The idea with the facade is that for the common case -- when you're using Rust in user space -- you get a simple, monolithic API surface. In particular, some of the modules that std exports are not straight re-exports of core, but include more functionality.

An alternative would have been to not have std at all, and instead ship with a set of crates (libcore, libcollections, libio, etc) some of which work in kernel space. That would effectively make things "no_std by default". Personally, I regret not pushing harder on that alternative, but doing that now would be a massive breaking change.

But given that std is stable, what can we do today?

  • We could expose core unconditionally and try to address the macro issues. Ultimately, this might lead in the direction of favoring core imports over std imports by default, but that leaves us in a pretty messy position imo (with reexports in std that are not recommended for use).
  • Ooh, I just saw @SimonSapin's suggestion:

We could have two different crates that are both called std, one of which only includes the core functionality, and use rustc --extern std=some/thing.rlib to disambiguate… I don’t have a opinion on whether this would be a good idea.

I actually think we should strongly consider such a direction. Part of the pre-stabilization work was to ensure that the module structure in core matched std. We'd probably need to do a bit more work to make sure they are in proper alignment, but the idea that, with a flip of a switch, you get something just like std but with no features that depend on system services... seems rather appealing to me. Certainly it would make migrating to "no_std" (which we'd want to rename...) trivial, modulo not actually depending on extra std features.

Member

aturon commented Sep 25, 2015

@briansmith

These are all good questions.

I think a lot of the friction here is due to the std facade, which proposed making the standard library largely a shim over a bunch of smaller crates. (Over time, we've scaled this design back significantly.)

The idea with the facade is that for the common case -- when you're using Rust in user space -- you get a simple, monolithic API surface. In particular, some of the modules that std exports are not straight re-exports of core, but include more functionality.

An alternative would have been to not have std at all, and instead ship with a set of crates (libcore, libcollections, libio, etc) some of which work in kernel space. That would effectively make things "no_std by default". Personally, I regret not pushing harder on that alternative, but doing that now would be a massive breaking change.

But given that std is stable, what can we do today?

  • We could expose core unconditionally and try to address the macro issues. Ultimately, this might lead in the direction of favoring core imports over std imports by default, but that leaves us in a pretty messy position imo (with reexports in std that are not recommended for use).
  • Ooh, I just saw @SimonSapin's suggestion:

We could have two different crates that are both called std, one of which only includes the core functionality, and use rustc --extern std=some/thing.rlib to disambiguate… I don’t have a opinion on whether this would be a good idea.

I actually think we should strongly consider such a direction. Part of the pre-stabilization work was to ensure that the module structure in core matched std. We'd probably need to do a bit more work to make sure they are in proper alignment, but the idea that, with a flip of a switch, you get something just like std but with no features that depend on system services... seems rather appealing to me. Certainly it would make migrating to "no_std" (which we'd want to rename...) trivial, modulo not actually depending on extra std features.

@perlun

This comment has been minimized.

Show comment
Hide comment
@perlun

perlun Sep 25, 2015

Contributor

I actually think we should strongly consider such a direction. Part of the pre-stabilization work was to ensure that the module structure in core matched std. We'd probably need to do a bit more work to make sure they are in proper alignment, but the idea that, with a flip of a switch, you get something just like std but with no features that depend on system services... seems rather appealing to me. Certainly it would make migrating to "no_std" (which we'd want to rename...) trivial, modulo not actually depending on extra std features.

Not implying the current state of affairs is optimal in any way - wouldn't there be a great risk that such an approach would be confusing to our beloved users? I mean, if std sometimes is the current std and sometimes core... hmm. 😄 It doesn't sound entirely optimal; it sounds like it could look confusing.

But I do agree that it will probably have massive advantages to people developing crates. The risk that someone makes stuff that only works with std and not with core even though it doesn't depend on any std-only features would be much smaller; making a crate that works in both "standalone" and "regular" modes is merely a matter of "not depending on extra std features" like you put it.

Contributor

perlun commented Sep 25, 2015

I actually think we should strongly consider such a direction. Part of the pre-stabilization work was to ensure that the module structure in core matched std. We'd probably need to do a bit more work to make sure they are in proper alignment, but the idea that, with a flip of a switch, you get something just like std but with no features that depend on system services... seems rather appealing to me. Certainly it would make migrating to "no_std" (which we'd want to rename...) trivial, modulo not actually depending on extra std features.

Not implying the current state of affairs is optimal in any way - wouldn't there be a great risk that such an approach would be confusing to our beloved users? I mean, if std sometimes is the current std and sometimes core... hmm. 😄 It doesn't sound entirely optimal; it sounds like it could look confusing.

But I do agree that it will probably have massive advantages to people developing crates. The risk that someone makes stuff that only works with std and not with core even though it doesn't depend on any std-only features would be much smaller; making a crate that works in both "standalone" and "regular" modes is merely a matter of "not depending on extra std features" like you put it.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Sep 25, 2015

Member

@perlun

Definitely a legitimate concern! I think if we went this direction, we'd want to take steps to guarantee that the "core version" of std is always a subset of the "full version".

That said, this isn't so different from the situation in the std::os::* modules, for example -- where even the existence of a given module in that part of the tree depends on what platform you're on. This is basically saying that std is conditionally compiled in two ways, with one API a subset of the other. This kind of thing also happens in the Cargo universe due to features.

I also think that the majority of users wouldn't need to ever think about this -- that's part of the goal here, after all. And I think the reason is exactly as you spelled out; it's just a simple toggle.

Member

aturon commented Sep 25, 2015

@perlun

Definitely a legitimate concern! I think if we went this direction, we'd want to take steps to guarantee that the "core version" of std is always a subset of the "full version".

That said, this isn't so different from the situation in the std::os::* modules, for example -- where even the existence of a given module in that part of the tree depends on what platform you're on. This is basically saying that std is conditionally compiled in two ways, with one API a subset of the other. This kind of thing also happens in the Cargo universe due to features.

I also think that the majority of users wouldn't need to ever think about this -- that's part of the goal here, after all. And I think the reason is exactly as you spelled out; it's just a simple toggle.

@petrochenkov

This comment has been minimized.

Show comment
Hide comment
@petrochenkov

petrochenkov Sep 25, 2015

Contributor

I may be saying nonsense, but can't #![no_std] inject the libcore import with renaming?

extern crate core as std;

If someone needs the choice between core and std based on a feature, he can do the same thing:

#[cfg(my_feature)]
extern crate core as std;
#[cfg(not(my_feature))]
extern crate std;

// The rest of the source code uses name std
// ...
Contributor

petrochenkov commented Sep 25, 2015

I may be saying nonsense, but can't #![no_std] inject the libcore import with renaming?

extern crate core as std;

If someone needs the choice between core and std based on a feature, he can do the same thing:

#[cfg(my_feature)]
extern crate core as std;
#[cfg(not(my_feature))]
extern crate std;

// The rest of the source code uses name std
// ...
@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Sep 25, 2015

Member

@petrochenkov That seems viable, yes. I think the bigger question here is whether this is the desired behavior :)

Member

aturon commented Sep 25, 2015

@petrochenkov That seems viable, yes. I think the bigger question here is whether this is the desired behavior :)

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 28, 2015

Member

@briansmith

I am surprised that it is so difficult to convey how confusing this is.

I'm sorry you feel this way, but the current design of libcore/libstd has been around for quite some time now and I'm trying to tease apart concerns that exist from a lack of documentation (which there is a sore need for in this area!) and those which are architectural.

@SimonSapin has an excellent explanation as to the current state of affairs in terms of technical details, and the plan he mentioned is indeed implemented. The key aspect of this is that there is zero duplication among libcore and libstd. Large parts of libstd are simply reexports of libcore, so libraries using one can easily interoperate with another.


@aturon, @SimonSapin, @perlun

It's certainly been considered in the past that std just has a different meaning in a #![no_std] context, but I am personally not a fan of this behavior because I fear that it ends up being confusing when you're looking at a module to know whether it's a "std module" or a "no_std module". By having two separate names for this concept, std and core, it's clear what's what and where any snippet of code is expected to work.

There are also many technical details which simply naturally entail us having two libraries. For example the compiler doesn't auto-inject any --extern flags, it'd be difficult having two binaries on the filesystem which are both "libstd-like", and it'd be interesting to see how the standard library itself would be interested in terms of linking to a "lower libstd". These sorts of things would want to be ironed out, but it seems better to me to stay within the same world we have today of linking, building, and naming libstd.


@briansmith

I think your example may not quite be as dire of a situation as it may seem, for example @petrochenkov has what I think is the right idea to have features which bring in std + more functionality. For example:

// src/lib.rs
#![no_std]
#[cfg(feature = "foo")]
extern crate std;

use core::{ ... };

#[cfg(feature = "foo")]
mod foo;

// src/foo.rs

use std::prelude::v1::*;

// .. code using libstd

In a world like this, once a library is #![no_std] it's always no_std (e.g. nothing conditional). When features are added which require the standard library they they can selectively link to and import the standard library in the modules that are implementing that functionality. There's certainly still a problem with macros, but that's a known deficiency of macros which I don't personally believe justifies a redesign of libcore/libstd to have the two named the same.

Note that this is of course only one method of encoding this sort of pattern, I expect more to arise! I think, however, that having a no_std-but-still-with-some-std-features library may not incur quite the annotation burden overhead you may be thinking.

Member

alexcrichton commented Sep 28, 2015

@briansmith

I am surprised that it is so difficult to convey how confusing this is.

I'm sorry you feel this way, but the current design of libcore/libstd has been around for quite some time now and I'm trying to tease apart concerns that exist from a lack of documentation (which there is a sore need for in this area!) and those which are architectural.

@SimonSapin has an excellent explanation as to the current state of affairs in terms of technical details, and the plan he mentioned is indeed implemented. The key aspect of this is that there is zero duplication among libcore and libstd. Large parts of libstd are simply reexports of libcore, so libraries using one can easily interoperate with another.


@aturon, @SimonSapin, @perlun

It's certainly been considered in the past that std just has a different meaning in a #![no_std] context, but I am personally not a fan of this behavior because I fear that it ends up being confusing when you're looking at a module to know whether it's a "std module" or a "no_std module". By having two separate names for this concept, std and core, it's clear what's what and where any snippet of code is expected to work.

There are also many technical details which simply naturally entail us having two libraries. For example the compiler doesn't auto-inject any --extern flags, it'd be difficult having two binaries on the filesystem which are both "libstd-like", and it'd be interesting to see how the standard library itself would be interested in terms of linking to a "lower libstd". These sorts of things would want to be ironed out, but it seems better to me to stay within the same world we have today of linking, building, and naming libstd.


@briansmith

I think your example may not quite be as dire of a situation as it may seem, for example @petrochenkov has what I think is the right idea to have features which bring in std + more functionality. For example:

// src/lib.rs
#![no_std]
#[cfg(feature = "foo")]
extern crate std;

use core::{ ... };

#[cfg(feature = "foo")]
mod foo;

// src/foo.rs

use std::prelude::v1::*;

// .. code using libstd

In a world like this, once a library is #![no_std] it's always no_std (e.g. nothing conditional). When features are added which require the standard library they they can selectively link to and import the standard library in the modules that are implementing that functionality. There's certainly still a problem with macros, but that's a known deficiency of macros which I don't personally believe justifies a redesign of libcore/libstd to have the two named the same.

Note that this is of course only one method of encoding this sort of pattern, I expect more to arise! I think, however, that having a no_std-but-still-with-some-std-features library may not incur quite the annotation burden overhead you may be thinking.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Sep 28, 2015

Member

@alexcrichton

It's certainly been considered in the past that std just has a different meaning in a #![no_std] context, but I am personally not a fan of this behavior because I fear that it ends up being confusing when you're looking at a module to know whether it's a "std module" or a "no_std module". By having two separate names for this concept, std and core, it's clear what's what and where any snippet of code is expected to work.

I'd like to understand a bit better the precise confusion you're worried about here.

If we take this approach, I imagine we'd ensure that core is a perfect subset of std, so that everything you can access and use in core is present and has the same meaning as with std. The only difference is that std contains a larger API surface.

Can you spell out in more detail what precise confusion you're worried about, given the above?

Put differently, I think the whole point that @briansmith is trying to get at is that it seems desirable to be able to write a program that begins by targeting std and, if it doesn't use non-core features, can be trivially made to run in a no_std setting.

I think that's, perhaps, the core disconnect here: how important do you consider it that it be easy -- even trivial -- to "port" libraries to no_std when they don't use non-core features?

There are also many technical details which simply naturally entail us having two libraries. For example the compiler doesn't auto-inject any --extern flags, it'd be difficult having two binaries on the filesystem which are both "libstd-like", and it'd be interesting to see how the standard library itself would be interested in terms of linking to a "lower libstd". These sorts of things would want to be ironed out, but it seems better to me to stay within the same world we have today of linking, building, and naming libstd.

That indeed sounds complicated, but is there some deep reason we can't take @petrochenkov's simple suggestion?

Member

aturon commented Sep 28, 2015

@alexcrichton

It's certainly been considered in the past that std just has a different meaning in a #![no_std] context, but I am personally not a fan of this behavior because I fear that it ends up being confusing when you're looking at a module to know whether it's a "std module" or a "no_std module". By having two separate names for this concept, std and core, it's clear what's what and where any snippet of code is expected to work.

I'd like to understand a bit better the precise confusion you're worried about here.

If we take this approach, I imagine we'd ensure that core is a perfect subset of std, so that everything you can access and use in core is present and has the same meaning as with std. The only difference is that std contains a larger API surface.

Can you spell out in more detail what precise confusion you're worried about, given the above?

Put differently, I think the whole point that @briansmith is trying to get at is that it seems desirable to be able to write a program that begins by targeting std and, if it doesn't use non-core features, can be trivially made to run in a no_std setting.

I think that's, perhaps, the core disconnect here: how important do you consider it that it be easy -- even trivial -- to "port" libraries to no_std when they don't use non-core features?

There are also many technical details which simply naturally entail us having two libraries. For example the compiler doesn't auto-inject any --extern flags, it'd be difficult having two binaries on the filesystem which are both "libstd-like", and it'd be interesting to see how the standard library itself would be interested in terms of linking to a "lower libstd". These sorts of things would want to be ironed out, but it seems better to me to stay within the same world we have today of linking, building, and naming libstd.

That indeed sounds complicated, but is there some deep reason we can't take @petrochenkov's simple suggestion?

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Sep 28, 2015

Contributor

The only difference is that std contains a larger API surface.

I’m worried about things that exist in both with the same name but with different behavior. The only example I can think of right now is the try! macro from std that uses std::convert::From in the error case, while try! from core does not. I don’t know if there are other cases (not necessarily macros).

Contributor

SimonSapin commented Sep 28, 2015

The only difference is that std contains a larger API surface.

I’m worried about things that exist in both with the same name but with different behavior. The only example I can think of right now is the try! macro from std that uses std::convert::From in the error case, while try! from core does not. I don’t know if there are other cases (not necessarily macros).

@briansmith

This comment has been minimized.

Show comment
Hide comment
@briansmith

briansmith Sep 28, 2015

It's certainly been considered in the past that std just has a different meaning in a #![no_std] context, but I am personally not a fan of this behavior because I fear that it ends up being confusing when you're looking at a module to know whether it's a "std module" or a "no_std module". By having two separate names for this concept, std and core, it's clear what's what and where any snippet of code is expected to work.

As somebody building stuff for no_std, in particular building libraries and services in Rust for microcontrollers with and without operating systems, I would say that the thing you are optimizing for isn't very useful. I can easily just build the code for a target that is missing some features and find out what features that code really needs, just like I do in C and C++. And, the cost of making no_std mode so different from std mode is very painful. Note that C and C++ don't force the programmer to learn two same-but-different APIs when targeting subsets. In general, the POSIX model of subsetting works well and IMO that is what programmers are expecting.

Anyway, while working on some stuff this weekend, I came across another problem that the current RFC does not address, AFAICT: Although I want my library to be #![no_std], I don't want the tests sub-modules of my library to be #![no_std]. In particular, my tests require Vec and HashMap. So, there needs to be some way of doing this:

#![no_std]
....
#[cfg(test)]
mod tests {
    #![no_std + liballoc]
    ...
}

I would also like to point out that many applications and libraries for embedded will require #![no_std] + liballoc, some will require #![no_std] + liballoc + threads, some will require #![no_std] + a clock that can return the current time, etc.

Also, AFAICT, there needs to be some way to tell the compiler to avoid generating the unwinding code to reduce code size when panic! is implemented to just reset the device instead of unwind.

Regarding injecting use core as std, that seems like a big improvement to the proposal. However, either way, the documentation at https://doc.rust-lang.org/std/ should be updated to indicate which features are in core and which aren't. In particular, there shouldn't be separate documentation at https://doc.rust-lang.org/core. In general, anything that exposes libcore as a separate entity to developers, except the developers of the standard library, seems like an indication of something going wrong.

It's certainly been considered in the past that std just has a different meaning in a #![no_std] context, but I am personally not a fan of this behavior because I fear that it ends up being confusing when you're looking at a module to know whether it's a "std module" or a "no_std module". By having two separate names for this concept, std and core, it's clear what's what and where any snippet of code is expected to work.

As somebody building stuff for no_std, in particular building libraries and services in Rust for microcontrollers with and without operating systems, I would say that the thing you are optimizing for isn't very useful. I can easily just build the code for a target that is missing some features and find out what features that code really needs, just like I do in C and C++. And, the cost of making no_std mode so different from std mode is very painful. Note that C and C++ don't force the programmer to learn two same-but-different APIs when targeting subsets. In general, the POSIX model of subsetting works well and IMO that is what programmers are expecting.

Anyway, while working on some stuff this weekend, I came across another problem that the current RFC does not address, AFAICT: Although I want my library to be #![no_std], I don't want the tests sub-modules of my library to be #![no_std]. In particular, my tests require Vec and HashMap. So, there needs to be some way of doing this:

#![no_std]
....
#[cfg(test)]
mod tests {
    #![no_std + liballoc]
    ...
}

I would also like to point out that many applications and libraries for embedded will require #![no_std] + liballoc, some will require #![no_std] + liballoc + threads, some will require #![no_std] + a clock that can return the current time, etc.

Also, AFAICT, there needs to be some way to tell the compiler to avoid generating the unwinding code to reduce code size when panic! is implemented to just reset the device instead of unwind.

Regarding injecting use core as std, that seems like a big improvement to the proposal. However, either way, the documentation at https://doc.rust-lang.org/std/ should be updated to indicate which features are in core and which aren't. In particular, there shouldn't be separate documentation at https://doc.rust-lang.org/core. In general, anything that exposes libcore as a separate entity to developers, except the developers of the standard library, seems like an indication of something going wrong.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Nov 15, 2015

Member

I agree with @phil-opp. Given the move toward exposing the facade, no_std is a precise and accurate description of what you're getting, and avoids unnecessary breakage.

use_core is unclear/suggestive of the wrong things, as others have mentioned, but core_only could also work.

Member

aturon commented Nov 15, 2015

I agree with @phil-opp. Given the move toward exposing the facade, no_std is a precise and accurate description of what you're getting, and avoids unnecessary breakage.

use_core is unclear/suggestive of the wrong things, as others have mentioned, but core_only could also work.

@bstrie

This comment has been minimized.

Show comment
Hide comment
@bstrie

bstrie Nov 28, 2015

Contributor

-1 to #![use_core], it fails to suggest that you're jettisoning the rest of std. I'm partial to the #![freestanding] name proposed earlier (down with underscores!), but #![only_core] is okay too (way way better than #![no_std], which sounds like a slogan for prophylactics).

Contributor

bstrie commented Nov 28, 2015

-1 to #![use_core], it fails to suggest that you're jettisoning the rest of std. I'm partial to the #![freestanding] name proposed earlier (down with underscores!), but #![only_core] is okay too (way way better than #![no_std], which sounds like a slogan for prophylactics).

@bstrie

This comment has been minimized.

Show comment
Hide comment
@bstrie

bstrie Nov 28, 2015

Contributor

Would anyone like to write up an RFC for the "std is nothing but re-exports" idea? @Ericson2314? @posborne?

Contributor

bstrie commented Nov 28, 2015

Would anyone like to write up an RFC for the "std is nothing but re-exports" idea? @Ericson2314? @posborne?

@thepowersgang

This comment has been minimized.

Show comment
Hide comment
@thepowersgang

thepowersgang Nov 29, 2015

Contributor

I'm probably for #[use_core]. While it's very close to C's freestanding concept, C's version doesn't require any libraries except the compiler support library (libgcc/libcompiler-rt).

Contributor

thepowersgang commented Nov 29, 2015

I'm probably for #[use_core]. While it's very close to C's freestanding concept, C's version doesn't require any libraries except the compiler support library (libgcc/libcompiler-rt).

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Dec 1, 2015

Contributor

@alexcrichton back to cross-compiling the standard library: say we give stable rustc a list of trusted hashes, the idea being that it will allow one to use unstable features only if the code to be compiled matches those hashes?

While the definitions of the standard library change, (the stable part of) their interface is stable, and the only thing one can do without invalidating the hash is use code a dependency. Therefore this doesn't invalid the "if your project compiles with the stable compiler it will continue to compile" principle.

Contributor

Ericson2314 commented Dec 1, 2015

@alexcrichton back to cross-compiling the standard library: say we give stable rustc a list of trusted hashes, the idea being that it will allow one to use unstable features only if the code to be compiled matches those hashes?

While the definitions of the standard library change, (the stable part of) their interface is stable, and the only thing one can do without invalidating the hash is use code a dependency. Therefore this doesn't invalid the "if your project compiles with the stable compiler it will continue to compile" principle.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Dec 1, 2015

Contributor

@bstrie I guess if we all agree that using the crates behind the facade is an acceptable, even promoted, thing to do, then I don't care as much about what happens to std. I rather just deprecate it once all such crates are so stabilized.

If someone else takes the lead with such an RFC I'd be happy to review it however.

Contributor

Ericson2314 commented Dec 1, 2015

@bstrie I guess if we all agree that using the crates behind the facade is an acceptable, even promoted, thing to do, then I don't care as much about what happens to std. I rather just deprecate it once all such crates are so stabilized.

If someone else takes the lead with such an RFC I'd be happy to review it however.

@dschatzberg

This comment has been minimized.

Show comment
Hide comment
@dschatzberg

dschatzberg Dec 1, 2015

Contributor

I'm against stabilizing #[no_std] as is.
My biggest concerns are:

  1. I compare the #[no_std] approach where library and binary authors opt-in to not using the runtime-dependent parts of the standard library to that of C and C++ where the decision for library authors is implicit. C/C++ developers write code as normal, if it doesn't use runtime-dependent parts, then binary developers can just use that library. This means that runtime-free binary developers can use libraries that were not intended to be used in such an environment. I've personally benefited from this while using C++.
  2. Macros - using core:: vs std:: in macros is a well-known issue. It forces developers to choose between macros which just work in a #[no_std] environment (when they use core::) vs. macros which just work otherwise. Is there any proposed solution? (Maybe default to extern crate core as std?) I feel very uncomfortable leaving this unresolved before stabilizing any parts of #[no_std].
Contributor

dschatzberg commented Dec 1, 2015

I'm against stabilizing #[no_std] as is.
My biggest concerns are:

  1. I compare the #[no_std] approach where library and binary authors opt-in to not using the runtime-dependent parts of the standard library to that of C and C++ where the decision for library authors is implicit. C/C++ developers write code as normal, if it doesn't use runtime-dependent parts, then binary developers can just use that library. This means that runtime-free binary developers can use libraries that were not intended to be used in such an environment. I've personally benefited from this while using C++.
  2. Macros - using core:: vs std:: in macros is a well-known issue. It forces developers to choose between macros which just work in a #[no_std] environment (when they use core::) vs. macros which just work otherwise. Is there any proposed solution? (Maybe default to extern crate core as std?) I feel very uncomfortable leaving this unresolved before stabilizing any parts of #[no_std].
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 2, 2015

Member

@Ericson2314 although unrelated to this issue, that seems plausible to have just a list of hashes.

@dschatzberg

Thanks for weighing in! I think the point about implicit opt-in vs explicit is somewhat subjective, however, as I would personally not want to run into a situation that looks like:

  • Library A implicitly works with libcore, but doesn't realize it
  • Library B starts depending on A, but requires that everything is libcore-compatible
  • Library A then changes to require libstd

This means that seemingly innocuous changes in your library can cause silent breakage downstream unknowingly. With an explicit declaration that a crate is core-compatible this problem goes away.

The point about macros is certainly valid and should be considered during stabilization. Currently I personally perceive it as a minor enough nuisance that it shouldn't block stabilization of this entire ecosystem as a whole. Macros working with core:: can be used in std crates via extern crate core at the root if necessary, and hopefully with macros 2.0 we can solve this problem.

Member

alexcrichton commented Dec 2, 2015

@Ericson2314 although unrelated to this issue, that seems plausible to have just a list of hashes.

@dschatzberg

Thanks for weighing in! I think the point about implicit opt-in vs explicit is somewhat subjective, however, as I would personally not want to run into a situation that looks like:

  • Library A implicitly works with libcore, but doesn't realize it
  • Library B starts depending on A, but requires that everything is libcore-compatible
  • Library A then changes to require libstd

This means that seemingly innocuous changes in your library can cause silent breakage downstream unknowingly. With an explicit declaration that a crate is core-compatible this problem goes away.

The point about macros is certainly valid and should be considered during stabilization. Currently I personally perceive it as a minor enough nuisance that it shouldn't block stabilization of this entire ecosystem as a whole. Macros working with core:: can be used in std crates via extern crate core at the root if necessary, and hopefully with macros 2.0 we can solve this problem.

@dschatzberg

This comment has been minimized.

Show comment
Hide comment
@dschatzberg

dschatzberg Dec 2, 2015

Contributor

@alexcrichton
This problem seems quite minor. Firstly, library authors can already make changes which cause downstream breakage. Secondly, those depending on implicitly libcore libraries could just use fixed versions and upgrade explicitly. Notice that this just shifts burden from the author of library A to library B (who has the requirement of nostd). I think this makes much more sense as nostd is much less likely and already more onerous.

As for macros, what do you expect? All macros will be modified to use core::? All users of macros will extern crate core as std? Neither of these are appealing.

Contributor

dschatzberg commented Dec 2, 2015

@alexcrichton
This problem seems quite minor. Firstly, library authors can already make changes which cause downstream breakage. Secondly, those depending on implicitly libcore libraries could just use fixed versions and upgrade explicitly. Notice that this just shifts burden from the author of library A to library B (who has the requirement of nostd). I think this makes much more sense as nostd is much less likely and already more onerous.

As for macros, what do you expect? All macros will be modified to use core::? All users of macros will extern crate core as std? Neither of these are appealing.

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Dec 2, 2015

Contributor

Using #![no_std] currently involves jumping through some hoops. Stabilizing it (or something like it) probably involves stabilizing other things. For example, http://os.phil-opp.com/setup-rust.html guides through writing a kernel in Rust.

  • Currently, the user needs to define eh_personality and panic_fmt language items, but #[lang_item] is unstable. Should it be stabilized as-is?
  • Linking without a libc can cause errors like undefined reference to 'memcpy'. The rlibc crate fixes this, but it’s hard to discover if a tutorial doesn’t tell you about it. I’m not sure what can be done here.
  • Same with undefined reference tofmodf'.--gc-sectionsis recommended, but what if%` is used? Should fmod and fmodf be added to rlibc?
  • undefined reference to_Unwind_Resume', fixed with-Z no-landing-pads. Is this the best way to deal with this? Where is_Unwind_Resume` normally defined?
Contributor

SimonSapin commented Dec 2, 2015

Using #![no_std] currently involves jumping through some hoops. Stabilizing it (or something like it) probably involves stabilizing other things. For example, http://os.phil-opp.com/setup-rust.html guides through writing a kernel in Rust.

  • Currently, the user needs to define eh_personality and panic_fmt language items, but #[lang_item] is unstable. Should it be stabilized as-is?
  • Linking without a libc can cause errors like undefined reference to 'memcpy'. The rlibc crate fixes this, but it’s hard to discover if a tutorial doesn’t tell you about it. I’m not sure what can be done here.
  • Same with undefined reference tofmodf'.--gc-sectionsis recommended, but what if%` is used? Should fmod and fmodf be added to rlibc?
  • undefined reference to_Unwind_Resume', fixed with-Z no-landing-pads. Is this the best way to deal with this? Where is_Unwind_Resume` normally defined?
@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Dec 2, 2015

Contributor

To be clear, getting rid of std is most elegant solution to both @dschatzberg points. I hope that its elegance (as opposed to practicality) is uncontroversial.

I think 1 can be refined--it's less that the silent transition from freestanding-capable to not freestanding-capable is the goal, but rather that there be only one style (a "normal form") for writing code that in principle should work free-standing. Right now there is two, with core and with std, and that's just awkward. Of course in a world without std its still possible to extern crate more than needed, but that's a very simple fix, and I consider cleaning imports part of the "stylistic normalization" process.

Now the elastic/subset std also means there is two ways to write such code, but at least they are equivalent in terms of the burdens they place downstream. Right now that the std way is more conventional but core strictly nicer to downstream. That means not only is there two different styles, but meaningful trade-offs between them.

@SimonSapin I'd hope we'd just be stabilizing the things in core that are already stable in std.

Contributor

Ericson2314 commented Dec 2, 2015

To be clear, getting rid of std is most elegant solution to both @dschatzberg points. I hope that its elegance (as opposed to practicality) is uncontroversial.

I think 1 can be refined--it's less that the silent transition from freestanding-capable to not freestanding-capable is the goal, but rather that there be only one style (a "normal form") for writing code that in principle should work free-standing. Right now there is two, with core and with std, and that's just awkward. Of course in a world without std its still possible to extern crate more than needed, but that's a very simple fix, and I consider cleaning imports part of the "stylistic normalization" process.

Now the elastic/subset std also means there is two ways to write such code, but at least they are equivalent in terms of the burdens they place downstream. Right now that the std way is more conventional but core strictly nicer to downstream. That means not only is there two different styles, but meaningful trade-offs between them.

@SimonSapin I'd hope we'd just be stabilizing the things in core that are already stable in std.

@dschatzberg

This comment has been minimized.

Show comment
Hide comment
@dschatzberg

dschatzberg Dec 2, 2015

Contributor

@SimonSapin
It seems perfectly reasonable to stabilize #![no_std] and core without the rest. This allows stable #[no_std] library crates. I think the further discussion about stabilizing the necessary support for #[no_std] binaries can be done separately.

  • fmod is only used for floating point modulus not modulus in general
  • _Unwind_Resume is usually defined in some runtime support library like libgcc
Contributor

dschatzberg commented Dec 2, 2015

@SimonSapin
It seems perfectly reasonable to stabilize #![no_std] and core without the rest. This allows stable #[no_std] library crates. I think the further discussion about stabilizing the necessary support for #[no_std] binaries can be done separately.

  • fmod is only used for floating point modulus not modulus in general
  • _Unwind_Resume is usually defined in some runtime support library like libgcc
@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Dec 2, 2015

Contributor

std is not going anywhere. It’s not just a facade, a lot of things are defined there. And there’s plenty of code out there with use std::something; that we promised not to break.

Contributor

SimonSapin commented Dec 2, 2015

std is not going anywhere. It’s not just a facade, a lot of things are defined there. And there’s plenty of code out there with use std::something; that we promised not to break.

@dschatzberg

This comment has been minimized.

Show comment
Hide comment
@dschatzberg

dschatzberg Dec 2, 2015

Contributor

@Ericson2314
I'm not quite sure what you mean about there being two styles for writing freestanding code. Could you elaborate?

@alexcrichton
I believe that making freestanding implicit and allowing users to unknowingly make their library non-freestanding is strictly better than making freestanding explicit. In the explicit case, developers who want to construct freestanding libraries must, well, be explicit about this, otherwise their library cannot be used in a freestanding environment. In the implicit case, those same developers who want to construct freestanding libraries can still use a lint to enforce their desire. The only difference is that freestanding users can still use those libraries whose developers haven't considered the freestanding environment. As a freestanding user this is a pure win - I get more libraries available to me.

Contributor

dschatzberg commented Dec 2, 2015

@Ericson2314
I'm not quite sure what you mean about there being two styles for writing freestanding code. Could you elaborate?

@alexcrichton
I believe that making freestanding implicit and allowing users to unknowingly make their library non-freestanding is strictly better than making freestanding explicit. In the explicit case, developers who want to construct freestanding libraries must, well, be explicit about this, otherwise their library cannot be used in a freestanding environment. In the implicit case, those same developers who want to construct freestanding libraries can still use a lint to enforce their desire. The only difference is that freestanding users can still use those libraries whose developers haven't considered the freestanding environment. As a freestanding user this is a pure win - I get more libraries available to me.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Dec 2, 2015

Contributor

@dschatzberg are you on IRC by any chance? might take a few iterations for me to express that clearly :)

Contributor

Ericson2314 commented Dec 2, 2015

@dschatzberg are you on IRC by any chance? might take a few iterations for me to express that clearly :)

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Dec 2, 2015

Contributor

@SimonSapin Sure no cold-turkey removal, but we can deprecate it, and since it is so easy to define as a facade, it's easy to keep around without removing. The downside is more appearing fickle making such a visible change after 1.0, but it's more eye-cactching than actually heavy-weight.

Contributor

Ericson2314 commented Dec 2, 2015

@SimonSapin Sure no cold-turkey removal, but we can deprecate it, and since it is so easy to define as a facade, it's easy to keep around without removing. The downside is more appearing fickle making such a visible change after 1.0, but it's more eye-cactching than actually heavy-weight.

@dschatzberg

This comment has been minimized.

Show comment
Hide comment
@dschatzberg

dschatzberg Dec 2, 2015

Contributor

@Ericson2314 yes - same user name

Contributor

dschatzberg commented Dec 2, 2015

@Ericson2314 yes - same user name

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Dec 2, 2015

Contributor

Ok, edited #27701 (comment) to hopefully clear up any confusion.

Contributor

Ericson2314 commented Dec 2, 2015

Ok, edited #27701 (comment) to hopefully clear up any confusion.

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Dec 2, 2015

Contributor

-1 on deprecating std. It’s just not worth the cost, which is huge: every existing program that is not #![no_std] and not trivial imports something from std.

Contributor

SimonSapin commented Dec 2, 2015

-1 on deprecating std. It’s just not worth the cost, which is huge: every existing program that is not #![no_std] and not trivial imports something from std.

@bstrie

This comment has been minimized.

Show comment
Hide comment
@bstrie

bstrie Dec 2, 2015

Contributor

"Deprecating std" isn't sufficiently well-defined at the moment, we'd need to see an RFC for it before it could be discussed seriously.

Contributor

bstrie commented Dec 2, 2015

"Deprecating std" isn't sufficiently well-defined at the moment, we'd need to see an RFC for it before it could be discussed seriously.

@bstrie

This comment has been minimized.

Show comment
Hide comment
@bstrie

bstrie Dec 2, 2015

Contributor

Regardless of whether or not this RFC is accepted, I'd like to propose that the actual stabilization PR be blocked on resolving #23355 , because otherwise we'll have the compiler guiding people into wholly silly situations.

Contributor

bstrie commented Dec 2, 2015

Regardless of whether or not this RFC is accepted, I'd like to propose that the actual stabilization PR be blocked on resolving #23355 , because otherwise we'll have the compiler guiding people into wholly silly situations.

@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Dec 2, 2015

Contributor

I really like the explicit #![no_std]. If you accidentally add some std dependent feature, it fails to compile. If no_std were implicit, it would silently include std and break all downstream crates. Travis would not catch this either, because it compiles without problems.

Contributor

phil-opp commented Dec 2, 2015

I really like the explicit #![no_std]. If you accidentally add some std dependent feature, it fails to compile. If no_std were implicit, it would silently include std and break all downstream crates. Travis would not catch this either, because it compiles without problems.

@bstrie

This comment has been minimized.

Show comment
Hide comment
@bstrie

bstrie Dec 2, 2015

Contributor

Actually never mind about #23355 , I just determined that it's already possible to follow rustc's advice on the stable branch to produce the silly situations that I had in mind.

Contributor

bstrie commented Dec 2, 2015

Actually never mind about #23355 , I just determined that it's already possible to follow rustc's advice on the stable branch to produce the silly situations that I had in mind.

@dschatzberg

This comment has been minimized.

Show comment
Hide comment
@dschatzberg

dschatzberg Dec 2, 2015

Contributor

@phil-opp
Implicit '#![no_std]` + a lint would resolve this issue.

Contributor

dschatzberg commented Dec 2, 2015

@phil-opp
Implicit '#![no_std]` + a lint would resolve this issue.

@bluss

This comment has been minimized.

Show comment
Hide comment
@bluss

bluss Dec 2, 2015

Contributor

Having no_std explicit or not is a red herring. I think it's very reasonable that if you have a crate using only core features, making the core-only compatible should only be an explicit attribute away. This requires (? may be an assumption too far!) a design where we use the name std for the library in any mode, we just don't fill it with as much features if you compile core-only.

Contributor

bluss commented Dec 2, 2015

Having no_std explicit or not is a red herring. I think it's very reasonable that if you have a crate using only core features, making the core-only compatible should only be an explicit attribute away. This requires (? may be an assumption too far!) a design where we use the name std for the library in any mode, we just don't fill it with as much features if you compile core-only.

@dschatzberg

This comment has been minimized.

Show comment
Hide comment
@dschatzberg

dschatzberg Dec 2, 2015

Contributor

@bluss I don't completely agree. I still think it's beneficial for freestanding users to use libraries for which their use case wasn't intended. While an explicit attribute is a relatively minor change, it's still a burden on the library developer to think to do it.

Contributor

dschatzberg commented Dec 2, 2015

@bluss I don't completely agree. I still think it's beneficial for freestanding users to use libraries for which their use case wasn't intended. While an explicit attribute is a relatively minor change, it's still a burden on the library developer to think to do it.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Dec 2, 2015

Contributor

@bstrie "deprecating std" in my mind would be encouraging imports from crates behind the facade rather than std were the former is stable. Once everything is available from a crate behind the facade, then std cannot be used at all without tripping a "try use core/alloc/instead for foobar" warning.

I would write an RFC for that, but I rather iron out the kinks in using the crates behind the facade exclusively first, so it's clear what we would be migrating to in the deprecate std RFC.

Contributor

Ericson2314 commented Dec 2, 2015

@bstrie "deprecating std" in my mind would be encouraging imports from crates behind the facade rather than std were the former is stable. Once everything is available from a crate behind the facade, then std cannot be used at all without tripping a "try use core/alloc/instead for foobar" warning.

I would write an RFC for that, but I rather iron out the kinks in using the crates behind the facade exclusively first, so it's clear what we would be migrating to in the deprecate std RFC.

@bluss

This comment has been minimized.

Show comment
Hide comment
@bluss

bluss Dec 2, 2015

Contributor

Just my opinion, but deprecating or linting std is a non-starter, no use in exploring that idea.

Contributor

bluss commented Dec 2, 2015

Just my opinion, but deprecating or linting std is a non-starter, no use in exploring that idea.

@phil-opp

This comment has been minimized.

Show comment
Hide comment
@phil-opp

phil-opp Dec 2, 2015

Contributor

AFAIK the std library is more than a facade. It implements everything operating specific (like threads, files, network, etc.).

So to deprecate std, we would need to move those parts to a new crate. And then the "normal" user would have to think about implementation details. For example, Hashmap is only in std (because it uses a thread local random number generator) but BTreeMap is also in collections.

Contributor

phil-opp commented Dec 2, 2015

AFAIK the std library is more than a facade. It implements everything operating specific (like threads, files, network, etc.).

So to deprecate std, we would need to move those parts to a new crate. And then the "normal" user would have to think about implementation details. For example, Hashmap is only in std (because it uses a thread local random number generator) but BTreeMap is also in collections.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Dec 2, 2015

Contributor

@phil-opp I'd consider finishing making std a facade a good thing whether or not we keep std. For example, embedded Systems ("internet of things") may have networking but not filesystems. As for your example, only the default hasher param uses the OS like that. I think everybody rather put HashMap in collections, and just have a wrapper type alias that adds the default hasher param in std, but there was some language restriction that prevented that.

Contributor

Ericson2314 commented Dec 2, 2015

@phil-opp I'd consider finishing making std a facade a good thing whether or not we keep std. For example, embedded Systems ("internet of things") may have networking but not filesystems. As for your example, only the default hasher param uses the OS like that. I think everybody rather put HashMap in collections, and just have a wrapper type alias that adds the default hasher param in std, but there was some language restriction that prevented that.

@bstrie

This comment has been minimized.

Show comment
Hide comment
@bstrie

bstrie Dec 2, 2015

Contributor

There are too many details to the std-as-facade/deprecating-std to propose and discuss them here in this comment section. Again, I implore those interested to write up an RFC if you wish to pursue it.

Contributor

bstrie commented Dec 2, 2015

There are too many details to the std-as-facade/deprecating-std to propose and discuss them here in this comment section. Again, I implore those interested to write up an RFC if you wish to pursue it.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 3, 2015

Member

The libs team discussed this during triage yesterday, as well as the core team two days ago, as well as the lang team a few weeks ago, and the conclusion is to stabilize.

The extension traits for primitive types in libcore will be left unstable for now with stable methods (e.g. the SliceConcatExt trick and otherwise all method stabilities will match that of the standard library.

Member

alexcrichton commented Dec 3, 2015

The libs team discussed this during triage yesterday, as well as the core team two days ago, as well as the lang team a few weeks ago, and the conclusion is to stabilize.

The extension traits for primitive types in libcore will be left unstable for now with stable methods (e.g. the SliceConcatExt trick and otherwise all method stabilities will match that of the standard library.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 16, 2015

Member

Closing as this was stabilized in #30187

Member

alexcrichton commented Dec 16, 2015

Closing as this was stabilized in #30187

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment