Skip to content

Commit

Permalink
Rust std from 6c4e236b955ba6a2dd8ef8e054f50ff64135a8be
Browse files Browse the repository at this point in the history
  • Loading branch information
brson committed Jul 28, 2015
1 parent 8e405cf commit e50c2b1
Showing 1 changed file with 347 additions and 0 deletions.
347 changes: 347 additions & 0 deletions std.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,347 @@
// Copyright 2012-2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

The Rust license header. This is on pretty much everything maintained by The Rust Project itself.

Rust has an unusual dual license: MIT and/or Apache 2. The Apache license is included because it ensures important patent grants from contributors, while MIT is left as an option because Apache 2 is not GPL 2 compatible (though it is GPL 3 and LGPL 2). Rust was relicensed in November 2012 from MIT-only.

Note that the date says 2012-2014 even though certainly somebody modified this file in 2015. There is occasional discussion about the amount of effort that should be expended on updating these headers and the current consensus is 'none'.


//! # The Rust Standard Library
//!
//! The Rust Standard Library provides the essential runtime
//! functionality for building portable Rust software.

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

This next section is just the crate docs, which correspond to the web docs. Scroll on.

//!
//! The Rust Standard Library is available to all Rust crates by
//! default, just as if contained an `extern crate std` import at the
//! crate root. Therefore the standard library can be accessed in
//! `use` statements through the path `std`, as in `use std::thread`,
//! or in expressions through the absolute path `::std`, as in
//! `::std::thread::sleep_ms(100)`.
//!
//! Furthermore, the standard library defines [The Rust
//! Prelude](prelude/index.html), a small collection of items, mostly
//! traits, that are imported into and available in every module.
//!
//! ## What is in the standard library
//!
//! The standard library is a set of minimal, battle-tested
//! core types and shared abstractions for the [broader Rust
//! ecosystem](https://crates.io) to build on.
//!
//! The [primitive types](#primitives), though not defined in the
//! standard library, are documented here, as are the predefined
//! [macros](#macros).
//!
//! ## Containers and collections
//!
//! The [`option`](option/index.html) and
//! [`result`](result/index.html) modules define optional and
//! error-handling types, `Option` and `Result`. The
//! [`iter`](iter/index.html) module defines Rust's iterator trait,
//! [`Iterator`](iter/trait.Iterator.html), which works with the `for`
//! loop to access collections.
//!
//! The common container type, `Vec`, a growable vector backed by an array,
//! lives in the [`vec`](vec/index.html) module. Contiguous, unsized regions
//! of memory, `[T]`, commonly called "slices", and their borrowed versions,
//! `&[T]`, commonly called "borrowed slices", are built-in types for which the
//! [`slice`](slice/index.html) module defines many methods.
//!
//! `&str`, a UTF-8 string, is a built-in type, and the standard library
//! defines methods for it on a variety of traits in the
//! [`str`](str/index.html) module. Rust strings are immutable;
//! use the `String` type defined in [`string`](string/index.html)
//! for a mutable string builder.
//!
//! For converting to strings use the [`format!`](fmt/index.html)
//! macro, and for converting from strings use the
//! [`FromStr`](str/trait.FromStr.html) trait.
//!
//! Data may be shared by placing it in a reference-counted box or the
//! [`Rc`](rc/index.html) type, and if further contained in a [`Cell`
//! or `RefCell`](cell/index.html), may be mutated as well as shared.
//! Likewise, in a concurrent setting it is common to pair an
//! atomically-reference-counted box, [`Arc`](sync/struct.Arc.html),
//! with a [`Mutex`](sync/struct.Mutex.html) to get the same effect.
//!
//! The [`collections`](collections/index.html) module defines maps,
//! sets, linked lists and other typical collection types, including
//! the common [`HashMap`](collections/struct.HashMap.html).
//!
//! ## Platform abstractions and I/O
//!
//! Besides basic data types, the standard library is largely concerned
//! with abstracting over differences in common platforms, most notably
//! Windows and Unix derivatives.
//!
//! Common types of I/O, including [files](fs/struct.File.html),
//! [TCP](net/struct.TcpStream.html),
//! [UDP](net/struct.UdpSocket.html), are defined in the
//! [`io`](io/index.html), [`fs`](fs/index.html), and
//! [`net`](net/index.html) modules.
//!
//! The [`thread`](thread/index.html) module contains Rust's threading
//! abstractions. [`sync`](sync/index.html) contains further
//! primitive shared memory types, including
//! [`atomic`](sync/atomic/index.html) and
//! [`mpsc`](sync/mpsc/index.html), which contains the channel types
//! for message passing.

// Do not remove on snapshot creation. Needed for bootstrap. (Issue #22364)
#![cfg_attr(stage0, feature(custom_attribute))]

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

OK, now is where it's going to start getting interesting, entering the attribute swamp.

#[cfg_attr(stage0, ...)] is something you see pretty often in the Rust code base. Rust is bootstrapped in 3 nearly-identical stages, 0, 1, 2. To make incompatible changes to the compiler though, it is common to use conditional compilation to cause the compiler to do the old thing in stage 0, while being compiled by the old compiler, and the new thing in later stages. Here, #[cfg_attr] applies feature(custom_attribute) only when building with the stage 0 snapshot compiler.

Usually this type of code is removed after a 'snapshot' (when the binary bootstrap compiler is updated and rebuilt), but this one comes with a dire warning not to remove the attribute after the snap.

This comment has been minimized.

Copy link
@Gankra

Gankra Jul 29, 2015

To elaborate on the bootstrap, the three stages are:

  • Stage0: download some blessed previous build of the compiler and build the current source with that to produce a new compiler. This compiler should have all the latest language and library features, but there are several reasons why it shouldn't be used. At this point the most prominent reasons are:
    • It was built with an old compiler, and therefore its generated code may not have all the sweetest new perf.
    • It was built with tons of weird cfg(stage0) tweaks that are intended to be removed -- we really want to verify the new code
  • Stage1: use the compiler we just built to build the current source again. This compiler has all the latest features, was built with the "true" source (no cfg(stage0)'s) and its generated binary should be the state-of-the-art. This compiler is in principle adequate except for one key detail: it's standard library and plugins were built with the the cfg(stage0)-y compiler. This may mean that its standard library has a different ABI than the the one it outputs! This would be disastrous to ship, as most code it compiles will want to link with the standard library!
  • Stage2: use the previous compiler to built the current souce again. This compiler should now be perfect: it was built by a compiler that has been thoroughly cleansed of the stage0 pollution. All of its standard library and plugins should have the same ABI that it generates. Note that it's not a problem that it was built by something with a weird standard library because the Rust source is entirely self hosted. It doesn't assume an external standard library to link against; it provides its own. This doesn't imply that everything is effectively no_std. Much of the standard library itself is -- but rustdoc and rustc in particular get to behave like normal rust programs with std defined.
#![crate_name = "std"]
#![stable(feature = "rust1", since = "1.0.0")]
#![staged_api]
#![crate_type = "rlib"]
#![crate_type = "dylib"]
#![doc(html_logo_url = "http://www.rust-lang.org/logos/rust-logo-128x128-blk-v2.png",
html_favicon_url = "https://doc.rust-lang.org/favicon.ico",
html_root_url = "http://doc.rust-lang.org/nightly/",
html_playground_url = "http://play.rust-lang.org/",
test(no_crate_inject, attr(deny(warnings))),
test(attr(allow(dead_code, deprecated, unused_variables, unused_mut))))]

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

Now this is some good stuff here! Some if it is common, some magical.

crate_name is obvious, so is crate_type: here it's telling rustc to produce both an 'rlib' and a 'dylib'. 'rlib's are the default library type in Rust (it's what you would get if you wrote #![crate_type = "lib"]) and are statically linked plus various Rust-specific metadata. dylibs are less common, but std needs to be distributed in both forms.

Although they are stable attributes neither crate_name nor crate_type are common in typical Rust code because Cargo tells the compiler that information.

The #![staged_api] attribute says this crate participates in feature staging, by which lots of the APIs in the standard library are tagged either 'stable' or 'unstable' (notice above that the entire std crate is declared 'stable' as of 1.0.0 via the #![stable(feature = "rust1", since = "1.0.0")] attribute). This is what controls the restrictions to which APIs can be used on stable releases. API staging is itself an unstable feature, controlled by #![feature(staged_api)].

Finally, the doc attribute. Oh, rustdoc. It's not well-documented all the things rustdoc does; most rustdoc features are added to address specific in-tree issues with the standard docs, and its internals are rumored to have become a scrambled and tortured mess (this is the second rustdoc. Do I sense a third coming on?). The URLs all control various links in the generated documentation. We never change that html_root_url link for stable releases, even though it says 'nightly'.

test(no_crate_inject) tells not to inject an extern crate item into the documentation test cases since that would shadow the default extern crate std and cause an error. All the other test(attr(...)) stuff is flipping lints in the documentation test cases.


#![feature(alloc)]
#![feature(allow_internal_unstable)]
#![feature(associated_consts)]
#![feature(borrow_state)]
#![feature(box_raw)]
#![feature(box_syntax)]
#![feature(char_internals)]
#![feature(clone_from_slice)]
#![feature(collections)]
#![feature(collections_bound)]
#![feature(const_fn)]
#![feature(core)]
#![feature(core_float)]
#![feature(core_intrinsics)]
#![feature(core_prelude)]
#![feature(core_simd)]
#![feature(fnbox)]
#![feature(heap_api)]
#![feature(int_error_internals)]
#![feature(into_cow)]
#![feature(iter_order)]
#![feature(lang_items)]
#![feature(libc)]
#![feature(linkage, thread_local, asm)]
#![feature(macro_reexport)]
#![feature(slice_concat_ext)]
#![feature(slice_position_elem)]
#![feature(no_std)]
#![feature(oom)]
#![feature(optin_builtin_traits)]
#![feature(rand)]
#![feature(raw)]
#![feature(reflect_marker)]
#![feature(slice_bytes)]
#![feature(slice_patterns)]
#![feature(staged_api)]
#![feature(str_char)]
#![feature(str_internals)]
#![feature(unboxed_closures)]
#![feature(unicode)]
#![feature(unique)]
#![feature(unsafe_no_drop_flag, filling_drop)]
#![feature(vec_push_all)]
#![feature(vec_resize)]
#![feature(wrapping)]
#![feature(zero_one)]
#![cfg_attr(windows, feature(str_utf16))]
#![cfg_attr(test, feature(float_from_str_radix, range_inclusive, float_extras))]
#![cfg_attr(test, feature(test, rustc_private, float_consts))]
#![cfg_attr(target_env = "msvc", feature(link_args))]

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

The Rust Standard Library uses like every unstable feature! This means the standard library itself cannot be built by stable Rust (it uses hacks to work around this limitation).

The #[cfg_attr(...)] attributes conditionally create feature attributes if certain configuration items are defined, e.g. the str_utf16 feature is only needed on windows, when the windows configuration item is defined.

This comment has been minimized.

Copy link
@Gankra

Gankra Jul 29, 2015

Interestingly the standard library actually uses only a fraction of the unstable features because it's almost entirely re-exports of other crates, and re-exporting an unstable feature doesn't require opting into it. These are all the features that code only defined in std actually uses.

For reference, there are about 50 features here, but the distribution exposes almost 200.


// Don't link to std. We are std.
#![no_std]

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

Yes, the standard library cannot link to the standard library .


#![allow(trivial_casts)]
#![deny(missing_docs)]

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

The standard library insists on being documented (#![deny(missing_docs)]). I don't know why it's so keen on trivial casts.

This comment has been minimized.

Copy link
@chris-morgan

chris-morgan Jul 29, 2015

trivial_casts is a lint which was put in as warn but has since been downgraded to allow because it was too annoying, as many of the “problems” it was reporting required either introducing temporary variables (onerous and ugly) or type ascription (which hasn’t been implemented yet). As a simple example, it wants let x = y as *const T; to be written let x: *const T = y;. Or, in a world with type ascription, let x = y: *const T;.

Because trivial-casts now defaults to allow, that attribute could therefore be removed.


#[cfg(test)] extern crate test;
#[cfg(test)] #[macro_use] extern crate log;

#[macro_use]
#[macro_reexport(assert, assert_eq, debug_assert, debug_assert_eq,
unreachable, unimplemented, write, writeln)]
extern crate core;

#[macro_use]
#[macro_reexport(vec, format)]
extern crate collections as core_collections;

#[allow(deprecated)] extern crate rand as core_rand;
extern crate alloc;
extern crate rustc_unicode;
extern crate libc;

#[macro_use] #[no_link] extern crate rustc_bitflags;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

These are the crates std depends on. core is the most important: it is the pure-Rust library at the bottom of the runtime stack. It has no dependencies at all and forms the foundation of all Rust software in every environment, defining such things as the marker traits, operators, methods on slices, Option and more. As you can see with the macro_reexport (which is again feature-gated), core also defines some of the important macros.

The core library defines a lot of stuff but not enough - not even a memory allocator, which is defined in the alloc crate. Besides the allocator itself, Box, Rc, and Arc are the only other interesting things that come from alloc. Vec, String and most other collections are in the collections crate.

These support crates make up 'The Standard Façade' ††, by which std reexports these lower-level dependencies as if they were part of std.

Note here that test and log are only linked to when #[cfg(test)]. These crates would otherwise be unused and just introduce bloat into normal builds.

// Make std testable by not duplicating lang items. See #2912
#[cfg(test)] extern crate std as realstd;
#[cfg(test)] pub use realstd::marker;
#[cfg(test)] pub use realstd::ops;
#[cfg(test)] pub use realstd::cmp;
#[cfg(test)] pub use realstd::boxed;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

OK, remember when I said 'the standard library cannot link to the standard library'?

That's not always true: when testing std (that is, compiling with --test), rustc implicitly links to both the test crate (which is unstable and cannot be directly accessed) and std. Because std contains some privileged definitions (lang items, which can only be defined once across all crates), this implicit linking to std causes everything in std to be defined twice when compiling std with --test. Effectively, when testing std, std is not std. This code then, replaces the modules containing lang items with those from 'the real std', only when testing. Below you'll see corresponding #[cfg(not(test))] rules eliding the real definitions.

The core library doesn't bother with these acrobatics and has its own test crate.

This comment has been minimized.

Copy link
@Gankra

Gankra Jul 29, 2015

libcollections also has a dedicated test crate because of similar lang-item shenanigans. In particular libcollections is the home of many primitive impls like str and slice because this is the point where Vec and String are defined for them to refer to.



// NB: These reexports are in the order they should be listed in rustdoc

pub use core::any;
pub use core::cell;
pub use core::clone;
#[cfg(not(test))] pub use core::cmp;
pub use core::convert;
pub use core::default;
pub use core::hash;
pub use core::intrinsics;
pub use core::iter;
#[cfg(not(test))] pub use core::marker;
pub use core::mem;
#[cfg(not(test))] pub use core::ops;
pub use core::ptr;
pub use core::raw;
pub use core::simd;
pub use core::result;
pub use core::option;
pub mod error;

#[cfg(not(test))] pub use alloc::boxed;
pub use alloc::rc;

pub use core_collections::borrow;
pub use core_collections::fmt;
pub use core_collections::slice;
pub use core_collections::str;
pub use core_collections::string;
#[stable(feature = "rust1", since = "1.0.0")]
pub use core_collections::vec;

pub use rustc_unicode::char;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

†† This is most of 'The Standard Façade'. Much of the standard library is defined in other crates and reexported.

/* Exported macros */

#[macro_use]
mod macros;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

By convention, crates that define and export macros often have a macros module tagged with #[macro_use], which makes all the macros in that module available to the entire crate. stds macros module contains the definitions of many of the macros you are familiar with, though some of the fundamental macros are magic, built into the compiler.


mod rtdeps;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

This module does nothing but specify linker arguments to make the standard library compile on various platforms.


/* The Prelude. */

pub mod prelude;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

The Rust Prelude. This module defines a small set of common definitions, mostly traits, that are implicitly imported into all Rust modules.



/* Primitive types */

// NB: slice and str are primitive types too, but their module docs + primitive doc pages
// are inlined from the public re-exports of core_collections::{slice, str} above.

#[path = "num/float_macros.rs"]
#[macro_use]
mod float_macros;

#[path = "num/int_macros.rs"]
#[macro_use]
mod int_macros;

#[path = "num/uint_macros.rs"]
#[macro_use]
mod uint_macros;

#[path = "num/isize.rs"] pub mod isize;
#[path = "num/i8.rs"] pub mod i8;
#[path = "num/i16.rs"] pub mod i16;
#[path = "num/i32.rs"] pub mod i32;
#[path = "num/i64.rs"] pub mod i64;

#[path = "num/usize.rs"] pub mod usize;
#[path = "num/u8.rs"] pub mod u8;
#[path = "num/u16.rs"] pub mod u16;
#[path = "num/u32.rs"] pub mod u32;
#[path = "num/u64.rs"] pub mod u64;

#[path = "num/f32.rs"] pub mod f32;
#[path = "num/f64.rs"] pub mod f64;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

Definitions relating to numeric primitives are all nearly the same, so defined by macros as in the uint_macros module. There aren't many primitive-related definitions in std since they are mostly defined in core.

Note that primitive types are not keywords, a design decision mostly driven by the desire to define the above modules with names that would otherwise conflict with the primitives'.

These definitions additionally show the #[path] attribute, which names the file from which parser loads modules.


pub mod ascii;

pub mod thunk;

/* Common traits */

pub mod num;

/* Runtime and platform support */

#[macro_use]
pub mod thread;

pub mod collections;
pub mod dynamic_lib;
pub mod env;
pub mod ffi;
pub mod fs;
pub mod io;
pub mod net;
pub mod os;
pub mod path;
pub mod process;
pub mod sync;
pub mod time;

#[macro_use]
#[path = "sys/common/mod.rs"] mod sys_common;

#[cfg(unix)]
#[path = "sys/unix/mod.rs"] mod sys;
#[cfg(windows)]
#[path = "sys/windows/mod.rs"] mod sys;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

All these public mods contain the bulk of the std crate. Mostly what std adds atop core, alloc and collections is the cross-platform gunk necessary for abstracting away the operating environment, so you see here a lot of 'systemsy' subjects: threads, clocks, FFI, I/O, processes. The implementation details of a lot of this stuff can be pretty hairy, particularly when abstracting the differences between Unix and Windows. The sys mod here is the internal abstraction layer.


pub mod rt;
mod panicking;
mod rand;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

Some runtime details. The rt module used to be 'the runtime' back when Rust had one. Now it's a sad backwater. It's public (but unstable) for historical reasons and will likely not last much longer.


// Some external utilities of the standard library rely on randomness (aka
// rustc_back::TempDir and tests) and need a way to get at the OS rng we've got
// here. This module is not at all intended for stabilization as-is, however,
// but it may be stabilized long-term. As a result we're exposing a hidden,
// unstable module so we can get our build working.
#[doc(hidden)]
#[unstable(feature = "rand")]
pub mod __rand {
pub use rand::{thread_rng, ThreadRng, Rng};
}

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

Now we're getting down toward the bottom of lib.rs where traditionally the hacks start to get heaviest.

Here is a module that shouldn't exist. Shortly before Rust 1.0 the rand crate was removed from the standard distribution. Because various parts of the compiler still need random numbers, this little module was stealthily added to keep them available without advertising them. You can tell this module is unloved because it is thrice-hidden: #[doc(hidden)] tells rustdoc to ignore it; #[unstable] makes it unusable on the stable release channel, and the double-underscore prefix makes it triply-unappealing.

Although Rust has several mechanisms to enforce access control and visibility, in some rare cases we do the classic __ hack to indicate 'this is super private'. If you see the __ prefix in Rust it should fill you with dread - this is not a module you should touch!


// Modules that exist purely to document + host impl docs for primitive types

mod array;
mod bool;
mod unit;
mod tuple;

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

The Rust language and the Rust standard library have an intimate relationship. Some things you might expect to be defined by the language are defined by the library, and that includes all methods on primitive types. Because primitive types are not defined in the library though, documenting their methods is awkward. So rustdoc allows special-case documentation specifically for the primitive types with the #[doc(primitive)] attribute. That's what these teensy modules do.


// A curious inner-module that's not exported that contains the binding
// 'std' so that macro-expanded references to std::error and such
// can be resolved within libstd.
#[doc(hidden)]
mod std {
pub use sync; // used for select!()
pub use error; // used for try!()
pub use fmt; // used for any formatting strings
pub use option; // used for thread_local!{}
pub use rt; // used for panic!()
pub use vec; // used for vec![]
pub use cell; // used for tls!
pub use thread; // used for thread_local!
pub use marker; // used for tls!

// The test runner calls ::std::env::args() but really wants realstd
#[cfg(test)] pub use realstd::env as env;
// The test runner requires std::slice::Vector, so re-export std::slice just for it.
//
// It is also used in vec![]
pub use slice;

pub use boxed; // used for vec![]
}

This comment has been minimized.

Copy link
@brson

brson Jul 28, 2015

Author Owner

Finally, the famous 'curious inner module'. This is the wart de resistance, the crown jewel of Rust warts.

Rust's macro system is not perfect, and by that I mostly mean Rust macros are not hygienic. Sure, they are sometimes hygienic, but there are gaps. A number of standard macros expect the std module to exist in the crate root. Normally, nobody ever notices this since most Rust code does have the std module available. However, inside std is a different story. When invoking one of these macros inside std, without this curious inner module, many paths would not resolve.

This comment has been minimized.

Copy link
@pnkfelix

pnkfelix Jul 30, 2015

"verrue", monsieur!

0 comments on commit e50c2b1

Please sign in to comment.