std: Rewrite the `sync` module #19274

alexcrichton · 2014-11-24T19:49:54Z

This commit is a reimplementation of std::sync to be based on the
system-provided primitives wherever possible. The previous implementation was
fundamentally built on top of channels, and as part of the runtime reform it has
become clear that this is not the level of abstraction that the standard level
should be providing. This rewrite aims to provide as thin of a shim as possible
on top of the system primitives in order to make them safe.

The overall interface of the std::sync module has in general not changed, but
there are a few important distinctions, highlighted below:

The condition variable type, Condvar, has been separated out of a Mutex.
A condition variable is now an entirely separate type. This separation
benefits users who only use one mutex, and provides a clearer distinction of
who's responsible for managing condition variables (the application).
All of Condvar, Mutex, and RWLock are now directly built on top of
system primitives rather than using a custom implementation. The Once,
Barrier, and Semaphore types are still built upon these abstractions of
the system primitives.
The Condvar, Mutex, and RWLock types all have a new static type and
constant initializer corresponding to them. These are provided primarily for C
FFI interoperation, but are often useful to otherwise simply have a global
lock. The types, however, will leak memory unless destroy() is called on
them, which is clearly documented.
The fundamental architecture of this design is to provide two separate layers.
The first layer is that exposed by sys_common which is a cross-platform
bare-metal abstraction of the system synchronization primitives. No attempt is
made at making this layer safe, and it is quite unsafe to use! It is currently
not exported as part of the API of the standard library, but the stabilization
of the sys module will ensure that these will be exposed in time. The
purpose of this layer is to provide the core cross-platform abstractions if
necessary to implementors.

The second layer is the layer provided by std::sync which is intended to be
the thinnest possible layer on top of sys_common which is entirely safe to
use. There are a few concerns which need to be addressed when making these
system primitives safe:
- Once used, the OS primitives can never be moved. This means that they
  essentially need to have a stable address. The static primitives use
  &'static self to enforce this, and the non-static primitives all use a
  Box to provide this guarantee.
- Poisoning is leveraged to ensure that invalid data is not accessible from
  other tasks after one has panicked.
In addition to these overall blanket safety limitations, each primitive has a
few restrictions of its own:
- Mutexes and rwlocks can only be unlocked from the same thread that they
  were locked by. This is achieved through RAII lock guards which cannot be
  sent across threads.
- Mutexes and rwlocks can only be unlocked if they were previously locked.
  This is achieved by not exposing an unlocking method.
- A condition variable can only be waited on with a locked mutex. This is
  achieved by requiring a MutexGuard in the wait() method.
- A condition variable cannot be used concurrently with more than one mutex.
  This is guaranteed by dynamically binding a condition variable to
  precisely one mutex for its entire lifecycle. This restriction may be able
  to be relaxed in the future (a mutex is unbound when no threads are
  waiting on the condvar), but for now it is sufficient to guarantee safety.
Condvars support timeouts for their blocking operations. The
implementation for these operations is provided by the system.

Due to the modification of the Condvar API, removal of the std::sync::mutex
API, and reimplementation, this is a breaking change. Most code should be fairly
easy to port using the examples in the documentation of these primitives.

[breaking-change]

Closes #17094
Closes #18003

rust-highfive · 2014-11-24T19:50:02Z

Warning

These commits modify unsafe code. Please review it carefully!

alexcrichton · 2014-11-24T19:51:54Z

Note that this is duplicating a lot of implementation details of librustrt and libsync as of this red hot minute. Once these two libs are merged into std I'll delete all the duplication and make sure that we've only got one copy of the definitions. This PR is meant to reflect the final state of the stdlib for std::sync and its implementation details.

sfackler · 2014-11-24T20:49:17Z

Condvars and RWLocks now support timeouts for their blocking operations. The
implementation for these operations is provided by the system.

Woo!

sfackler · 2014-11-24T20:56:41Z

src/libstd/sync/condvar.rs

+    /// specified duration.
+    ///
+    /// The semantics of this function are equivalent to `wait()` except that
+    /// the thread will be blocked for no longer than `dur`. If the wait timed


Might want to qualify this a bit - it's not really possible in general to unblock the thread after exactly the right duration.

sfackler · 2014-11-24T21:04:45Z

I didn't see any timeout stuff in the RWLock impl, btw

alexcrichton · 2014-11-24T21:55:41Z

Ah yes sorry, I mis-remembered what was added to rwlocks. RWLocks now support try_read and try_write, not timeouts just yet. I'd like to provide a unix extension trait to provide these functions, but Windows at least does not implement locking with a timeout.

alexcrichton · 2014-11-25T17:24:02Z

One thing I also just remembered, this removes the ability to use a condition variable with a RWLock for now. We can build something such as C++'s condition_variable_any in the future, but this PR is just aimed at exposing the system primitives for now.

alexcrichton · 2014-11-25T22:45:56Z

Now that #19255 has merged I've rebased on top and removed all implementations of primitives based on channels. At the same time I have also remove all concurrent queues from the public interface as I don't think we're going to be able to stabilize them before 1.0. Two are kept as implementation details of channels, but the chase-lev deque and mpmc queue have been entirely removed.

aturon · 2014-11-25T23:32:07Z

src/libstd/sync/barrier.rs

+            // We need a while loop to guard against spurious wakeups.
+            // http://en.wikipedia.org/wiki/Spurious_wakeup
+            while local_gen == lock.generation_id &&
+                  lock.count < self.num_threads {


Why is this second comparison needed?

This is mostly just copy-pasted from the existing implementation, but I believe the first comparison is to determine whether this was the final thread (the notifier) or a thread which needs to wait. The second comparison is redundant the first iteration of the loop, but all other iterations it's intended to help with spurious wakeups.

aturon · 2014-11-25T23:36:27Z

src/libstd/sync/barrier.rs

+        }
+    }
+
+    /// Block the current thread until all tasks has rendezvoused here.


"tasks have"

aturon · 2014-11-26T16:46:46Z

src/libstd/sync/semaphore.rs

+
+    #[test]
+    fn test_sem_runtime_friendly_blocking() {
+        // Force the runtime to schedule two threads on the same sched_loop.


I think this comment is out of date :P

aturon · 2014-11-26T16:49:40Z

OK, I've finished reading this over and it looks great! (I didn't scrutinize the OS bindings.)

I think there are a lot of questions going forward about the organization of this module, static variants, and additional system bindings, but this looks like a great step forward. I'm happy for it to land for now, and then we can talk through exactly what we want to stabilize a bit later.

r=me after nits and a rebase.

sfackler · 2014-11-27T06:42:28Z

Needs a rebase

nodakai · 2014-11-29T11:37:28Z

src/libstd/sync/mutex.rs

+            assert_eq!(*lock2, 1);
+            tx.send(());
+        });
+        rx.recv();


This rx.recv() is nice because the old version couldn't catch a possible assertion failure in the spawned proc. Can we perhaps define a macro to standardize this pattern and to make sure there's no longer any instances of meaningless assert_eq!?

This may be somewhat difficult to refactor out into a standard pattern, but for now I'm going to focus on landing this to unblock some more runtime removal.

thestinger · 2014-12-03T14:42:51Z

src/libstd/sys/unix/condvar.rs

+
+        // First, figure out what time it currently is
+        let mut tv = libc::timeval { tv_sec: 0, tv_usec: 0 };
+        let r = ffi::gettimeofday(&mut tv, 0 as *mut _);


This isn't correct. You need to use the monotonic clock support. The system time can and will often change.

http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_getclock.html

@thestinger Please remember that Linux CLOCK_MONOTONIC is susceptible to machine hibernation during which TSC doesn't count up. We would have to wait for futex(2) to support CLOCK_BOOTTIME for truly correct timeout operation. So IMHO it's acceptable to live with this code meantime (unless we don't forget that we're repeating the same bugs found in Java bug tracker for long...)

I don't personally want to debate these details on this PR as this is blocking more runtime removal and reform. I would like to prioritize pushing through this rewrite as much as possible so the wait_timeout functions are now non-public with comments pointing at this PR to review the implementation before making them public.

thestinger · 2014-12-03T14:50:15Z

It's painful (but possible) to do timeout support properly on Windows. It should be left out if it's not going to be done correctly because having incorrect implementations is worse than not having the feature.

This commit is a reimplementation of `std::sync` to be based on the system-provided primitives wherever possible. The previous implementation was fundamentally built on top of channels, and as part of the runtime reform it has become clear that this is not the level of abstraction that the standard level should be providing. This rewrite aims to provide as thin of a shim as possible on top of the system primitives in order to make them safe. The overall interface of the `std::sync` module has in general not changed, but there are a few important distinctions, highlighted below: * The condition variable type, `Condvar`, has been separated out of a `Mutex`. A condition variable is now an entirely separate type. This separation benefits users who only use one mutex, and provides a clearer distinction of who's responsible for managing condition variables (the application). * All of `Condvar`, `Mutex`, and `RWLock` are now directly built on top of system primitives rather than using a custom implementation. The `Once`, `Barrier`, and `Semaphore` types are still built upon these abstractions of the system primitives. * The `Condvar`, `Mutex`, and `RWLock` types all have a new static type and constant initializer corresponding to them. These are provided primarily for C FFI interoperation, but are often useful to otherwise simply have a global lock. The types, however, will leak memory unless `destroy()` is called on them, which is clearly documented. * The `Condvar` implementation for an `RWLock` write lock has been removed. This may be added back in the future with a userspace implementation, but this commit is focused on exposing the system primitives first. * The fundamental architecture of this design is to provide two separate layers. The first layer is that exposed by `sys_common` which is a cross-platform bare-metal abstraction of the system synchronization primitives. No attempt is made at making this layer safe, and it is quite unsafe to use! It is currently not exported as part of the API of the standard library, but the stabilization of the `sys` module will ensure that these will be exposed in time. The purpose of this layer is to provide the core cross-platform abstractions if necessary to implementors. The second layer is the layer provided by `std::sync` which is intended to be the thinnest possible layer on top of `sys_common` which is entirely safe to use. There are a few concerns which need to be addressed when making these system primitives safe: * Once used, the OS primitives can never be **moved**. This means that they essentially need to have a stable address. The static primitives use `&'static self` to enforce this, and the non-static primitives all use a `Box` to provide this guarantee. * Poisoning is leveraged to ensure that invalid data is not accessible from other tasks after one has panicked. In addition to these overall blanket safety limitations, each primitive has a few restrictions of its own: * Mutexes and rwlocks can only be unlocked from the same thread that they were locked by. This is achieved through RAII lock guards which cannot be sent across threads. * Mutexes and rwlocks can only be unlocked if they were previously locked. This is achieved by not exposing an unlocking method. * A condition variable can only be waited on with a locked mutex. This is achieved by requiring a `MutexGuard` in the `wait()` method. * A condition variable cannot be used concurrently with more than one mutex. This is guaranteed by dynamically binding a condition variable to precisely one mutex for its entire lifecycle. This restriction may be able to be relaxed in the future (a mutex is unbound when no threads are waiting on the condvar), but for now it is sufficient to guarantee safety. * Condvars now support timeouts for their blocking operations. The implementation for these operations is provided by the system. Due to the modification of the `Condvar` API, removal of the `std::sync::mutex` API, and reimplementation, this is a breaking change. Most code should be fairly easy to port using the examples in the documentation of these primitives. [breaking-change] Closes rust-lang#17094 Closes rust-lang#18003

mpmc was removed from stdlib, so we just vendor it for now (rust-lang/rust#19274)

sfackler reviewed Nov 24, 2014
View reviewed changes

alexcrichton force-pushed the rewrite-sync branch 2 times, most recently from 48e3d43 to 5420ea8 Compare November 24, 2014 21:54

alexcrichton force-pushed the rewrite-sync branch from 5420ea8 to 45afc55 Compare November 25, 2014 17:22

alexcrichton force-pushed the rewrite-sync branch from 45afc55 to 86d7810 Compare November 25, 2014 22:44

aturon reviewed Nov 25, 2014
View reviewed changes

alexcrichton force-pushed the rewrite-sync branch from 86d7810 to 77f9bbd Compare November 25, 2014 23:36

aturon reviewed Nov 25, 2014
View reviewed changes

src/libstd/sync/barrier.rs

}

}

/// Block the current thread until all tasks has rendezvoused here.

Copy link

Member

aturon Nov 25, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"tasks have"

aturon reviewed Nov 26, 2014
View reviewed changes

alexcrichton force-pushed the rewrite-sync branch from ac68b58 to ddc82a5 Compare November 26, 2014 17:41

alexcrichton force-pushed the rewrite-sync branch from ddc82a5 to 2de3582 Compare November 27, 2014 06:50

alexcrichton force-pushed the rewrite-sync branch from 2de3582 to 7bbac39 Compare November 27, 2014 20:18

alexcrichton force-pushed the rewrite-sync branch from 7bbac39 to 25cc7c8 Compare November 27, 2014 21:48

alexcrichton force-pushed the rewrite-sync branch from 25cc7c8 to f0a4eef Compare November 28, 2014 00:03

emk mentioned this pull request Nov 28, 2014

sync::mutex::StaticMutex disappeared from public API, perhaps unintentionally? #19379

Closed

nodakai reviewed Nov 29, 2014
View reviewed changes

aturon mentioned this pull request Dec 2, 2014

update docs with removal of libgreen #19402

Closed

thestinger reviewed Dec 3, 2014
View reviewed changes

alexcrichton force-pushed the rewrite-sync branch from f0a4eef to 3c8f8cf Compare December 5, 2014 08:57

Fall out of the std::sync rewrite

c3adbd3

alexcrichton force-pushed the rewrite-sync branch from 3c8f8cf to c3adbd3 Compare December 5, 2014 17:13

alexcrichton merged commit c3adbd3 into rust-lang:master Dec 5, 2014

alexcrichton deleted the rewrite-sync branch December 5, 2014 23:50

elinorbgr mentioned this pull request Dec 6, 2014

mpmc_bounded_queue is not longer in Rust std public API tokio-rs/mio#62

Closed

rozaliev added a commit to rozaliev/mio that referenced this pull request Dec 8, 2014

add mpmc_bounded_queue

08b3ae6

mpmc was removed from stdlib, so we just vendor it for now (rust-lang/rust#19274)

aturon mentioned this pull request Dec 16, 2014

Stabilization metabug: 1.0-alpha #19260

Closed

alexcrichton mentioned this pull request Dec 29, 2014

mutexes should be separate from condition variables #10574

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

std: Rewrite the `sync` module #19274

std: Rewrite the `sync` module #19274

alexcrichton commented Nov 24, 2014

rust-highfive commented Nov 24, 2014

alexcrichton commented Nov 24, 2014

sfackler commented Nov 24, 2014

sfackler Nov 24, 2014

sfackler commented Nov 24, 2014

alexcrichton commented Nov 24, 2014

alexcrichton commented Nov 25, 2014

alexcrichton commented Nov 25, 2014

aturon Nov 25, 2014

alexcrichton Nov 26, 2014

aturon Nov 25, 2014

aturon Nov 26, 2014

aturon commented Nov 26, 2014

sfackler commented Nov 27, 2014

nodakai Nov 29, 2014

alexcrichton Dec 5, 2014

thestinger Dec 3, 2014

thestinger Dec 3, 2014

nodakai Dec 4, 2014

alexcrichton Dec 5, 2014

thestinger commented Dec 3, 2014

std: Rewrite the sync module #19274

std: Rewrite the sync module #19274

Conversation

alexcrichton commented Nov 24, 2014

rust-highfive commented Nov 24, 2014

alexcrichton commented Nov 24, 2014

sfackler commented Nov 24, 2014

Choose a reason for hiding this comment

sfackler commented Nov 24, 2014

alexcrichton commented Nov 24, 2014

alexcrichton commented Nov 25, 2014

alexcrichton commented Nov 25, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aturon commented Nov 26, 2014

sfackler commented Nov 27, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thestinger commented Dec 3, 2014

std: Rewrite the `sync` module #19274

std: Rewrite the `sync` module #19274