Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial review of parking_lot submodule #1

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ rust:
- beta
- nightly

# REVIEW: needs to be tested on OSX

before_script:
- |
pip install 'travis-cargo<0.2' --user &&
Expand Down Expand Up @@ -39,6 +41,13 @@ script:
- cargo run --release --bin rwlock -- 1 1 1 0 1 2
- cd ..

# REVIEW: needs a test that parking_lot still builds when structured like
# libstd (aka including modules manually). This is easy to break by accident and
# CI should help that import paths and such are always organized correctly.

# REVIEW: needs at least one builder for all of `thread_parker` implementations
# (at least building it, executing it would be best), missing ones are wasm/OSX
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I add macOS testing here: Amanieu#147

But I'm not sure how we could test wasm. It requires the specially built libstd with atomic support, but we can't easily get that on travis, can we?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's not really a great way to test it via execution since engines haven't fully caught up yet all over the place, but I'd recommend using xargo plus RUSTFLAGS to ensure that the support here at least builds and produces a binary.


env:
global:
- TRAVIS_CARGO_NIGHTLY_FEATURE=nightly
Expand Down
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ lazy_static = "1.0"
bincode = {version = "1.1.3"}

[build-dependencies]
# REVIEW: would recommend `autocfg` instead of `rustc_version`
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
rustc_version = "0.2"

[features]
Expand Down
60 changes: 60 additions & 0 deletions NOTES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Notes from review

* WebKit's documentation and intro on parking lot and its paradigm is quite good
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved

* `cargo test` fails with compilation errors on Linux
* had an ancient lock file on my system and `cargo update` was needed

* Various bits and pieces of the library are centered around compatibility with
various versions of Rust. This naturally isn't needed when integrating into
libstd, but it can often reduce the readability of code and/or make it more
difficult to follow. Do we want to have a clear version policy for parking_lot
on crates.io which is geared towards simplifying the code?

* Overall quite worried about atomic orderings. Everything should be `SeqCst`
until proven otherwise, and even then needs quite a lot of proof (IMO) to not
use `SeqCst`. Being fast-and-loose with orderings seems like we're trying to
be too clever by half.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree. In this case the orderings are very clear: Acquire when acquiring a lock and Release when releasing a lock. All other operations have no ordering requirements, and can use Relaxed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry but this isn't gonna fly with me at least. Atomic orderings are notoriously hard to get right, and I think it's just wrong that "hey it's a lock so acquire/release and everything that looks unrelated is relaxed".

I've personally got two requirements for atomic orderings:

  • Either everything is SeqCst
  • Or there's documentation lengthily explaining why SeqCst isn't used. This involves concrete benchmarks showing the performance different, pointers to other code in the world which is also thoroughly vetted and using more relaxed atomics, etc.

There's basically no way that this can be maintained to the level of quality expected from the standard library if our attitude is "no documentation necessary, and if it looks like a lock acquire/release otherwise relaxed everything".

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that atomic algorithms need to be carefully documented to argue for their correctness. With Relaxed, I agree that it should explain why there are no ordering requirements, and why it is okay not to synchronize this access.

However, I disagree that using SeqCst makes the argument any simpler. SeqCst works fine as a "I don't want to think about this now so let's use the least-likely-to-be-incorrect option". So, it makes things simpler if you don't have a careful argument. But reasoning based on SeqCst basically requires reasoning by exhaustively looking at all interleavings, and that is certainly not maintainable. Hence I find any remaining SeqCst to be a strong sign that something really strange is going on that needs sequential consistency (which is extremely rare).

After all, acquire/release operations are literally called that way because they are exactly what one needs to acquire/release a lock.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the absolute bare minimum anything not SeqCst needs to be documented in my opinion. It's a good point though that I personally assume SeqCst isn't "I thought about this and need it" but rather "I didn't think about this so this is the default I chose". I think it's safe to assume that in the sense that if an ordering isn't documented and it's SeqCst we can assume it's just "didn't think about this will relax later if necessary".

I am not personally comfortable enough with atomic orderings to sign off on this if the non-SeqCst orderings are documented and remain. I'd want review from someone who is comfortable reviewing the orderings.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can help check the orderings. And it seems we are in agreement that more docs are needed either way, that should hopefully make this easier. The docs should explain either why no synchronization is needed, or which kind of synchronization is happening. So e.g. the acquire load on a lock-acquire method should say that this synchronizes-with the release store of the lock-release operation of whoever previously held the lock, thus establishing the necessary synchronization between the previous thread that held the lock and the new thread that now got it.

Copy link

@KamilaBorowska KamilaBorowska Oct 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree about SeqCst by default. If I see SeqCst my immediate reaction is "the programmer likely has no idea about atomics, and this code is full of bugs". Atomics, even when "sequentially consistent" guarantee much less than most programmęrs would expect, and actually needing "sequential consistency" guarantee is very rare to the point where I've never seen it being necessary (albeit there may be usecases where it may need to be used - if you have one, there better be a comment explaining why you couldn't use Acquire/Release).

Additionally, using SeqCst everywhere makes code harder to read as the necessary guarantees aren't visible from the source code.

std uses SeqCst pretty much everywhere in its mpsc implementation. The implementation is pretty much broken as evidenced by issues like rust-lang/rust#39364.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I agree with the sentiment that release-acquire should be the default and anything else require extra comments -- but that is not a consensus in the Rust teams, see rust-lang/rfcs#2503 for some discussion.


* The `ThreadParker` API seems pretty subtle. For example the `UnparkHandle`
type can outlive the `ThreadParker` itself but that's ok. This seems currently
undocumented?

* Can `parking_lot::Mutex` be used in as many places as the current OS-based
mutex? I think so but it's worth considering. For example you may be able to
use `std::sync::Mutex` today to protect a global allocator, but with parking
lot you won't be able to. Similarly about thread local internals with libstd,
but I think it's just internal stuff to libstd.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You make a good point about the dependency on the global allocator. park may need to grow the hash table if there are many parked threads. I'm not really sure if there is a good way to resolve this issue, maybe we could force parking_lot to use the system allocator?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's really a great answer to this either, I think it's just something we'll have to acknowledge as the libs team and be sure to indicate in release notes and/or changelogs. If it becomes a big issue we can try to use the system allocator or some other mechanism, but in general we don't have a great fix for this.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Redox, for example, I believe that the Rust global allocator is the system allocator! So parking_lot won’t work in that case. Furthermore, some code (such as OS kernels, filesystem drivers, etc) must recover from allocation failure. That might be much more difficult if taking a lock requires an allocation.

Would it be possible to use an intrusive linked list instead? This assumes that a mutex cannot be moved while it is locked, which seems like a reasonable assumption ― it will be borrowed by its guard.


* Where possible it'd be best to use `alloc`-the-crate now that it's stable
instead of using collections through `std`. Ideally only thread locals are
used from std (and other synchronization things if necessary).

* There is a **massive** number of `#[inline]` annotations everywhere. I suspect
almost all of them are not necessary and can probably be removed. A
particularly egregious offender, for example, is `parking_lot_core::park`.
It's already generic and therefore inlined across crates, and it's a pretty
massive function to hint even more strongly that it should be inlined.

* In general there's very little documentation beyond the bare bones required by
the `missing_docs` lint. Even that documentation isn't really anything beyond
what's already given in the function name and signature. Could benefit from
quite a few more comments about implementation strategies, protocols
implemented, meanings of constants, etc. More high level docs would also be
quite welcome explaining what each module is doing and how it relates to the
WTF parking lot.

* It seems that `parking_lot_core` is fundamentally unsafe in a way that can't
really be library-ified? Given that `parking_lot`'s functions are `unsafe` due
to ensuring that the tokens aren't reused, it basically requires that
literally everything using the interface agrees on what tokens to allocate?
Using pointers works, but it seems like it's worthwhile documenting that it's
basically very difficult to use `parking_lot_core` in a safe fashion unless
you're doing the exact same thing as everyone else.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mutex uses tokens to pass information from the unpaker to the thread being unparked. One token in particular (TOKEN_HANDOFF) indicates that the ownership of the mutex has been directly transferred to the unparked thread, without unlocking it. If a TOKEN_HANDOFF is sent out of nowhere, it could result in a thread assuming that it owns the mutex when it really doesn't. This is the main reason why the parking_lot_core methods are unsafe.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having that documented in the function as a concrete case of unsafety would be useful!


* There's virtually no tests from what I can tell other than weak smoke tests
from the standard library. The `parking_lot_core` crate, for example, seems to
have virtually no tests of its APIs.

* `Condvar` still can only be used with one mutex at a time (originally thought
this was going to be fixed)
alexcrichton marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 3 additions & 0 deletions appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,6 @@ test_script:
- travis-cargo test
- travis-cargo --only nightly test -- --features=deadlock_detection
- travis-cargo doc

# REVIEW: travis tests for linux seem more extensive, could configuration be
# shared to test more on windows?
15 changes: 15 additions & 0 deletions core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,13 @@
feature(thread_local, checked_duration_since)
)]

// REVIEW: for maintainability this crate should unconditionally be
// `#![no_std]`, and then the `libstd` module is exclusively used for importing
// items from `std`. Items from `core` can be imported as usual.

// REVIEW: the "i-am-libstd" feature would probably best be something like
// `#[cfg(rustbuild)]` or something like that printed out by the build script in
// rust-lang/rust.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Anything that makes future = "i-am-libstd" a bit shorter would be really good.

#[cfg(not(feature = "i-am-libstd"))]
use cfg_if::cfg_if;

Expand All @@ -72,6 +79,11 @@ mod libstd {
}

cfg_if! {
// REVIEW: following the logic here of when and where `i-am-libstd` is
// included is somewhat confusing, it would probably be best to simply have
// libstd's own build script agree with parking_lot's own build script and
// print out the same `cfg` annotations that the `parking_lot` crate does on
// crates.io.
if #[cfg(all(any(has_sized_atomics, feature = "i-am-libstd"), target_os = "linux"))] {
#[path = "thread_parker/linux.rs"]
mod thread_parker;
Expand Down Expand Up @@ -100,6 +112,9 @@ cfg_if! {
} else if #[cfg(all(feature = "i-am-libstd", not(target_arch = "wasm32")))] {
compile_error!("Not allowed to fall back to generic spin lock based thread parker");
} else {
// REVIEW: this implementation isn't really appropriate for most use
// cases other than "at least it compiles", and for libstd we'll want a
// compile error if this is ever used.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the above compile_error! does precisely wat you want, except for allowing wasm to use the generic impl.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. When used inside std the above compile_error should make sure we never fall back to the generic one.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would work, yes, but wasm should continue to not use the generic impl since it should panic if a request to wait is ever received because it won't otherwise wake up.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So wasm should do what? Currently the wasm ThreadParker requires the atomics feature that is not available by default. Should we add a non-atomics wasm ThreadParker? How would we implement that? My understanding was that the atomics feature was needed in order to do any parking. Or do you mean adding a wasm thread parker that just panics? In other words, making the library compile, but not usable?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well this should compile on CI with both atomics and without atomics. The nonatomic version should just panic if it ever requests to block, and the atomics version should do what it does today. It's difficult to run this so it's fine to just check it compiles. There's example CI configuration in wasm-bindgen of how to set up an atomics target

#[path = "thread_parker/generic.rs"]
mod thread_parker;
}
Expand Down
Loading