Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider bumping crossbeam-deque #586

Closed
kpcyrd opened this issue Jul 31, 2018 · 7 comments · Fixed by #656

Comments

@kpcyrd
Copy link

commented Jul 31, 2018

Hey!

I noticed rayon-core currently depends on an old version of crossbeam-deque, Cargo.toml states:

# This is deliberately not the latest version, because we want
# to support older rustc than crossbeam-deque 0.3+ does.
[dependencies.crossbeam-deque]
version = "0.2.0"

This would require us (the debian rust maintainers) to package an outdated version of crossbeam-deque and its outdated dependencies. Please reconsider bumping the dependency. This currently prevents cc from transitioning to debian testing.

This seems to be related to #480

@cuviper

This comment has been minimized.

Copy link
Member

commented Jul 31, 2018

Alternatively, you could just patch your package to use the later version. AFAIK the API is compatible, at least enough for Rayon's needs, just not the compiler support. We patch it in Fedora, and it works fine.

(And yes, I'm caught in opposing roles here -- holding conservative support in the crate while using the latest in Fedora...)

@kpcyrd

This comment has been minimized.

Copy link
Author

commented Sep 2, 2018

@cuviper do you have a link for for that patch? It seems rayon-core doesn't build with the latest crossbeam-deque without patching the source as well :/ Thanks!

@cuviper

This comment has been minimized.

Copy link
Member

commented Sep 2, 2018

You're right, we've only bumped it to 0.4 without source changes:
https://src.fedoraproject.org/rpms/rust-rayon-core/blob/master/f/rayon-core-1.4.1-fix-metadata.diff

I think the newer versions have shuffled around the worker/stealer types a bit. I can try to prepare a branch for this next week, if you don't get to it first... 😀

@kpcyrd

This comment has been minimized.

Copy link
Author

commented Sep 10, 2018

At the rayon-rs team: Can you please check which system is currently targeted with >= 1.13.0?

debian stable now ships rustc 1.24.0 and I can't find a distro that ships anything older (openbsd 6.3 ships 1.24.0 as well). This currently makes it difficult to transition cc to testing which is required for anything that links to C libs and/or uses nix.

Related to BurntSushi/ripgrep#10

@cuviper

This comment has been minimized.

Copy link
Member

commented Sep 10, 2018

Here's the port: master...cuviper:crossbeam-deque-0.6. I haven't done any benchmarking, but at least it builds and passes tests on rust 1.28.

@kpcyrd We don't have a specific target -- it's just a snapshot of what we were able to support as we were preparing for rayon 1.0. We don't really know where rayon is actually used! But since you cite Debian stable, note in their package status that they only have 1.24 for amd64, whereas arm64 and i386 are on 1.14.

The difference with ripgrep is that it is an application, whereas we are a library for others to use. Our decision forces a minimum on every crate depending on us, which is why I try to be as conservative as I can. We saw this problem ourselves when rand started requiring a higher version, then it broke rayon builds on the older versions we meant to support.

@lnicola

This comment has been minimized.

Copy link

commented Feb 7, 2019

But since you cite Debian stable, note in their package status that they only have 1.24 for amd64, whereas arm64 and i386 are on 1.14.

All Debian stable architectures are on 1.24 now.

One often overlooked disadvantage of depending on older versions is the build time and binary size increase that it brings. It also means missing bug fixes and optimizations in the dependencies.

I think that a project that's stuck on an almost two and a half years old compiler is not likely to update rayon too often, so they won't be inconvenienced by a new release.

@kornelski

This comment has been minimized.

Copy link
Contributor

commented Feb 15, 2019

Reading about all the improvements in crossbeam made me worried that Rayon is not benefiting from any of them.

I use Debian on my servers, but I don't use Debian's Rust. It doesn't help that Rayon supports it, because pretty much nothing else does.

Even crates that say they support older Rust, don't really, because they either break something accidentally (e.g. match ergonomics), or their dependencies drop support. The ecosystem is migrating to 2018 edition already.

I've also tried to use cargo's -Z minimal-versions, but that also terribly fails on almost every crate, e.g. by finding a way to include winapi 0.1 that requires pre-1.0 Rust.

The state of the ecosystem is that, for better or worse, any less-than-latest Rust version is not supported in general, and in practice absolutely unusable without an old Cargo.lock. Debian's Rust is just totally useless.

bors bot added a commit that referenced this issue May 8, 2019
Merge #656
656: Rust 1.26 and assorted cleanups r=nikomatsakis a=cuviper

In anticipation of [RFC 3](rayon-rs/rfcs#3), giving us a policy to soon raise our minimum to Rust 1.26, I started this branch to make the conversion. After the primary motivation of upgrading crossbeam, I also looked for ways to use newer language features.

Throughout all of this, the public API should be completely unaffected.

Closes #586 for newer crossbeam, even though we're not quite at the *newest*.

Co-authored-by: Josh Stone <cuviper@gmail.com>
bors bot added a commit that referenced this issue May 8, 2019
Merge #656
656: Rust 1.26 and assorted cleanups r=nikomatsakis a=cuviper

In anticipation of [RFC 3](rayon-rs/rfcs#3), giving us a policy to soon raise our minimum to Rust 1.26, I started this branch to make the conversion. After the primary motivation of upgrading crossbeam, I also looked for ways to use newer language features.

Throughout all of this, the public API should be completely unaffected.

Closes #586 for newer crossbeam, even though we're not quite at the *newest*.

Co-authored-by: Josh Stone <cuviper@gmail.com>

@bors bors bot closed this in #656 May 8, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.