Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

allow explicit control over single-threaded vs multi-threaded #25

Closed
oconnor663 opened this issue Jan 15, 2020 · 6 comments
Closed

allow explicit control over single-threaded vs multi-threaded #25

oconnor663 opened this issue Jan 15, 2020 · 6 comments

Comments

@oconnor663
Copy link
Member

Right now there is only the top-level rayon feature, so multi-threading is all-or-nothing. It would be better to allow each caller to choose, and perhaps to provide other executors besides Rayon. We could keep the default governed by the rayon feature, or we could make the default always single-threaded, not sure yet.

This might allow us to support a flag in b3sum that controls multithreading at runtime, for example.

@pcwalton mentioned one approach here: https://twitter.com/pcwalton/status/1215349922267987970
@oconnor663
Copy link
Member Author

Copying a question that just tweeted. Not sure what to do about the standalone functions:

Do you know if there's a practical way to do this for a standalone function, a la hash()? I just ran into the limitation that functions can't have default type parameters.

@oconnor663 oconnor663 mentioned this issue Feb 5, 2020
Closed
@oconnor663
Copy link
Member Author

rust-lang/rust#36887

@oconnor663
Copy link
Member Author

oconnor663 commented Feb 6, 2020 •
edited
Loading

Rather than create duplicated versions of all the top level functions to work around this, I'm going to add a single update_with_join method to Hasher. We'll provide a SerialJoin (which most of the API uses internally by default) and a RayonJoin (gated by the rayon feature), and using the latter with update_with_join will get you multi-threading. This will make the top level functions always single-threaded, which is probably for the best.

oconnor663 added a commit that referenced this issue Feb 6, 2020
@oconnor663
Hasher::update_with_join
fc219f4 
This is a new interface that allows the caller to provide a
multi-threading implementation. It's defined in terms of a new `Join`
trait, for which we provide two implementations, `SerialJoin` and
`RayonJoin`. This lets the caller control when multi-threading is used,
rather than the previous all-or-nothing design of the "rayon" feature.

Although existing callers should keep working, this is a compatibility
break, because callers who were relying on automatic multi-threading
before will now be single-threaded. Thus the next release of this crate
will need to be version 0.2.

See #25 and
#54.
@oconnor663
Copy link
Member Author

I just pushed fc219f4, implementing the update_with_join idea described above. I'm going to close this for now, but we can reopen it as we get more experience with the API discover any problems.

@oconnor663
Copy link
Member Author

I just filed an issue with rust-lang/rust over the test failures above. rust-lang/rust#68905

@oconnor663
Copy link
Member Author

update_with_join just shipped as part of version 0.2.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@oconnor663