Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

a way to parallelize with less contention #352

Closed
wants to merge 2 commits into from
Closed

a way to parallelize with less contention #352

wants to merge 2 commits into from

Conversation

Byron
Copy link
Owner

@Byron Byron commented Mar 12, 2022

No description provided.

@Byron
more flexible in_parallel closures due to FnMut (#301)
18d4fb4 
Previously they were Sync so FnMut wasn't so useful.
Now they are thread-local, and thus should be mutable
for max flexibility.
@Byron
A sketch for work-stealing parallelization that has mostly uncontende…
79d5f64 
…d locks (#301)
@Byron Byron mentioned this pull request Mar 12, 2022
Open
44 tasks
@Byron Byron closed this Mar 14, 2022
@Byron Byron deleted the shared-parallel branch March 14, 2022 14:37
@Byron
Copy link
Owner Author

Byron commented Mar 14, 2022

Failure - tried shared-memory with chunks and work-stealing, chunks without work-stealing, and individual items without work-stealing.
It was always slower than channels, fastest was chunks without work-stealing, but still about 10% slower for some reason. Chunk-size is typically 1000.
I think it's mainly the file system that is sensitive to the order in which files are written and what not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@Byron