New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ep 9: Some ideas #45

Merged
merged 6 commits into from Sep 3, 2018

Conversation

Projects
None yet
3 participants
@killercup
Contributor

killercup commented Sep 2, 2018

Hey, after watching episode 9 I wanted to play around with another way of doing this: Instead of using a Mutex, we can also create a new hash map each time and combine them later on in a reduce. And Rayon has a pretty nice way of doing that.

I have not done any benchmarks whatsoever. See commit messages for more details.

killercup added some commits Sep 2, 2018

Use structopt
This is a CLI app. It deserves proper argument parsing and stuff. This
has the side effect of collecting the files into a Vec but if you pass
so many files that this matters you are doing some weird stuff
Actually, file names are PathBufs not Strings
Super tiny thing but why not support people's non UTF8 file names?
Add im: WIP -- I wanted to capture the error
This is very unhelpful:

    error[E0387]: cannot borrow data mutably in a captured outer variable in an `Fn` closure
      --> src/main.rs:30:57
      |
    30 |         .for_each(|filename| tally_words(filename, &mut words).unwrap());
      |                                                         ^^^^^
      |
    help: consider changing this closure to take self by mutable reference
      --> src/main.rs:30:19
      |
    30 |         .for_each(|filename| tally_words(filename, &mut words).unwrap());
      |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Use Rayon's `reduce` and im's `union_with`
This creates a new HashMap in each `tally_words` call. We can thus omit
using a Mutex. Instead of doing `for_each` with side effects, we use
`reduce` (Rayon's parallel version of it) to build the final map of
words.

The reason I used the im crate is because it has a very nice
`union_with` method on its HashMap; you can also replace `im::HashMap`
with `im::OrdMap` if you want to get sorted output directly.

I have no idea if this is faster, but it does show a different way of
using concurrency.

@killercup killercup force-pushed the killercup:9-some-ideas-from-pascal branch from 74b45b9 to 152163e Sep 2, 2018

@mre

This comment has been minimized.

Collaborator

mre commented Sep 3, 2018

As always, top notch work by @killercup.
Could you also add your improvement to the errata section similar to this one?
Note to self: I should probably rename this section to "Improvements and errata".

@killercup

This comment has been minimized.

Contributor

killercup commented Sep 3, 2018

Thanks! Added.

@mre mre merged commit 9ed335e into hello-rust:master Sep 3, 2018

@mre

This comment has been minimized.

Collaborator

mre commented Sep 3, 2018

👄 beautiful

@JuxhinDB

This comment has been minimized.

JuxhinDB commented Sep 4, 2018

Very interesting approach, well done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment