Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use all available cores #21

Open
simonschmidt opened this issue Jan 8, 2014 · 0 comments
Open

Use all available cores #21

simonschmidt opened this issue Jan 8, 2014 · 0 comments

Comments

@simonschmidt
Copy link
Contributor

json2csv uses a lot of CPU, but only one core. json.Unmarshal seems to be taking up most of the time so I made a forked version that launches a whole bunch of them in parallel

A 207Mb json file with 241804 lines takes 15s to convert to csv with the single-threaded version, using all cores (2 real + 2 hyperthreading) it takes 7s. However using just one core on the multi-threaded version takes 17s It might be worth to keep the single-threaded version in the code and use that when running on one core.

parallel version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant