Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster sorting (run formation) #119

Open
Mortal opened this issue Oct 24, 2013 · 2 comments
Open

Faster sorting (run formation) #119

Mortal opened this issue Oct 24, 2013 · 2 comments
Assignees

Comments

@Mortal
Copy link
Collaborator

Mortal commented Oct 24, 2013

In the merge sorter, we need to overlap internal sorting (highly CPU bound) and writing the sorted runs (highly I/O bound).

We should split the available internal memory in two, using one half for accumulating a sorted run and the other half as an I/O buffer for the sorted elements. Sorting and I/O should be overlapped explicitly (e.g. with two separate threads) to run both in parallel.

@svendcs
Copy link
Collaborator

svendcs commented Apr 24, 2014

An exploratory implementation of the merge sorter has been implemented in the parallel-merge-phase branch. A progress document has been created for this issue in order to keep track of test results.
https://gist.github.com/svendcsvendsen/11253182

@svendcs svendcs self-assigned this May 13, 2014
@SSoelvsten
Copy link
Contributor

This branch by Svend seems to have been removed and the gist is also dead. Is this still a desired feature or did it turn out not to improve performance as much as one hoped?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants