Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize SF3 loading #746

Merged
merged 8 commits into from Feb 6, 2021
Merged

Parallelize SF3 loading #746

merged 8 commits into from Feb 6, 2021

Conversation

derselbst
Copy link
Member

@derselbst derselbst commented Jan 19, 2021

Loading SF3 files require decoding individual samples. Decoding is CPU-expensive and could make use of multiple cores. This PR distributes the decoding across multiple threads by using OpenMP tasks. This resolves #710.

The problem here is that the SF3 file is (usually) accessed via a common FILE*. Multiple threads seeking and reading around in the file would cause a race condition. Opening the FILE* multiple times is not an option either, as this is implementation-defined behavior. And since most implementations do not document this case, it becomes undefined behavior.

Originally, Marcus and I thought that using a memory-mapped file would be the solution to this. When I started implementing this I realized that this would create a "parallel-world" to the existing sfloader-callback mechanism. Secondly, it cannot be used if a user has set up custom callbacks via fluid_sfloader_set_callbacks(). Thirdly, memory-mapping is not supported for every file. If mmapping fails, we would still need to fall back to FILE*.

In this PR, I've chosen to synchronize the calls to fread() instead:

lock(mutex);
callback->fseek(); // seek to sample-offset
callback->fread(); // read into libsndfile buffer
unlock(mutex);

The fseek before the fread is required to ensure the sample is being read from the correct position.

Loading three SF3 file on my machine now takes only 4.5 seconds, rather than 13 seconds single-threaded before.

@derselbst derselbst added this to the 2.2 milestone Jan 19, 2021
@sonarcloud
Copy link

sonarcloud bot commented Feb 6, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@derselbst derselbst merged commit 0d76403 into master Feb 6, 2021
@derselbst derselbst deleted the parallelize-sf3-loading branch February 6, 2021 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Parallel SF3 sample decoding
2 participants