Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

seekable_format/examples/parallel_compression.c is not parallel #3980

Open
EbiArnie opened this issue Mar 19, 2024 · 0 comments
Open

seekable_format/examples/parallel_compression.c is not parallel #3980

EbiArnie opened this issue Mar 19, 2024 · 0 comments

Comments

@EbiArnie
Copy link

Describe the bug
The parallel compression example does not seem to be working in parallel.
It runs on a single thread only, regardless of how many threads are requested.

To Reproduce

  1. Build the example parallel_compression
  2. Run:
    ./parallel_compression data.txt $((2*1024*1024)) 4
    data.txt can be any sufficiently large file so that compression takes a handful of seconds.
    In my case, this is ASCII data, 2.8 GB.
  3. While the compression is running, look at e.g. top
  4. Just a single thread is used. This is usually slower than the single threaded seekable_compression,

Expected behavior
Parallel execution should use multiple cores and be faster than single threaded.
The full zstd executable works as expected; given the -T option it is faster than without and saturates all cores.

Also, I'd love for --seekable to become a standard option for zstd, but that should probably be a separate issue.

Desktop:

  • OS: Linux
  • Version Ubuntu 22.04
  • Compiler: gcc 11.4
  • Flags: -O3
  • System: Quad-core Intel Core i7-1165G7
  • Build system Makefile
  • Built from git version:
    commit 559762d, Mar 14 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant