Motivation
threaded_pipeline_t in H5Zpipeline.hpp has been a do-nothing stub since the pipeline was introduced. H5Qall.hpp (#195) provides the correct substrate: a bounded ring to pass chunk descriptors between the split-to-chunk tiler, filter worker threads, and the HDF5 chunk writer.
Design
split_to_chunk_write ──► ring_in (spsc / mpsc)
│
N worker threads:
pop raw chunk → apply filter chain → push ring_out
│
ring_out (spsc)
│
writer thread: H5Dwrite_chunk
Thread topology selection:
n_threads == 1 → bounded::ring::spsc_t (zero CAS on both sides)
n_threads > 1 → bounded::ring::mpsc_t (CAS on producer head, single consumer)
Controlled by H5CPP_PIPELINE_WORKERS cmake option (default 1, preserving single-threaded behavior as the ABI-stable baseline).
Ring capacity is fixed at set_cache() time:
size = filter_scratch_bound(block_size) * 2 * n_workers
Because ring::adaptor_t uses a compile-time n_bytes_v, the implementation caps at H5CPP_MAX_CHUNK_BYTES (new compile-time constant, proposed default 32 MiB) and asserts at set_cache() if the actual chunk exceeds this.
Changes
h5cpp/H5Zpipeline.hpp: replace stub write_chunk_impl / read_chunk_impl in threaded_pipeline_t with ring-backed implementation
- New
h5cpp/H5Zpipeline_threaded.hpp (implementation file, included from H5Zpipeline.hpp)
CMakeLists.txt: add H5CPP_PIPELINE_WORKERS option; add H5CPP_MAX_CHUNK_BYTES constant
test/H5Zpipeline.cpp: add threaded pipeline round-trip test with filter chain
Depends on
#194 (C++20), #195 (H5Qall)
Motivation
threaded_pipeline_tinH5Zpipeline.hpphas been a do-nothing stub since the pipeline was introduced.H5Qall.hpp(#195) provides the correct substrate: a bounded ring to pass chunk descriptors between the split-to-chunk tiler, filter worker threads, and the HDF5 chunk writer.Design
Thread topology selection:
n_threads == 1→bounded::ring::spsc_t(zero CAS on both sides)n_threads > 1→bounded::ring::mpsc_t(CAS on producer head, single consumer)Controlled by
H5CPP_PIPELINE_WORKERScmake option (default1, preserving single-threaded behavior as the ABI-stable baseline).Ring capacity is fixed at
set_cache()time:size = filter_scratch_bound(block_size) * 2 * n_workersBecause
ring::adaptor_tuses a compile-timen_bytes_v, the implementation caps atH5CPP_MAX_CHUNK_BYTES(new compile-time constant, proposed default 32 MiB) and asserts atset_cache()if the actual chunk exceeds this.Changes
h5cpp/H5Zpipeline.hpp: replace stubwrite_chunk_impl/read_chunk_implinthreaded_pipeline_twith ring-backed implementationh5cpp/H5Zpipeline_threaded.hpp(implementation file, included fromH5Zpipeline.hpp)CMakeLists.txt: addH5CPP_PIPELINE_WORKERSoption; addH5CPP_MAX_CHUNK_BYTESconstanttest/H5Zpipeline.cpp: add threaded pipeline round-trip test with filter chainDepends on
#194 (C++20), #195 (H5Qall)