Skip to content

feature, async-mode descriptors + FAPL executor scaffold #252

@steven-varga

Description

@steven-varga

Phase II of the h5cpp multithreading workplan (tasks/h5cpp-fapl-multithreading-workplan.md §4) — first of two PRs.

This PR introduces the descriptor types, FAPL executor property, and executor thread that together give Phase II its lifecycle. Operation overloads (h5::write / h5::read / h5::create for ds/gr/attr on async fds) are out of scope here — they land in a follow-up issue once the scaffold is stable.

Surface

namespace h5::async {
    using fd_t   = impl::hid_t<file_t,  H5Fclose, false, false, ...>;
    using ds_t   = impl::hid_t<ds_t,    H5Dclose, false, false, ...>;
    using gr_t   = impl::hid_t<group_t, H5Gclose, false, false, ...>;
    using attr_t = impl::hid_t<attr_t,  H5Aclose, false, false, ...>;

    fd_t create(const char* path, unsigned flags, /* FCPL/FAPL... */);
    fd_t open  (const char* path, unsigned flags, /* FAPL... */);
}

The false, false specialization disables operator ::hid_t() so user code can't bypass the executor by calling HDF5 directly:

h5::async::fd_t fd = h5::async::create("data.h5", H5F_ACC_TRUNC);
H5Gcreate2(fd, "/g", H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
// COMPILE ERROR: 'operator ::hid_t()' deleted on h5::async::fd_t

h5::is_async_v<T> trait + a concepts::file_descriptor<T> covering both classic and async are added so future operation overloads can dispatch via if constexpr.

Mechanism

Mirror of the Phase I (#250) shared-ptr-in-slot pattern, applied to an executor instead of a pool:

  • H5Pfapl_async.hpp — executor FAPL property; h5::async::create / h5::async::open install the slot.
  • H5executor.hpp — single executor thread per async fd (std::jthread on C++20, stoppable_thread_t shim on C++17). MPSC command queue (256 slots), submit_and_wait facade.
  • The executor is the only thread that calls HDF5. Compression continues to parallelize through the worker pool from feature, FAPL-scoped worker pool with h5::threads{N} for parallel filter compression #250 (executor holds a shared_ptr<worker_pool_t> resolved from the FAPL or auto-installed at default size).
  • Exception in an HDF5 call propagates back to the originating user thread through the std::future returned by the command queue.

Out of scope (follow-up issue)

  • Operation overloads (h5::write, h5::read, h5::append, h5::flush).
  • Dataset / group / attr creation overloads (h5::create(async_fd, ...)).
  • Mode-transitive deduction tests.
  • h5::pt_t CTAD class template — separate refactor; tracked alongside the operation overloads.
  • Performance benchmarks vs. HDF5 --enable-threadsafe.

Acceptance criteria

Mandatory
H5Gcreate2(async_fd, ...) and equivalents are compile-fail with a clear = delete diagnostic
h5::async::create / open round-trip — file opens, closes, executor joins
Executor + pool lifecycle: file close fully drains, stops, joins both threads
Exception in a queued HDF5 call propagates back to the submitter via submit_and_wait
TSAN-clean executor lifecycle across 8+ user threads creating and destroying async fds
Should-have
H5Pfapl_async.hpp reuses the slot helper extracted from #250 (deduplicate copy_cb / close_cb pattern across pool and executor)
Default-size worker_pool_t auto-installed when h5::async::create / open is called without h5::threads{N}

Coupling

Design discussion (settled)

  • Namespace shape: h5::async::fd_t (nested namespace) over h5::async_fd_t (prefix).
  • Factory shape: named factories h5::async::create / open. No h5::async tag literal, no explicit template argument at call sites, no property-pack dispatch.
  • Operations stay in h5:: and dispatch via concept-constrained function templates + TAD on the FD type.
  • h5::pt_t becomes a class template template <class DS = h5::ds_t> deduced via CTAD — tracked in the follow-up issue.
  • h5::threads{N} | h5::backpressure{K} is orthogonal to h5::async: Phase I and Phase II are composable but distinct concerns.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions