v0.5

Improvements:

• Per-transaction compression ratio variability is now active
  • CompressionReductionMethod::get_reduced_variable_global_size() and
    get_reduced_variable_local_size() now accept an optional transaction_id
    parameter and call get_effective_ratio(transaction_id) to compute the
    actual ratio, which may be perturbed around the nominal value when
    ratio_variability > 0
  • Engine::put() passes the current transaction id to both calls, so the
    simulated transfer size varies realistically from transaction to transaction
    when ratio_variability is set
  • The same transaction_id always produces the same effective ratio
    (deterministic hash of variable name and transaction id), making
    simulations reproducible
  • ReductionMethod base-class virtuals and all overrides (Compression,
    Decimation) updated accordingly; Decimation ignores the parameter as its
    reduced sizes are transaction-independent
  • Python bindings updated: get_reduced_variable_global_size() and
    get_reduced_variable_local_size() accept an optional transaction_id
    keyword argument (default 0)
• Transaction cancellation support
  • New Engine::cancel_transaction(transaction_id) allows an external actor
    to cancel a specific in-flight transaction, unblocking both its
    publishers and subscribers
  • Publishers and subscribers receive a TransactionCanceledException;
    catching it allows them to survive the cancellation and continue to
    subsequent transactions
  • Works with both File and Staging engines across any number of publishers
    and subscribers
• Memory efficiency: progressive eviction of metadata transaction entries
  • Once all subscribers have consumed a transaction, its entries are evicted
    from the in-memory Metadata::transaction_infos_ map
  • When metadata export is enabled and publishers and subscribers coexist
    (file streaming), evicted entries are progressively flushed to
    per-variable temporary files; the final metadata file is assembled at
    pub_close() from those files and any remaining in-memory entries,
    preserving the existing format and transaction count
  • When the stream is opened by subscribers only after all publishers have
    closed (sequential scenario), memory-only eviction is performed (the
    metadata file has already been written by pub_close())
  • Memory footprint of the File engine now grows as O(N_pub) instead of
    O(N_pub × N_transactions) for long-running concurrent streaming workloads
• Improved test coverage
  • Comprehensive C++ and Python test suite for transaction cancellation,
    covering cancellation while waiting for a subscriber, while waiting for
    a publisher, and mid-transaction for both File and Staging engines
  • Tests for the 3-parameter form of DTL::add_stream
  • Tests for propagation of reduction parameters to subscribers for both
    compression and decimation methods

Bug Fixes:

• SZ compression model: the ratio formula incorrectly used -log10(accuracy)
  instead of accuracy^beta, inverting the accuracy-to-ratio relationship;
  constants recalibrated to give ratio ≈ 7 at accuracy = 1e-3 and ratio ≈ 2
  at accuracy = 1e-6
• CompressionReductionMethod did not register the subscriber-side variable
  during inquire_variable, preventing Engine::get() from simulating
  decompression costs
• DecimationReductionMethod did not propagate its parameterization to
  subscribers, preventing them from querying reduced variable sizes after
  inquire_variable
• Transaction cancellation in StagingEngine: CancelException and
  NetworkFailureException thrown during pub_close and sub_close were not
  caught when a cancellation was in progress, leaving in-flight activities
  in an inconsistent state
• StagingEngine::begin_sub_transaction did not decrement
  num_subscribers_starting_ when a transaction was canceled, causing an
  imbalance in subsequent synchronization
• CompressionReductionMethod::ParameterizedCompression stored a raw pointer
  to the associated Variable; replaced with the variable name string to
  eliminate potential dangling references

API Changes:

• New Engine method:
  • Engine::cancel_transaction(unsigned int transaction_id) cancels a
    specific transaction performed by this engine. This method must be
    called from an external actor not involved in the transaction. It raises
    a TransactionCanceledException that publishers and subscribers must catch
    to survive the cancellation and continue to subsequent transactions. If
    both sides have already moved past the given transaction ID, the call is
    a no-op to avoid accidentally canceling a subsequent transaction.
• New Stream helper methods:
  • Stream::get_engine_type() and Stream::get_transport_method() respectively
    return the enum value of the engine type and transport method for a
    Stream.
  • As a consequence, in the Python bindings, the Stream.engine_type and
    Stream.transport_method readonly properties that return the engine type
    and transport method as a string have been renamed to
    Stream.engine_type_str and Stream.transport_method_str; the existing
    properties now return the enum values.
  • Stream::get_reduction_method(name) retrieves a reduction method
    associated to the stream by name, returning std::nullopt if not found.
• New ReductionMethod virtual method:
  • ReductionMethod::propagate_for_subscriber(publisher_var, subscriber_var)
    copies the publisher-side parameterization to the subscriber variable;
    the default implementation is a no-op; both DecimationReductionMethod
    and CompressionReductionMethod provide overrides called by
    Stream::inquire_variable.
• Behavior modifications:
  • DTL::add_stream can now take an Engine::Type and a Transport::Method as
    optional parameters (both defaulting to Undefined if not specified).
  • Reduction methods can now be declared in the DTL configuration file via
    a "reduction_methods" array in the stream definition.