Skip to content

0.0.10

Choose a tag to compare

@jaydu1 jaydu1 released this 10 Aug 03:04
· 84 commits to main since this release
  • Disk-space awareness – crispyx now estimates the disk space a
    streaming call is about to need and warns -- without blocking the call --
    when free space on the relevant filesystem looks tight or the write is
    unusually large. This covers the disk-backed intermediate accumulators
    behind cx.pb.normalized_effects (batch-corrected path),
    cx.pb.aggregate, cx.pb.effects, cx.tl.t_test,
    cx.tl.wilcoxon_test, cx.tl.nb_glm_test, cx.tl.batch_process, and
    quality-control filtering, plus the ~2x transient disk requirement of
    whole-file CSR/CSC conversion (:func:crispyx.convert_to_csc,
    :func:crispyx.convert_to_csr, and
    normalize_total_log1p(..., format_mismatch_policy="convert")). The
    check is automatic and has no configurable budget analogous to
    memory_limit_gb: it always reads real free space via
    shutil.disk_usage and exists purely as a feasibility heads-up, not a
    resource allocator.
  • New: crispyx.estimate_disk_usage – an on-demand, standalone query
    to check disk usage before committing to a run:
    cx.estimate_disk_usage(func, data, **kwargs) accepts a function name
    (e.g. "compute_normalized_effects", "t_test",
    "convert_to_csc") or the function object itself, plus the same
    arguments the real call would take, and returns the estimated bytes
    required versus free space at each filesystem location involved (e.g.
    $TMPDIR for intermediate accumulators, the output directory for the
    final result). It reads only cheap obs/uns metadata in backed
    mode and never touches the expression matrix. Also available as
    cx.tl.estimate_disk_usage for Scanpy-style namespace discovery (the
    same pattern already used for compute_overlap). See :ref:disk-space
    in the usage guide.
  • Cross-platform robustness – disk-space checks now degrade gracefully
    instead of raising when free space cannot be determined at all (an
    unreachable network mount, a permission error on a Windows junction, a
    drive ejected mid-check): the affected DiskEstimate reports
    free_bytes=None and sufficient=True (fail open) rather than
    crashing the caller's real computation. The "large write" heads-up still
    fires in this case since it doesn't depend on free space.
  • Documentation now notes that the memory/speed figures throughout the
    README, docs, and tutorial assume adequate free scratch disk for
    streaming intermediates and output files.
  • verbose now defaults to True across the package (was False
    on most differential-expression and pseudo-bulk functions). A first-time
    call already reports what it did -- what file is being read, what was
    inferred, what was written -- without passing verbose= explicitly.
    Pass verbose=False (or 0) for the previous silent behaviour. This
    is a behavioural default change, not a signature change: no parameter was
    removed or renamed.
  • Filtering feedback – :func:crispyx.pp.filter_cells,
    :func:crispyx.pp.filter_genes, :func:crispyx.pp.filter_perturbations,
    and :func:crispyx.pp.qc_summary now report kept/total counts and warn
    when a filter removes more than half the data (cells, genes, or
    perturbations), a common sign of a misconfigured threshold.
  • Progress bars extended beyond differential expression to CSC/CSR
    conversion, cx.pb.aggregate, cx.tl.batch_process, and the QC
    streaming passes. They use tqdm when available and degrade to a
    no-op otherwise, gated on the same verbose as everything else.
  • Chunk-size and streaming-strategy reporting – functions that
    auto-select a chunk size, or choose between a single-pass and a
    streaming strategy internally, now say so at the default verbosity
    (e.g. chunk_size=4096 (auto), Strategy — column-streaming).
  • Disk-usage confirmation – every warn_if_disk_space_low call site
    now also prints a verbose-gated confirmation of the estimate
    computed (required GB vs. free GB), shown whether or not the
    unconditional warning fired.
  • Fixed a naming regression in :func:crispyx.pp.qc_summary's verbose
    output (it printed qc.quality_control: instead of
    pp.qc_summary:, left over from before the function was renamed).
  • Warnings for missing batch/grouping values and untestable groups in
    cx.tl.batch_process, cx.pb.aggregate, cx.pb.effects, and
    cx.tl.wilcoxon_test (batch-stratified) are now prefixed with their
    originating function, matching the convention already used by the
    disk-space warnings.
  • See the new :ref:Messaging and verbosity <messaging-and-verbosity>
    section in the usage guide for the full picture of what prints, what
    warns, and what stays at logger level.