Skip to content

v2.0.0

Choose a tag to compare

@OliverHennhoefer OliverHennhoefer released this 22 Oct 10:41
· 97 commits to main since this release
f1b3d70

Added

  • Added group-conditional (Mondrian) calibration. Pass groups= with one label
    per row to ConformalDetector.fit() or calibrate(), and the test labels to
    compute_p_values(), select(), or compute_p_value(group=...). Each test
    point is then ranked only among calibration scores from its own group. With
    Split or detached calibration and exchangeable data within each group,
    p-values stay valid within each group when the test batch has a different
    group mix than the reference data. select() applies BH once to the whole
    batch. ConformalDetector.calibration_groups returns the group of each
    calibration score. Supported for unweighted Split, detached Split,
    CrossValidation, and JackknifeBootstrap with classical Empirical
    p-values; as without groups, CrossValidation and JackknifeBootstrap
    carry no finite-sample guarantee. Randomized, conditional, and KDE p-values, weighted mode,
    DerandomizedSplits, method="storey", FDP and FPR certificates, and
    result-aware e-values reject grouped calibration. Behavior without groups=
    is unchanged.
  • Added nonconform.fdr.FPRCertificate with immutable simultaneous raw-score
    false-positive-rate bounds, target-FPR threshold inversion, and native
    ConformalDetector.fpr_bounds() / ConformalResult.fpr_bounds() integration
    for unweighted Split calibration independently of the p-value estimator.
    A deterministic finite-sample one-sided KS band covers the population inlier
    false-alarm curve with the requested probability over the calibration draw,
    conditional on a scoring map fixed independently of calibration. Coverage
    requires clean, i.i.d. calibration and future inlier scores from the same
    distribution. Returned thresholds preserve NumPy precision
    when compared with lower-precision score arrays.
  • Added nonconform.martingales.MixtureMartingale for fixed-weight composition
    of independently maintained martingale, harmonic restart, CUSUM, and
    Shiryaev-Roberts evidence, including direct ExchangeabilityMonitor support.
    Existing SimpleMixtureMartingale behavior is unchanged.
  • Added DerandomizedSplits for repeated split-conformal e-value selection
    through ConformalDetector.fit() and select(), with automatic randomized
    tie handling and defensive diagnostics in last_selection_result. Existing
    strategies and standalone e-value functions retain their behavior.
  • Added result-aware derandomized conformal e-value selection through
    nonconform.fdr.select_conformal_e_values, plus low-level e-value and e-BH
    primitives. For unmodified result snapshots, the workflow verifies
    split-conformal provenance and exact test batch identity, rejects score ties
    by default, and supports reproducible randomized tie-breaking through tie_seed.
  • Added method="storey" to ConformalDetector.select() for Storey's adaptive
    BH, which estimates the null proportion as
    (1 + #{p >= lambda}) / (m (1 - lambda)) with lambda on the classical
    p-value grid at or just above 1/2, and applies BH at alpha divided by that
    estimate. For classical Empirical p-values from Split or detached
    calibration with exchangeable, continuous scores, it controls FDR at alpha.
    It requires unweighted classical Empirical or ConditionalEmpirical
    p-values and rejects randomized, KDE-based, weighted, and DerandomizedSplits
    selection. The default method="bh" is unchanged.
  • Added ConformalResult.adjusted_p_values(method="bh" | "storey") for native
    unweighted p-value snapshots. Thresholding the adjusted values at alpha
    reproduces the corresponding select() mask.
  • select() now logs a warning on the nonconform.fdr logger when a standard
    selection of classical Empirical p-values is empty and the p-value floor
    1 / (n_calibration + 1) exceeds the first step-up threshold, so a single
    anomaly could not have been discovered. The warning states the calibration
    size needed. Selections are unchanged.

Changed

  • The fdr and all extras now require online-fdr>=1.0.0, matching the
    documented online_fdr.p_values API.

  • Breaking: FDP certification API replacement. Use
    ConformalDetector.fdp_bounds(x, ...), ConformalResult.fdp_bounds(...), or
    nonconform.fdr.FDPCertificate.from_p_values(...). Certificates are immutable
    and prepare threshold queries once. Move thresholds= from construction to
    to_frame(thresholds=...) or bound_at(...); omit inapplicable method options.
    Native entry points expose only confidence and use the default boosted
    Monte Carlo THC envelope; advanced settings belong to the expert factory.
    Berk-Jones numerical tolerance is internal. Monte Carlo confidence covers
    data and independent envelope sampling jointly, not every fixed seed or
    realized envelope; settings must be chosen before inspecting bounds.
    Native entry points require unmodified native empirical Split/detached,
    unweighted snapshots. External p-values use the explicit expert factory.
    Version 2.0.0 is a major release because it replaces the
    released FDP helpers without compatibility aliases. See the
    FDP migration guide
    for the replacement entry points and query options.

  • Failed fits and supported detached calibration attempts now leave every
    strategy unfitted, with results and prepared weights cleared. Fit or calibrate
    again before requesting evidence. Rejected set_params updates preserve the
    existing configuration and calibration; constructor snapshots returned by
    get_params are detached copies.
    Recreate and refit detector objects serialized with 1.x, whose internal fitted
    state uses the previous layout.

  • Validate non-negative integer seeds and two-dimensional batches consistently;
    scoring requires the fitted feature count and one score per input row.

Removed

  • Removed FDPBoundResult, conformal_fdp_upper_bound, and
    conformal_fdp_upper_bound_from_result without compatibility aliases.

Fixed

  • Nested parameter updates no longer mutate stored clone parameters before
    validation, and rejected updates no longer mix old calibration with a new
    configuration.
  • Prepared weight batches own their arrays, so external estimator refits cannot
    replace weights for the recorded batch. Failed weight preparation invalidates
    earlier prepared weights. Frozen bootstrap weighting shares the detector's
    value-based batch identity, including independently allocated object arrays.
  • Array-conversion decorators preserve public method signatures for typed
    consumers instead of erasing arguments and return values to Any.
  • FPR certificates now support copying and pickle roundtrips while retaining
    the prepared KS band and immutable evidence. Default diagnostics and target
    inversion share a complete decision grid, including thresholds above tied
    calibration scores and the empty rule at positive infinity. Its FPR bound
    is now exactly zero instead of the KS margin; finite-threshold bounds and
    the statistical assumptions are unchanged.
  • FDP higher-criticism shape parameters now reject beta > 1, outside the
    range supported by the jump-point statistic calculation. Berk-Jones
    inversion now returns conservative lower endpoints, includes p-values equal
    to those endpoints, handles unreachable lower roots, and terminates when
    floating-point bisection stalls. Previously,
    accepted expert settings could understate FDP bounds or hang construction;
    Berk-Jones bounds can increase to preserve coverage.
  • HC certificates with a positive infinite Monte Carlo cutoff now use a
    conservative unit ECDF envelope instead of producing endpoint NaNs:
    nonempty selections have FDP bound 1, empty selections have bound 0.
    Sampling, the quantile convention, and finite-cutoff formulas are unchanged.

Full changelog: b8aefdc...v2.0.0