Skip to content

0.14.0

Latest

Choose a tag to compare

@github-actions github-actions released this 29 Oct 12:16

Metrics

  • Feature A metric's model attribute, a ModelInterface, gives an
    estimator of your own what Empulse's models use of a loss: its parameters, decision costs,
    decision threshold, loss, training objectives and cost scale. See User Guide.
  • API Change Direction, which a custom MetricStrategy
    passes to its constructor and every metric reports as direction, is exported from
    metrics. It could only be imported from a private module.
  • Feature Capability has two new members. LOGIT_VALUE_OBJECTIVE states
    that a strategy can train a logistic regression with an optimizer that only compares losses, such
    as GeneticAlgorithmOptimizer. Every strategy with LOGIT_OBJECTIVE
    has it, and so do EmpiricalMaxProfit,
    EmpiricalMinCost and AUEPC. EXPECTED_COST
    is present for Cost, Profit and
    Savings.
  • Feature [constrain>](https://empulse.readthedocs.io/en/stable/reference/generated/CostMatrix.constrain <empulse.metrics.CostMatrix.html#CostMatrix.constrain <empulse.metrics.CostMatrix.constrain>) and
    [constrain>](https://empulse.readthedocs.io/en/stable/reference/generated/MixtureMetric.constrain <empulse.metrics.MixtureMetric.html#MixtureMetric.constrain <empulse.metrics.MixtureMetric.constrain>) take a condition relating
    several parameters, written as a comparison such as 'clv > incentive_cost' or as a sympy
    relation, which may name aliases. Unlike a lambda, which cannot be pickled, it keeps the metric and
    the models fitted with it picklable, for instance by dump. A condition on a name that
    is no symbol or alias raises when the metric is built.
  • Fix A MixtureMetric of Savings components
    decides and trains on the weighted sum of their savings rather than of their costs, so the
    component with the largest costs no longer decides alone. This applies to optimal_threshold,
    optimal_rate, the training objectives, the cost-sensitive trees,
    CSThresholdClassifier and CostSensitiveSampler.
    Because the weights now depend on the labels, such a mixture can no longer decide from costs passed
    to predict.
  • Fix The optimal_threshold and optimal_rate of a MixtureMetric
    no longer average its components' rates. A mixture of components of one strategy type that decide
    from their costs alone, such as Cost, takes them from the weighted
    combined costs. A mixture without Capability.OPTIMAL_RATE or Capability.OPTIMAL_THRESHOLD,
    such as one of a Profit and a Savings component,
    raises a ValueError.
  • API Change MixtureMetric rejects components with different optimization
    directions, and components that are amounts of money mixed with ones that are pure numbers, such
    as AUEPC. Weights must not be negative and must sum to 1: constant
    weights are checked when the mixture is built, the others when it is called.
  • API Change A MixtureComponent weight can be an expression, such as
    '1 - success_rate - default_rate', whose parameters are parameters of the mixture. They can be
    passed to a model's fit and are reported when missing. A callable weight is deprecated.
    empcs_score uses an expression weight, so it and the models trained on it
    can be pickled.
  • Fix [constrain>](https://empulse.readthedocs.io/en/stable/reference/generated/MixtureMetric.constrain <empulse.metrics.MixtureMetric.html#MixtureMetric.constrain <empulse.metrics.MixtureMetric.constrain>) rejects a lower
    bound above the upper one, a name that is not a parameter, and a stochastic variable, which was
    accepted and never checked. Called twice, it keeps the tighter bound. The bounds and conditions
    also check a component's default when the parameter is not passed, unless the components have
    different defaults.
  • Fix A metric accepts a parameter under its symbol name or any of its aliases wherever it takes
    one, and set_default and
    constrain can be called before
    alias. In a MixtureMetric, a bound on
    an alias was bypassed by passing the symbol, a predicate raised a KeyError for a parameter
    passed by its alias, and fixed parameters and defaults applied under one spelling only. A mixture
    no longer warns about a parameter that only another component uses. A constraint on a name that is
    neither a symbol nor an alias raises instead of being ignored.
  • Fix [mark_outlier_sensitive>](https://empulse.readthedocs.io/en/stable/reference/generated/CostMatrix.mark_outlier_sensitive <empulse.metrics.CostMatrix.html#CostMatrix.mark_outlier_sensitive <empulse.metrics.CostMatrix.mark_outlier_sensitive>)
    accepts an alias, finds a symbol created with assumptions such as positive=True, and works for
    a symbol with several aliases, which RobustCSClassifier could not use. An
    outlier-sensitive parameter is reported under its symbol name, and one that only enters a
    distribution is classified by the costs that distribution enters. A parameter that components of a
    MixtureMetric mark for different classes is checked for outliers on every
    sample, and one that every component fixes is no longer outlier-sensitive.
  • Fix EmpiricalMaxProfit and EmpiricalMinCost
    raise a ValueError on empty input, as MaxProfit does, instead of a
    ZeroDivisionError.
  • Fix The optimal_threshold of Cost, Profit,
    Savings and LogCost is a single number when a
    parameter is an array of length 1, which applies to every sample, instead of an array of one.
  • Efficiency Scoring the empirical strategies is up to a quarter faster, and scoring
    Cost, Savings and
    LogCost gains another tenth on top of that. This speeds up the models
    that score them on every iteration.
  • Efficiency Copying a metric, as cloning or fitting a model with it as loss does, reuses its
    compiled expressions instead of compiling them again. Cloning
    CSLogitClassifier(loss=empc_score) is 2.5 times as fast.
  • API Change parameter_names lists every name a caller can pass, aliases
    included. It lists a stochastic variable's distribution parameters instead of the variable, which
    is drawn rather than passed. A model's set_fit_request no longer offers the variable either.
  • API Change Building a Metric raises a ValueError when the cost matrix
    names an alias after another of its symbols, which could then not be passed under its own name. It
    also raises for a default or an outlier-sensitive marker on a stochastic variable, and for an
    outlier-sensitive marker on a name that is neither a symbol nor an alias, which surfaced only as a
    misleading error when fitting a RobustCSClassifier.
  • Fix The parameters of a stochastic variable's distribution are checked for reserved and duplicate
    names like any other symbol, instead of failing with an unclear TypeError or a SyntaxError
    from sympy.
  • API Change Metric and MixtureMetric raise a
    TypeError when passed sample_weight, for instance by a scorer that requests it through
    metadata routing. They accepted it and ignored it, so a weighted cross-validation score was the
    unweighted one. The costs of the cost matrix are what weigh each sample: fold any extra weight into
    them, for example as an instance-dependent parameter. A cost-matrix parameter named
    sample_weight is still accepted.
  • Fix Savings, savings_score and
    expected_savings_score rank models correctly when the baseline cost is
    negative, as it is for a matrix of benefits: savings are now divided by the magnitude of the
    baseline cost. A baseline that costs nothing returns nan with a warning instead of a huge number.
  • Fix The 'prior' baseline of Savings predicts the positive class
    prior, as documented, and a scalar or list baseline is accepted.
  • Fix [optimal_threshold>](https://empulse.readthedocs.io/en/stable/reference/generated/LogCost.optimal_threshold <empulse.metrics.LogCost.html#LogCost.optimal_threshold <empulse.metrics.LogCost.optimal_threshold>) and
    optimal_rate are the Bayes-optimal decision for the scores of a model trained on the log cost,
    0.5 whenever the cost matrix targets some but not all samples, instead of reusing
    Cost's threshold for calibrated probabilities.
  • Fix When targeting nobody is optimal, optimal_threshold and
    classification_threshold return inf instead of 1.0, which still
    targeted a probability of exactly 1.0 and some samples' decision scores. Cost,
    Profit, Savings, LogCost,
    CSThresholdClassifier, cost_loss and
    savings_score no longer invert the decision when the costs of the wrong
    decisions are smaller than those of the right ones, as with fp_cost=1 and tp_cost=3. Costs
    that make targeting pay off the less likely a sample is to be positive, which no threshold can
    express, raise a ValueError.
  • Fix Cost, Profit,
    Savings and LogCost no longer raise in
    optimal_threshold and optimal_rate when, for some sample, the costs do not depend on the
    predicted probability. Such a sample is always targeted if that costs less and otherwise never,
    including when both decisions cost the same.
  • Fix A cost-matrix symbol named after a function its expression calls, such as a Weibull scale
    named gamma, no longer breaks the metric.
  • Fix An instance-dependent distribution parameter is validated when another parameter of the same
    distribution is a scalar or a single-element list or array, e.g. a negative alpha array beside
    a scalar beta.
  • Fix Metrics accept boolean labels.
  • Fix MaxProfit and MinCost average each
    instance-dependent cost and benefit over the samples, rather than the parameters they are computed
    from, so a cost such as the product of two instance-dependent parameters is averaged correctly.
    CSThresholdClassifier learns its single threshold from the same averaged
    costs for any loss, MixtureMetric included, and
    ProfLogitClassifier with a gradient-free optimizer optimizes the same
    score. A stochastic cost that is not linear in instance-dependent parameters warns that it is
    approximated.
  • Fix MaxProfit with stochastic variables finds the best threshold when
    targeting loses money. When targeting either class loses, the optimum jumps between targeting
    everybody and nobody. When targeting a positive loses and targeting a negative gains, it searched
    only the part of the ROC convex hull above its diagonal. Other cost matrices are scored as before.
  • Fix MaxProfit integrates numerically wherever a stochastic variable's
    mass lies. With integration_method='quad', or a single variable whose expected profit has no
    closed form, a distribution far from zero, such as a Laplace(200, 2), or much narrower or wider
    than one gave about zero without a warning. Concentrated distributions, such as a Logistic or a
    Gamma, Chi-squared or Beta with large shape parameters, no longer give nan or raise an
    OverflowError. This fixes empc_score with a concentrated Beta prior. A
    large estimated integration error warns.
  • Fix MaxProfit finds every point where the optimal threshold changes
    for a profit that is not a polynomial in its stochastic variable, such as exp(v / 2), which
    gave nan or a wrong optimal rate.
  • Fix MaxProfit with a discrete stochastic variable, such as a Poisson or
    a Bernoulli one, uses Monte Carlo integration under integration_method='auto', and
    'quad' raises a clear error, instead of failing with an IndexError or an
    AttributeError. 'quasi-monte-carlo' with a distribution it has no quantile function for
    raises a clear error instead of a KeyError.
  • Fix MaxProfit with integration_method='monte-carlo' gives the same
    result for the same distribution parameters, whichever were evaluated before. Under
    'monte-carlo' and 'quasi-monte-carlo', the optimal rate is computed on the same samples as
    the score, so the two describe the same operating point. Results for a given random_state
    change.
  • Fix A string term of a CostMatrix can no longer run Python code,
    which made it unsafe to build a cost matrix from untrusted input. A string term may only use
    names, numbers, arithmetic and sympy's mathematical functions such as exp or Max; build
    anything else, such as sympy.Rational(1, 2), from sympy objects.
  • Fix A callable passed to [constrain>](https://empulse.readthedocs.io/en/stable/reference/generated/CostMatrix.constrain <empulse.metrics.CostMatrix.html#CostMatrix.constrain <empulse.metrics.CostMatrix.constrain>) works
    with instance-dependent parameters: it may answer per sample, and every sample must satisfy it.
  • Fix The LaTeX of Savings shows the savings relative to the baseline
    instead of the cost divided by it.
  • Fix Strategies that reduce a stochastic variable, such as Cost and
    EmpiricalMaxProfit, take the expectation of a term that is not linear in
    it, such as gamma ** 2, instead of evaluating the term at the variable's mean. A term that is
    not a polynomial in it, such as exp(gamma), warns that it is approximated.
  • API Change [cost_matrix>](https://empulse.readthedocs.io/en/stable/reference/generated/Metric.cost_matrix <empulse.metrics.Metric.cost_matrix>.html#Metric.cost_matrix <empulse.metrics.Metric.cost_matrix>) returns a copy: a metric is
    compiled from its cost matrix when it is built, so changing the matrix afterwards had no effect on
    scoring. Build a new metric from the changed cost matrix instead.
  • API Change MetricStrategy.requires_dynamic_boost_objective is removed. Use
    Capability.PRECOMPUTED_BOOST_OBJECTIVE not in strategy.capabilities instead.
  • API Change Metrics, strategies and training objectives pickled with 0.13 or earlier are no longer
    supported. Build or fit them again with this release before pickling them.
  • Efficiency MaxProfit is faster with stochastic variables. A profit of
    degree three or more is scored up to a third faster, the analytic gradients that
    ProfLogitClassifier and CSBoostClassifier train
    with are 10 to 20% faster, and the optimal rate up to 15% faster. optimal_threshold of a metric
    without stochastic variables is up to twice as fast.
  • Enhancement The optimal rate of a MaxProfit metric with one stochastic
    variable under integration_method='auto' is exact for every distribution that
    'quasi-monte-carlo' supports, such as Laplace and Logistic, and two to three times as
    fast for those.
  • Fix The docstrings of max_profit_score and
    MaxProfit correct the formula and say how instance-dependent parameters
    are averaged.

Models

  • API Change ProfLogitClassifier, ProfTreeClassifier,
    ProfSRClassifier, ProfMPMClassifier and
    ProfMEMPMClassifier take a threshold parameter. With the default
    'auto', predict predicts the positive class above threshold_, the probability that costs
    the training samples least, instead of above 0.5. Across nine benchmark datasets this raised the
    share of attainable savings on held-out data by 0.03 to 0.33 on average, depending on the model.
    Pass threshold=0.5 for the previous predictions. See User Guide.
  • Feature CSThresholdClassifier and CSRateClassifier
    take a cv parameter to learn a decision that depends on the scores, such as a
    MaxProfit threshold or any rate, from out-of-fold scores instead of the
    scores of the training rows, which flatter an estimator that overfits them. A threshold that
    depends only on the costs ignores it.
  • API Change CSForestClassifier accepts min_impurity_decrease=None, as
    CSTreeClassifier does, and both it and the default tree of
    CSBaggingClassifier now use it: a tree only splits while that lowers the
    cost of its training samples, instead of growing until its leaves are pure. On the 15 bundled
    datasets the ensembles' own predictions save more, with a tenth of the leaves or fewer and fits
    taking half the time. Pass min_impurity_decrease=0.0 for the previous trees.
  • Enhancement CSThresholdClassifier, CSRateClassifier,
    RobustCSClassifier, BiasRelabelingClassifier,
    BiasResamplingClassifier and BiasReweighingClassifier
    pass X to their estimator unchanged, so it can be a Pipeline that
    encodes strings, imputes missing values or selects columns by name, and it keeps the feature names.
    Their input tags follow the estimator's, and for RobustCSClassifier also
    its outlier_estimator's, which by default rejects missing values. Without metadata routing,
    CSThresholdClassifier and CSRateClassifier forward
    their other fit keyword arguments, such as sample_weight, to the estimator, and the costs too
    when its fit accepts them. BiasReweighingClassifier passes its weights
    to a pipeline's last step, or, with metadata routing enabled, to the steps that request them, and
    rejects estimator=None.
  • Enhancement CSBoostClassifier and B2BoostClassifier
    accept missing values in X, which XGBoost, LightGBM and CatBoost all handle, and so do
    CSThresholdClassifier and CSRateClassifier around
    them. CSBaggingClassifier accepts missing values when its base estimator
    does, as its default CSTreeClassifier and
    CSForestClassifier do.
  • API Change CSLogitClassifier and ProfLogitClassifier
    raise a ValueError before fitting when their optimizer needs a gradient the loss has no
    objective for, such as LBFGSBOptimizer with an
    EmpiricalMaxProfit metric, and the message names an optimizer that works.
    They raised a NotImplementedError from inside the fit.
  • Fix The baseline of a Savings loss can be passed to the fit of
    cost-sensitive models and routed through metadata routing like the cost matrix's parameters, also
    in a nested MixtureMetric whose components use different strategies. Most
    models raised a TypeError, a nested mixture ignored it with an unknown-parameter warning, and
    CSThresholdClassifier and CSRateClassifier passed it
    on to their estimator, so a mixture of savings always trained against the 'zero_one' baseline.
  • Fix A loss parameter passed to fit as a list or a pandas Series is treated like an array.
    CSThresholdClassifier learned one threshold per training sample from it,
    so predict failed on other data or applied the wrong thresholds, and weighted voting in
    CSForestClassifier and CSBaggingClassifier raised
    a length mismatch.
  • Fix Cost-sensitive models accept an array fit parameter whose length is not the number of
    samples, such as a per-feature weight, when it is not a cost or loss parameter.
  • Fix After set_params(loss=...), set_fit_request accepts the new loss's parameters.
  • Fix CSThresholdClassifier and CSRateClassifier
    without a loss keep each cost not passed to predict at its value in fit, instead of the
    constructor's, which predicted every sample positive after fit(X, y, fp_cost=5, fn_cost=10)
    and predict(X, fn_cost=10). A predict-time cost with a length other than the samples' is
    rejected with a clear error. Fitted without costs, or with every cost zero even over non-zero
    constructor costs, they learn no decision, forget the decision of an earlier fit, and fit the same
    calibrated estimator as with costs.
  • Fix CSThresholdClassifier's default calibrator passes the costs on to a
    cost-sensitive estimator, which was trained on fp_cost=fn_cost=1 instead, and with metadata
    routing it is fitted with the sample weights the estimator requests, which raised before.
  • Fix CSThresholdClassifier and CSRateClassifier
    with pos_label set to the smaller class pass a cost-sensitive estimator the costs as it sees
    them, with the true and false positives and negatives swapped, instead of inverting its training.
  • Fix CSRateClassifier predicts exactly ceil(rate_ * n_samples) samples
    positive. Tied scores at the cut-off made it predict more, up to five times the rate on a shallow
    tree's scores, and rounding error one more for rates such as 0.07 of 100 samples. Its documentation
    now says that the rate applies to each predict batch.
  • Fix RobustCSClassifier standardizes the residuals of the costs, so
    outlier_threshold is a number of standard deviations. It divided the absolute residuals by
    their own standard deviation, as the original paper does, so the default of 2.5 flagged around 13%
    of normally distributed costs instead of 1%.
  • Fix RobustCSClassifier measures and imputes the outliers of a cost that
    only enters one class's rows, such as fn_cost, among that class's rows. The other class's
    values widened the spread, so real outliers were missed and rows the cost never uses were imputed.
    It also detects outliers on the positive class of a wrapped estimator's pos_label, and accepts
    an outlier-sensitive parameter given as a list.
  • Fix RobustCSClassifier wraps a Pipeline whose
    last step is a cost-sensitive classifier. It looked for the loss and the costs on the pipeline itself,
    so it missed a loss set on the classifier, and passed the costs to Pipeline.fit under their
    plain names, which it rejects. Without metadata routing, the costs now go to the last step as
    <step>__<cost>; with it, to the steps that request them.
  • Fix RobustCSClassifier keeps the costs set on the wrapped estimator
    when neither fit nor its own constructor sets them, instead of overriding them with zero.
  • Fix Refitting CSLogitClassifier or
    ProfLogitClassifier with fit_intercept=False removes the
    intercept_ of an earlier fit.
  • Fix CSTreeClassifier and CSForestClassifier
    accept a class_weight dict keyed by the class labels, as documented. The dict was looked up by
    the 0/1 encoding of the labels, so only the labels 0 and 1 worked. A key that is no class raises a
    clear error, so a dict keyed by 0 and 1 for other labels has to be keyed by the labels instead.
  • Fix CSTreeClassifier, CSForestClassifier and
    CSBaggingClassifier warn when the strategy of their loss is not an
    expected cost, such as MaxProfit: they grow their trees on the metric's
    costs, minimizing their expected cost, whatever the strategy.
  • Fix The criterion="gini" impurity of CSTreeClassifier and
    CSForestClassifier is per unit of weight, like "cost" and
    "entropy". It grew with node size, which favoured even splits and let noise pass
    min_impurity_decrease. Trees grown with it change.
  • Fix A node of CSTreeClassifier, CSForestClassifier
    and CSBaggingClassifier that holds a single class is a leaf, unless its
    samples disagree on which prediction costs less. With negative or non-zero correct-decision costs
    such nodes kept splitting without changing any prediction.
  • Fix Weighted voting in CSForestClassifier and
    CSBaggingClassifier weighs each estimator by how much better than average
    its out-of-bag loss is, relative to the average cost of a wrong decision, and gives a tree without
    out-of-bag samples the average weight. The worst estimator always got no vote, so of two nearly
    equal trees only one counted, and a tree without out-of-bag samples made the forest weigh every tree
    alike and bagging fail.
  • Fix CSForestClassifier passes a callable oob_score the class labels,
    as scikit-learn's forests do, instead of their 0/1 encoding.
  • Fix CSBaggingClassifier fits when a bootstrap sample holds only one
    class, which then gives a tree of a single leaf. It raised a ValueError.
  • Fix CSBaggingClassifier with warm_start=True keeps the estimators
    fitted earlier and only fits the new ones. It refitted all of them.
  • Fix CSBaggingClassifier raises a clear error for a base estimator whose
    fit does not accept the costs.
  • Fix CSBoostClassifier and B2BoostClassifier
    encode the labels of an eval_set passed in fit_params as they encode y. String labels
    crashed XGBoost, and labels such as 1/2 or -1/1 were scored as if they were 0/1.
  • Fix ProfTreeClassifier optimizes a
    MixtureMetric of deterministic MaxProfit
    metrics, instead of the maximum profit of their averaged costs.
  • Fix ProfTreeClassifier raises a clear error when every variation operator
    rate is zero, instead of a ZeroDivisionError. The rates are documented as the relative weights
    they are: they need not sum to 1.
  • Fix ProfTreeClassifier reports n_iter_ correctly when stopping early,
    and reads min_samples_split=1.0 and min_samples_leaf=1.0 as fractions of the samples.
  • Fix BiasReweighingClassifier weighs every sample alike, with a warning,
    when the sensitive feature has a single value, as the resampling and relabeling models do. With
    only the value 0, the weights followed the class prior instead.

Bias mitigation and samplers

  • Fix BiasRelabelingClassifier passes its fit parameters, such as the
    parameters of a Metric loss, to the estimator that ranks the relabelling
    candidates too.
  • Fix The bias mitigation models and samplers raise a clear error for a sensitive_feature that
    is not coded as 0 and 1 (after transform_feature), or whose length differs from y, instead
    of failing with an index error or silently doing nothing.
  • Fix BiasRelabler without a sensitive_feature returns the data
    unchanged, like BiasResampler, and relabels a pandas target with any index.
  • Fix BiasResampler and BiasResamplingClassifier
    keep the number of samples and no longer return the resample ordered by class and group. Results
    for a given random_state change.
  • Fix CostSensitiveSampler rejects oversampling_norm=0.

Datasets

  • Fix The fetch_* loaders only download over https. A http, ftp or file URL, such
    as one returned by the UCI or OpenML API or reached through a redirect, raises an error instead.

Packaging and dependencies

  • Enhancement Support for Python 3.15, including the free-threaded build (cp315t). It needs
    numpy 2.5.2, scipy 1.18.1, scikit-learn 1.9.1 and, for the datasets extra, pyarrow 26.0 or
    later. CatBoost has no Python 3.15 wheels yet, so there the boosting extra installs XGBoost
    and LightGBM only. The polars backend of datasets needs polars 2.0 on Python 3.15.
  • Fix Building Empulse from source requires Cython 3.1.2 or later. Cython 3.1.0 and 3.1.1 built
    a CSBoostClassifier and B2BoostClassifier that
    failed to fit or pickle with XGBoost.