Categorical (binary) column incorrectly treated as continuous for Univariate Drift Detection

[nikml]

Describe the bug
The binary predictions from the synthetic binary classification are treated as continuous rather than categorical.

To Reproduce
Steps to reproduce the behavior:
Run the Univariate Drift Example Notebook from where documentation is created.
y_pred is treated as continuous instead of categorical.

Expected behavior
Column would be treated as continuous.

Screenshots & scripts
The variable is present in the continuous drift results for v0.8.1:
https://nannyml.readthedocs.io/en/v0.8.1/_images/drift-guide-continuous.svg

[nnansters]

Hey Nikos,

this behavior is correct. The columns are designated by NannyML as continuous or categorical in the base module.

You are right however that this is not the expected behavior given the example in the docs. This can be fixed by explicitly setting the y_pred column as categorical. I'll update this in the documentation.

reference_df['y_pred'] = reference_df['y_pred'].astype("category")
analysis_df['y_pred'] = analysis_df['y_pred'].astype("category")

column_names = ['distance_from_office', 'salary_range', 'gas_price_per_litre', 'public_transportation_cost', 'wfh_prev_workday', 'workday', 'tenure', 'y_pred_proba', 'y_pred']
calc = nml.UnivariateDriftCalculator(
    column_names=column_names,
    timestamp_column_name='timestamp',
    continuous_methods=['kolmogorov_smirnov', 'jensen_shannon'],
    categorical_methods=['chi2', 'jensen_shannon'],
)

[nikml]

I looked a bit further into this. Quickstart is also affected. And actually the issue was introduced in version 0.7.0 when we removed the StatisticalOutputDriftCalculator.

So we should also fix that and see if documentation needs a little more polishing.

[nikml]

Closing as quickstart also received a hot fix - we can polish the docs later.