Hist bins and explicit zero customization

[eccodolf]

Add explicit hist bins customization in form of i.e. dict in mapping section:
column_mapping['numerical_features'] = {'feature1: {hist_bins: 5}, feature2: (hist_bins:10)}

Also it is common than some features have preavailing 0 values (i.e. product counts within clients purchases). Currently auto binning combines theese zeros with another counts to make bins like 0-4, 5-9 etc. But theese zeros need to be shown separately.

My suggestion is to include this config in mapping section:
column_mapping['numerical_features'] = {'feature1: {hist_bins: 5, explicit_zero: True}, feature2: {hist_bins:10, explicit_zero: False}}

[emeli-dral]

Hi @eccodolf

Thanks for sharing this feature request. I feel your pain with nulls (been there!), and bin customization is something I also wish to add!

We definitely plan to do that, just slightly later on.

The reasoning is the following:

• existing column mapping configuration is easy to use, and we do not want to compromise on this usability since many users shared they like the plug-n-play

• there are some more parameters we want to make configurable in addition to bins (like p-value boundaries for statistical tests etc.)
• we think the best way to combine ease of setup and ability to customize the reports is to add a second (optional) config. It will include all these more sophisticated parameters that some users might want to tweak.

We will add this more complex config in future releases and address the explicit zero issue. And thanks a lot for proposing the implementation.

Happy to hear your thoughts!

[aakash-dusane]

Add explicit hist bins customization in form of i.e. dict in mapping section:
column_mapping['numerical_features'] = {'feature1: {hist_bins: 5}, feature2: (hist_bins:10)}

Also it is common than some features have preavailing 0 values (i.e. product counts within clients purchases). Currently auto binning combines theese zeros with another counts to make bins like 0-4, 5-9 etc. But theese zeros need to be shown separately.

My suggestion is to include this config in mapping section:
column_mapping['numerical_features'] = {'feature1: {hist_bins: 5, explicit_zero: True}, feature2: {hist_bins:10, explicit_zero: False}}

I was also trying to find a way to change the same. Automated binning becomes a problem when you have explicit values such as -9999 which should be a category/bar in itself.

[phaniparsa]

Hello, Could you please provide a way to pass in bin-edges as well for the Histogram in Data Drift Widget. I want to get the bin edges from "reference" data and use the same bin edges for "current" data. I tried exploring graph_objs code but I don't see a way to pass in these explicitly. Will the code work if I swap graph_objs with plotly_express? Or will there be any issues downstream if I modify this code?

[emeli-dral]

Hi @phaniparsa, @eccodolf, @aakash-dusane , we just added something very similar to what you requested: the possibility to set custom bins for the data drift report using column_mapping. Here are the docs: https://docs.evidentlyai.com/reports/data-drift#histogram-plots-customization and here is an example: https://github.com/evidentlyai/evidently/blob/main/evidently/examples/boston_dataset_drift_with_customized_bins.ipynb

Would be great if you could test (please download the latest package version 0.1.23.dev0) and tell us if it solves your feature request!

[phaniparsa]

Thank you @emeli-dral. I'll test this from my end and let you know soon.