ENH: add missing/infinite values counts in .describe

[lcrmorin]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

As scikit-learn often fail with missing or infinite values, I'd like a default way to count missing / infinite values. Currently we need to use describe and some .isna().sum() separately. It would be nice if the describe method could provide missing / infinite values counts. This could even be extended to count some user defined 'sentinel values'.

Feature Description

Add new rows to the output of the .describe method, with the count of missing values, count of infinite values.

Some parameters can be added to the describe function:

• a list of 'sentinel' values so that the describe method also provide counts for those.
• an option to provide frequency (proportion of total) instead of counts
• an option to enable / disable those counts entirely

Alternative Solutions

The current solution is to do the counts outside of the describe function.

Additional Context

No response

[adirajput2000]

Hey @lcrmorin
I am excited to work with you on this.
Can you please help me out with a discussion?

[adirajput2000]

take

[jesse-sealand]

This definitely serves a need. Consider, if you haven't already, the ability to toggle it on and off. There are cases where the amount of data coming from describe can be overwhelming. Also, sometimes it's needed and sometimes it's not.

[mroeschke]

Thanks for the suggestion but it doesn't seem like this feature request has gotten much traction from the core team so closing as inactive