Summarized metadata visualization

[amontanez24]

Description

In many enterprise datasets, there are a very large number of columns. It is infeasible to print out or even visualize the full list of columns. In this case, it would be nice to summarize the metadata in a concise way, allowing to you to see, at-a-glance, what it contains.

API

In visualize, update the show_table_details parameter to accept 3 values:

• (default) 'full': Show the full column names and sdtype (equivalent to today's show_table_details=True)
• None: Don't show any of the column names (equivalent to today's show_table_details=False)
• 'summarized': Show the number of columns for each sdtype (new!)

metadata.visualize(show_table_details='summarized')

Output can show only the total number of columns of each type. Something like this (not exactly!).

Other Details

• The main sdtype are: id, numerical, categorical, datetime and boolean.
• The user may have other sdtypes the represent PII values. Count these all under other.

Backwards Compatibility

If a user still inputs show_table_details=True or show_table_details=False, do not crash. Instead:

1. Convert it to the corresponding new parameter name ('full' or None) and
2. Throw a FutureWarning

# if show_table_details=True
FutureWarning: Using True or False for 'show_table_details' is deprecated. Use table_details='full' to show all table details.

# if show_table_details=False
FutureWarning: Using True or False for 'show_table_details' is deprecated. Use table_details=None to hide table details.