Add count labels and ordering options to visualize_feature bar plots

[idanmoradarthas]

Overview

This pull request enhances the visualize_feature function in the ds_utils/preprocess.py module by introducing new parameters for displaying count labels on bar plots and customizing the order of categories. These changes address the limitations highlighted in issue #77, where users noted the difficulty in estimating exact counts from the y-axis in categorical visualizations. By replacing the Seaborn countplot with a custom Matplotlib-based implementation, we gain greater flexibility, including optional count annotations and sorting options. This improves readability, especially for high-cardinality data, and makes the visualizations more suitable for reports and presentations.

The updates also include dependency upgrades, a package version bump, refreshed documentation with examples, updated test cases, and new baseline images to reflect the changes.

Key Changes

New Features in visualize_feature

• Added show_counts Parameter:

  • Type: Boolean (default: True)
  • Functionality: When enabled, displays the exact count values on top of each bar in count plots for categorical, object, boolean, and integer features.
  • This directly resolves the core request from issue Add count labels to bar plots in visualize_feature method #77 by making frequencies explicit without relying on axis scales.
  • For backward compatibility, users can set show_counts=False to revert to the previous behavior.
  • Additional considerations: Counts are formatted with thousand separators (e.g., "742,454") for readability. Font sizes can be adjusted via Matplotlib if needed for dense plots.

• Added order Parameter:

  • Type: String, List of strings, or None (default: None)
  • Functionality: Allows custom sorting of categories in bar plots.
    • String options:
      • "count_desc": Sort by descending count (most frequent first).
      • "count_asc": Sort by ascending count.
      • "alpha_asc": Sort alphabetically in ascending order.
      • "alpha_desc": Sort alphabetically in descending order.
    • List: Provide an explicit list of category names for custom ordering.
  • If None, uses the default index order from value_counts().
  • This feature enhances data exploration by allowing users to focus on high/low frequency items or maintain a specific sequence.

• Added ax Parameter:

  • Type: Matplotlib Axes (optional)
  • Functionality: Allows plotting on a user-provided Axes object for integration into larger figures.

Code Refactoring

• Introduced a new helper function _plot_count_bar in ds_utils/preprocess.py:

  • Handles the creation of bar charts from a pandas.Series of value counts.
  • Supports the order and show_counts logic, including sorting the series and annotating bars with ax.bar_label().
  • Replaces the direct call to sns.countplot, improving maintainability and customization without Seaborn dependencies for this plot.

• Updated visualize_feature to use _plot_count_bar for relevant feature types (categorical, object, boolean, integer).

• Minor adjustments for handling high-cardinality features (e.g., limiting to top 10 categories with a warning, as before).

Documentation Updates

• README.md:
  • Expanded the section on visualize_feature to describe the new parameters (show_counts, order).
• Updated image links to new baselines.
• docs/source/preprocess.rst:
  • Detailed docstring updates for visualize_feature, including parameter descriptions and behavior notes.
  • Added examples for high-cardinality handling and integration with remove_na.
  • Refreshed image references for float, datetime, integer, categorical, and boolean visualizations to show count labels.

Testing Enhancements

• Added parameterized tests in relevant test files:
  • Validate show_counts=True/False outputs.
  • Test various order values, including string options, lists, and invalid inputs (raises ValueError with a clear message).
  • Cover edge cases like high-cardinality data and different feature types.
• Adjusted plot sizes in tests for better visual comparison.
• Updated baseline images to match new plot formats with labels and ordering.
• New tests for _plot_count_bar behavior indirectly through visualize_feature.

Testing and Validation

• All tests pass with the new changes.
• Visual inspections confirm count labels appear correctly and ordering applies as expected.
• Backward compatibility: Setting show_counts=False and order=None reproduces the original behavior.

Related Issue

Closes #77 by adding the requested count labels and extending functionality with ordering options for better usability.

This PR improves the overall utility of the DataScienceUtils package for data exploration and visualization tasks. Feedback welcome!