Skip to content

Exclude index columns for pivot for count/sum agg#8769

Merged
Light2Dark merged 1 commit intomainfrom
sham-fix-polars-test
Mar 19, 2026
Merged

Exclude index columns for pivot for count/sum agg#8769
Light2Dark merged 1 commit intomainfrom
sham-fix-polars-test

Conversation

@Light2Dark
Copy link
Copy Markdown
Collaborator

📝 Summary

This was caught in CI tests.

Fix pivot fill_null(0) casting boolean index columns

handle_pivot applied nw.all().fill_null(0) to every column — including index columns. On boolean index columns this either silently casts the dtype to Int32 or raises an InvalidOperationError depending on the Polars version. Use nw.exclude(*index_columns).fill_null(0) so only the aggregated value columns are filled.

Adds a regression test in test_handlers.py.

🔍 Description of Changes

📋 Checklist

  • I have read the contributor guidelines.
  • For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on Discord, or the community discussions (Please provide a link if applicable).
  • Tests have been added for the changes made.
  • Documentation has been updated where applicable, including docstrings for API changes.
  • Pull request title is a good summary of the changes - it will be used in the release notes.

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 19, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment Mar 19, 2026 8:43am

Request Review

@Light2Dark Light2Dark added the bug Something isn't working label Mar 19, 2026
@Light2Dark Light2Dark marked this pull request as ready for review March 19, 2026 11:24
Copilot AI review requested due to automatic review settings March 19, 2026 11:24
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a pivot transform edge case where fill_null(0) was being applied to pivot index columns, which could cast boolean index columns to integers (or error) in some Polars versions. The handler now fills nulls only for the aggregated value columns, and adds a regression test to prevent dtype regressions.

Changes:

  • Update handle_pivot to apply fill_null(0) only to non-index (aggregated) columns for count/sum.
  • Add a Polars regression test ensuring boolean index column dtypes are preserved for count pivots.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
marimo/_plugins/ui/_impl/dataframes/transforms/handlers.py Excludes pivot index columns from fill_null(0) to avoid dtype casting/errors.
tests/_plugins/ui/_impl/dataframes/test_handlers.py Adds a regression test asserting boolean index dtype is preserved after a count pivot.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Light2Dark Light2Dark merged commit 753f9bf into main Mar 19, 2026
48 of 49 checks passed
@Light2Dark Light2Dark deleted the sham-fix-polars-test branch March 19, 2026 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants