Skip to content

feat(styling): init_sd as augmentation channel (nested merge + delete_keys + demo)#748

Closed
paddymul wants to merge 9 commits into
mainfrom
feat/init-sd-augmentation-channel
Closed

feat(styling): init_sd as augmentation channel (nested merge + delete_keys + demo)#748
paddymul wants to merge 9 commits into
mainfrom
feat/init-sd-augmentation-channel

Conversation

@paddymul
Copy link
Copy Markdown
Collaborator

Summary

Turns init_sd into a proper augmentation channel for per-column display config — one that plays nice with lowcode ops (Search → highlight) instead of clobbering them like column_config_overrides does. Two style_column changes + a delete-keys escape hatch + a working demo notebook.

Depends on

#744 (jlisp 2-tuple transform contract) and #745 (polars Search → highlight_regex via sd). The test test_init_sd_displayer_args_and_search_highlight_coexist_on_same_column references Search's contribution shape; the notebook demo relies on the styling layer reading highlight_* from merged_sd.

After #744 and #745 merge, this PR will rebase and the duplicate commits drop via patch-id.

Pieces

  1. feat(styling): merge column_metadata displayer_args + ag_grid_specs — two shallow merges in DefaultMainStyling.style_column. When column_metadata carries displayer_args (e.g. {'max_length': 5000}) or ag_grid_specs (e.g. {'wrapText': True, 'width': 400}), they merge into the styled bag instead of replacing. Caller wins per-key. Critically, this lets a Search op's highlight_regex and a user's max_length coexist on the same column — column_config_overrides's replace semantics would have clobbered the highlight.

  2. feat(styling): init_sd delete_keyscolumn_metadata.delete_keys: list[str] pops the named top-level keys from the final base_config. Motivating case: drop the auto-attached tooltip_config on a string column the user doesn't want a permanent tooltip on, without subclassing the styling analysis. Operates on top-level only (tooltip_config, ag_grid_specs, displayer_args, col_name) — nested removal isn't needed for the motivating case.

  3. Restaurant-Complaints demo notebook — working example combining init_sd (nested displayer_args + ag_grid_specs + delete_keys) with extra_grid_config (rowHeight, pinnedRowHeight) and a viol_status == 'Fail' filter on the polars Enum.

Usage

init_sd = {
    'comments': {
        'displayer_args': {'displayer': 'string', 'max_length': 5000},
        'ag_grid_specs': {'wrapText': True, 'width': 400, 'maxWidth': 800},
        'delete_keys': ['tooltip_config'],
    },
}
PolarsBuckarooInfiniteWidget(df, init_sd=init_sd, ...)

This shape is intentionally the same as column_config_overrides would take — but init_sd augments via merge_sds (each key wins per-call), while column_config_overrides replaces via shallow row.update().

Test plan

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 16, 2026

📦 TestPyPI package published

pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.13.5.dev25992032577

or with uv:

uv pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.13.5.dev25992032577

MCP server for Claude Code

claude mcp add buckaroo-table -- uvx --from "buckaroo[mcp]==0.13.5.dev25992032577" --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo-table

📖 Docs preview

🎨 Storybook preview

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1721611fcf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread buckaroo/customizations/polars_commands.py
paddymul and others added 9 commits May 17, 2026 09:20
Lets a Command thread styling-relevant metadata alongside its df result,
without a separate channel through the dataflow. configure_buckaroo wraps
each registered primitive: if a transform returns a 2-tuple ending in a
dict, the second element is captured in a per-call sd_accumulator and only
the df flows to the lisp interpreter. buckaroo_transform exposes
get_last_sd_updates() for the caller; autocleaning's handle_ops_and_clean
merges the snapshot into cleaning_sd via merge_sds.

Single-return transforms (all existing ones) are unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ing _type

The rest of buckaroo's sd is keyed by internal a/b/c column names (the
analysis classes run against a renamed df), but a Command's transform sees
only the original column names. After collecting (df, sd_updates) from
the interpreter, autocleaning now rewrites the op-supplied keys via the
cleaned_df's column→letter mapping so the updates merge into the existing
sd entries instead of sitting alongside as orphans.

Belt-and-suspenders: DefaultMainStyling.style_column falls back to obj
when `_type` is missing, rather than KeyError-ing. This shouldn't trigger
in normal flow now that the rename is correct, but it stops a noisy
warning if an op contributes a column not present in summary_sd.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Search.transform now returns (filtered_df, {string_col: {'highlight_regex': val}})
so the search term flows into cleaning_sd through the interpreter's 2-tuple
return contract (jlisp). DefaultMainStyling.style_column picks up
highlight_phrase / highlight_regex / highlight_color from col_meta and
copies them into the string displayer_args, which is already understood
by the JS-side string displayer.

No changes to dataflow.py or the widget — the lowcode op contributes
styling metadata via the sd merge, which the styling layer reads natively.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two widget-level tests pinning both delivery paths:

- Search op → cleaning_sd → merged_sd → renamed col_meta → string
  displayer_args.highlight_regex on every polars-String column (and
  not on integer columns).
- column_config_overrides with highlight_phrase + highlight_color
  survives the merge_column_config step intact.

Both inspect df_display_args['main']['df_viewer_config']['column_config']
— the same structure that ships to the JS-side renderer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
init_sd users can already carry the same nested shape they'd put in
column_config_overrides — e.g. {'comments': {'displayer_args': {'max_length':
2000}, 'ag_grid_specs': {'wrapText': True, 'width': 400}}}. But where
column_config_overrides REPLACES the styled bag via shallow row.update(),
init_sd needs to AUGMENT it so the op-supplied highlights from Search (which
contribute flat top-level keys onto the same column_metadata) coexist with
the user's per-column displayer config.

Two shallow merges in DefaultMainStyling.style_column:

  - column_metadata['displayer_args'] (dict) merges into the styled disp
    bag — caller wins per-key. Keeps highlight_regex / highlight_phrase /
    highlight_color (read separately from flat top-level keys) intact.
  - column_metadata['ag_grid_specs'] merges into base_config['ag_grid_specs']
    — caller wins per-key, on top of styling's computed minWidth.

This is what makes init_sd a viable augmentation channel for per-column
display config that needs to play nice with lowcode ops.

Tests:
  - test_style_column_merges_nested_displayer_args_and_ag_grid_specs:
    nested displayer_args.max_length and ag_grid_specs.{wrapText,width,
    maxWidth} all flow through, minWidth still computed.
  - test_init_sd_displayer_args_and_search_highlight_coexist_on_same_column:
    the actual point — Search's highlight_regex AND user's max_length both
    end up on the same column's displayer_args after merge_sds.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
style_column's type dispatch unconditionally attaches tooltip_config to
string / time / categorical / period / interval / binary / fallback
columns. init_sd's nested-merge channel can augment displayer_args and
ag_grid_specs, but had no way to *remove* a styled default — so a user
who wanted a string column without the permanent tooltip had no path
short of subclassing the styling analysis.

delete_keys is a top-level list on column_metadata; after all merging,
each listed key is popped from base_config. Operates on top-level keys
only (tooltip_config, ag_grid_specs, displayer_args, col_name) — nested
removal isn't needed for the motivating case and would complicate the
merge semantics.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Working example of init_sd carrying displayer_args + ag_grid_specs +
delete_keys (drops the auto-attached tooltip_config) on the comments
column. Pairs with rowHeight: 105 and wrapText so long complaint text
wraps inside a tall cell.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add df.filter(pl.col('viol_status') == 'Fail') to demonstrate the
typical "filter + view" flow with the live-search widget. Tighten
rowHeight to 70px (was 105) — five lines was overkill for the median
comment length. Note in a comment that the Polars Enum is case-sensitive
('Fail' not 'fail') — previously committed version used 'fail' which
silently filters to 0 rows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@paddymul paddymul force-pushed the feat/init-sd-augmentation-channel branch from b7cb29d to 7e84cbf Compare May 17, 2026 13:21
@paddymul
Copy link
Copy Markdown
Collaborator Author

Resume note (2026-05-17): what's left to salvage from this branch after #758 lands.

Main + #758 already cover the first 4 commits here (older parallel impl of the SDResult/highlight plumbing — d9983945, 48458e4a, e7e25d03, cddbf99d are superseded by #755's SDResult class on main plus #758's de38f9c3 + f57289f4). #749 is unrelated (widget remount work).

Net-new content worth keeping:

  1. 39ca7046 feat(styling): merge column_metadata displayer_args + ag_grid_specsstyle_column shallow-merges column_metadata['displayer_args'] into computed disp, and column_metadata['ag_grid_specs'] over {minWidth}. Caller wins per-key. Makes init_sd an augmentation channel rather than a replacement.
  2. 31ac848e feat(styling): init_sd delete_keys — top-level list on column_metadata; each listed key (e.g. tooltip_config) popped from base_config after merge. Lets init_sd drop styling-added defaults.
  3. ffa6e088 + b4b17c3a docs: Restaurant-Complaints.ipynb — working demo of init_sd with displayer_args + ag_grid_specs + delete_keys (and Polars Enum case fix 'fail''Fail').
  4. Tests for (1) and (2) in autocleaning_pl_test.py: test_style_column_merges_nested_displayer_args_and_ag_grid_specs, test_init_sd_displayer_args_and_search_highlight_coexist_on_same_column.
  5. 7e84cbf5 is formatting-only.

Plan for resumption: after #758 merges, rebase this branch and drop the four superseded commits — only the styling.py augmentation features + their tests + the demo notebook should remain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant