Skip to content

feat(scoped-sd): wire cleaned_* into merged_sd + Codex P2 fix#789

Merged
paddymul merged 2 commits into
mainfrom
feat/scoped-sd-followups
May 21, 2026
Merged

feat(scoped-sd): wire cleaned_* into merged_sd + Codex P2 fix#789
paddymul merged 2 commits into
mainfrom
feat/scoped-sd-followups

Conversation

@paddymul
Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #785. Wires the cleaned_* scope into merged_sd, fixes the
broken filter_active gate (which was on key-inequality and mislabelled
cleaning-affected stats as filtered), and applies Codex's P2 fix so
mutating analysis_klasses invalidates the scope cache.

Changes

  • _merged_sd gates on chain-shape: cleaning_active = chains['clean'] != chains['raw'], filter_active = chains['filt'] != chains['clean']. Layers cleaned_* keys between raw bare keys and filtered_*.
  • _scope_cache_key includes analysis_klasses identity in the extra arg. A klasses swap now invalidates cached scope SDs.

Tests

  • Failing-tests commit (one CI run with red): four new tests covering cleaned_* emission, the cleaning-only no-filtered_* gate, both-scopes-present, and analysis_klasses cache-key invalidation.
  • Fix commit: the dataflow changes. The previously-failing test_cleaning_only_does_not_emit_filtered_keys (the gate-bug pin from the failing-tests commit) now passes naturally.

Closes the deferred items documented in plans/0785-cleaning-scope-known-issues.md and plans/0785-codex-p2-analysis-klasses.md.

The analysis_klasses test is asserted at the _scope_cache_key level rather than end-to-end through merged_sd because analysis_klasses is currently a plain class attribute (not a traitlet) on DataFlow — setting it on the instance doesn't fire the @observe('analysis_klasses') callbacks. The cache-key contract is the load-bearing invariant; making analysis_klasses observable is an unrelated architectural change.

[Claude Code]

paddymul added 2 commits May 21, 2026 15:49
…ation

Adds four tests pinning the deferred items on #785:

- ``test_cleaned_keys_appear_when_cleaning_active`` — clean scope SD
  must be layered into ``merged_sd`` with a ``cleaned_*`` prefix when
  cleaning ops are active.
- ``test_cleaning_only_does_not_emit_filtered_keys`` — the broken
  ``filter_active`` gate currently mislabels cleaning-affected stats
  as ``filtered_*`` when no search filter is active. The right gate is
  on chain-shape diff (``filt != clean``).
- ``test_filter_and_clean_both_emit_correctly`` — both scopes layered
  without cross-talk; ``cleaned_null_count`` reflects the clean df,
  ``filtered_null_count`` reflects the search-nulled df.
- ``test_analysis_klasses_change_invalidates_scoped_sd`` — Codex P2
  pin: a swap of ``analysis_klasses`` must invalidate the per-scope
  SD cache so the new stat klass's keys surface in ``merged_sd``.

All four are expected to fail on this commit; fixes follow.
…Codex P2

A. ``_merged_sd``: reads the clean scope's SD from the keyed cache and
   layers ``cleaned_*`` keys between the raw bare keys and the
   ``filtered_*`` keys. Adds ``clean_sd_key`` to the observed set so the
   observer fires after ``_populate_sd_cache`` has updated the pointer.

   Replaces the broken ``filter_active = filt_sd_key != raw_sd_key``
   gate (which fired whenever cleaning was active, mislabelling
   cleaning-affected stats as ``filtered_*``) with chain-shape diffs:

       chains = split_chain_by_scope(self.operations)
       cleaning_active = chains['clean'] != chains['raw']
       filter_active   = chains['filt']  != chains['clean']

   ``filtered_*`` now only fires when the filt chain extends the clean
   chain — i.e. a real quick-command op was added.

B. ``_scope_cache_key``: includes ``id(self.analysis_klasses)`` in the
   ``extra`` arg passed to ``hash_chain``. Pins Codex P2: a klass-list
   swap with an unchanged op chain now produces a distinct cache key,
   so the scope SD is recomputed against the new klasses.

The previously-failing ``test_cleaning_only_does_not_emit_filtered_keys``
(the gate-bug pin from the failing-tests commit) now passes naturally;
no test changes in this commit.
@paddymul paddymul force-pushed the feat/scoped-sd-followups branch from ea865c6 to 8faea19 Compare May 21, 2026 19:49
@paddymul paddymul changed the base branch from feat/scoped-sd-merged-prefix-v2 to main May 21, 2026 19:49
@github-actions
Copy link
Copy Markdown
Contributor

📦 TestPyPI package published

pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.14.3.dev26249360817

or with uv:

uv pip install --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo==0.14.3.dev26249360817

MCP server for Claude Code

claude mcp add buckaroo-table -- uvx --from "buckaroo[mcp]==0.14.3.dev26249360817" --index-strategy unsafe-best-match --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ buckaroo-table

📖 Docs preview

🎨 Storybook preview

@paddymul paddymul added this pull request to the merge queue May 21, 2026
Merged via the queue into main with commit 76743bf May 21, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triaged Reviewed and triaged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant