Skip to content

B2: port concat_summaries feature set onto collect options (#706) - #736

Merged
jepegit merged 1 commit into
masterfrom
706-collect-summary-pipeline
Jul 26, 2026
Merged

B2: port concat_summaries feature set onto collect options (#706)#736
jepegit merged 1 commit into
masterfrom
706-collect-summary-pipeline

Conversation

@jepegit

@jepegit jepegit commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Epic B, arc B2 (#706) — the collectors feature port

Carries the meaty three feature families of the legacy helpers.concat_summaries onto the SummaryOptions model built in B1, polars-native and using the cell's schema headers (the legacy headers_summary / headers_step_table singletons are deprecated-for-removal in 2.1).

Feature families

  • Rate filtering — keep only cycles whose rate_on step ran at the requested C-rate (rate / rate_on / rate_std / rate_column / rate_inverse / rate_inverted). Ported from select_summary_based_on_rate as a polars step mask returning surviving cycles.
  • Group averaging — average per journal group into a tidy long frame (group, cycle_num, variable, mean, std). Ported from _make_average + collect_frames; keeps the "<2 cells per group disables grouping" guard and names the aggregate mean regardless of average_method (backward-compat).
  • CV partition — split each numeric metric into *_non_cv / *_cv via make_summary(exclude_step_types=["cv_"]). Ported from _partition_summary_based_on_cv_steps; column selection expands to keep the derived split.

Plus max_cycle / remove_last / only_selected cycle selection, normalized (equivalent) cycle exposure, and inf/extreme cleanup.

Layout

  • collect/_summary_ops.py — per-cell polars ops (rate mask, CV partition, extraction, cleanup, group average).
  • collect/summary.py — orchestration: per-cell extract → combine → select → group-average → cleanup → transforms.
  • collect/options.pySummaryOptions grown to the full pipeline, one source of truth (replaces ~30 concat_summaries kwargs).

Tests

tests/test_collect.py — golden tests for max_cycle, rate filter (+ inverted), CV columns, and group averaging (mean/std values checked). Full local run: 52 passed, 5 planned xfails (B3/B4 collectors placeholders).

Closes #706

…706)

Carry the meaty three feature families of the legacy
helpers.concat_summaries onto SummaryOptions / collect_summaries,
polars-native and using the cell's schema headers (the legacy
headers_summary / headers_step_table are deprecated-for-removal in 2.1):

- rate filtering: keep only cycles whose rate_on step ran at the
  requested C-rate (rate/rate_on/rate_std/rate_column/inverse/inverted),
  ported from select_summary_based_on_rate as a polars step mask.
- group averaging: average per journal group into a tidy long frame
  (group, cycle_num, variable, mean, std), ported from _make_average +
  collect_frames; keeps the "<2 cells per group disables grouping" guard
  and names the aggregate "mean" regardless of average_method.
- CV partition: split each numeric metric into *_non_cv / *_cv via
  make_summary(exclude_step_types=["cv_"]), ported from
  _partition_summary_based_on_cv_steps; column selection expands to keep
  the derived split.

Plus max_cycle / remove_last / only_selected cycle selection, normalized
(equivalent) cycle exposure, and inf/extreme cleanup. Per-cell cycle work
lives in collect/_summary_ops.py; combination, selection, averaging and
cleanup orchestrate in collect/summary.py. Adds golden tests for each.

Closes #706

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 26, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@jepegit
jepegit enabled auto-merge (squash) July 26, 2026 22:38
@jepegit
jepegit merged commit 6855293 into master Jul 26, 2026
5 of 6 checks passed
@jepegit
jepegit deleted the 706-collect-summary-pipeline branch July 26, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

B2: collectors — options + pipeline (fixes cross-cell bug)

1 participant