fix(stats): threshold not applied along breakdown dimension in plot.effects (#719)

[FBumann]

Problem

flow_system.stats.plot.effects(by='component', threshold=...) still shows non-invested components. In the reproducer from #719, source S2 never gets invested (zero cost contribution) yet remains in the plot despite threshold=1.0.

Root cause: with by='component'/by='contributor', the breakdown lays entities out along a coordinate dimension of a single data variable (e.g. costs), not as separate data variables. The threshold filter (_filter_small_variables) only dropped whole data variables — costs as a whole is well above threshold, so nothing was removed and the zero-cost S2 survived.

Fix

Generalize the filter to optionally also drop small entries along a coordinate dimension:

• Split into two named helpers — _drop_small_data_vars (whole variables) and _drop_small_along_dim (entries along a coordinate) — so the logic is self-documenting instead of commented.
• The public helper accepts dim: str | list[str] | None; plot.effects passes the active breakdown dimension.
• Renamed _filter_small_variables → _drop_small, since it no longer filters variables only.

Only plot.effects collapses entities into a coordinate dimension; every other threshold call site keeps one entity per data variable, so per-variable filtering there is already correct and is unchanged.

Tests

Added a parametrized regression test (by='component' and by='contributor') building the issue's scenario. It fails on main and passes with this change. threshold=None still keeps the zero-cost entry. Full plotting + comparison suites pass (111 tests).

Fixes #719

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Improved small-value filtering in plots so insignificant values can be removed consistently across all chart types.
  • Effects breakdown charts now handle filtering more accurately when grouped by component or contributor, keeping only meaningful entries.
  • You can now disable this filtering entirely by leaving the threshold unset.

• Tests

  • Added coverage for threshold-based filtering in effects breakdown views, including cases where uninvested items should be hidden or retained.

[coderabbitai]

Warning

Review limit reached

@FBumann, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 25 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 82f78105-7440-4730-83a9-a1af5167661e

📥 Commits

Reviewing files that changed from the base of the PR and between 0097fec and 5ae16ce.

📒 Files selected for processing (2)

• flixopt/statistics_accessor.py
• tests/plotting/test_solution_and_plotting.py

📝 Walkthrough

Walkthrough

The small-value threshold filtering logic in the statistics accessor was refactored into _drop_small_data_vars, _drop_small_along_dim, and a consolidated _drop_small function supporting dimension-level filtering. All plot methods now call _drop_small, and the effects method passes a breakdown dimension when grouping by component or contributor. A test was added to verify this fixes uninvested components not being filtered.

Changes

Threshold filtering refactor

Layer / File(s)	Summary
New drop-small helper functions flixopt/statistics_accessor.py	Adds _drop_small_data_vars and _drop_small_along_dim helpers plus a consolidated _drop_small(ds, threshold, dim=...) wrapper replacing _filter_small_variables.
Plot methods updated to use _drop_small flixopt/statistics_accessor.py	balance, carrier_balance, heatmap, flows, sizes, duration_curve, charge_states, and storage now call _drop_small(ds, threshold) instead of the old filter function.
Dimension-aware effects breakdown filtering and test flixopt/statistics_accessor.py, tests/plotting/test_solution_and_plotting.py	effects passes dim=breakdown_dim to _drop_small when grouping by component or contributor; a new parametrized test verifies uninvested components are dropped at threshold=1.0 and retained at threshold=None.

Estimated code review effort: 2 (Simple) | ~12 minutes

Related issues: #719 — fixes threshold not filtering uninvested components in stats.plot.effects when grouping by component/contributor.

Suggested reviewers: FBumann

Poem

A rabbit hopped through datasets wide,
Small values vanished, nothing to hide,
By component or contributor's name,
Uninvested sources drop from the frame,
Threshold now works — hop, hop, applied!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly states the stats.plot.effects threshold bug fix and references the linked issue.
Description check	✅ Passed	The description covers the problem, fix, tests, and related issue, with only some template checkboxes omitted.
Linked Issues check	✅ Passed	The changes address #719 by applying thresholding along the breakdown dimension and adding a matching regression test.
Out of Scope Changes check	✅ Passed	The PR stays focused on the threshold filtering fix and its regression test, with no obvious unrelated changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/effects-threshold-719

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@flixopt/statistics_accessor.py`:
- Around line 283-302: Both `_drop_small_data_vars` and `_drop_small_along_dim`
currently fall back to returning the original dataset when nothing meets the
threshold, which defeats filtering entirely. Change these helpers so the empty
case returns an empty filtered result instead of `ds` (for example, an empty
selection over `data_vars` or `dim`), while keeping the existing behavior when
some items survive. Add or extend a regression test around the
`effects`/`by='component'` or `by='contributor'` path to cover the
all-below-threshold case and verify nothing is reintroduced.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 38aff483-09c1-4083-80c8-898bd1937cd6

📥 Commits

Reviewing files that changed from the base of the PR and between b1c0d27 and 0097fec.

📒 Files selected for processing (2)

• flixopt/statistics_accessor.py
• tests/plotting/test_solution_and_plotting.py