Skip to content

feat: parallelize slice_dims aggregation #12

Description

@FBumann

Motivation

When using slice_dims, each slice runs an independent tsam.aggregate() call. These are embarrassingly parallel — no shared state between slices. tsam already implements file-based parallelization in find_pareto_front() (via n_jobs parameter, no pickling needed), so we can follow the same pattern.

Proposed API

result = tsam_xarray.aggregate(
    da,
    n_clusters=8,
    time_dim="time",
    stack_dims=["variable", "region"],
    slice_dims=["scenario"],
    n_jobs=-1,  # all CPUs, or int for specific count
)

Implementation

  • Add n_jobs: int | None = None parameter to aggregate()
  • None or 1 = sequential (current behavior)
  • n_jobs > 1 or -1 = parallel via concurrent.futures or joblib
  • Each slice is independent — no shared state, trivial to parallelize
  • Follow tsam's approach: file-based parallelization avoids pickling issues
  • Same pattern applies to find_optimal_combination() and find_pareto_front() wrappers (feat: wrap tuning functions (find_optimal_combination, find_pareto_front) #10)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions