Conversation
* test(bounds): add optest4 to verify complex bounds with non-trivial time/scenario-dependent values
optest4 is identical to optest1 except minimum_generation_modulation is set
to non-zero constant values (0.5/0.3/0.4) for unique_prod/unique_prod2/unique_prod3,
so the lower-bound expression min(p_max_cluster, min_gen_mod*unit_count*p_max_unit)
actually constrains generation instead of trivially evaluating to 0.
Closes #9
https://claude.ai/code/session_01VoihrWyHoXtpxyBHrwmwCA
* test(bounds): rework optest4 with 2 scenarios and genuine time+scenario-varying bounds
- All scenario-dependent TSV files now have 2 columns (2 scenarios, 168 timesteps)
- minimum_generation_modulation alternates (0.0, 0.5) / (0.3, 0.1) per timestep for
[scenario0, scenario1], making it genuinely time- AND scenario-dependent (non-trivial)
- New test function test_model_behaviour_scenario_and_time_dependent_bounds runs with
scenarios=2, verifying that the min/ceil bound expressions in test_lib.thermal are
correctly handled when the parameter varies across both time and scenarios
- Removes the reference CSV (no longer needed; OPTIMAL status + gap check suffices)
Closes #9
https://claude.ai/code/session_01VoihrWyHoXtpxyBHrwmwCA
* expected results
* add test
* Refactor to vectorized linopy solver and remove OR-Tools dependency (#1)
* refactor: replace OR-Tools pipeline with vectorized linopy solver
Replace the scalar OR-Tools based build_problem pipeline with a fully
vectorized linopy/xarray pipeline. A single AST traversal now produces
linopy LinearExpressions covering all components × time steps × scenarios,
adding all constraints in one add_constraints() call per constraint type.
Key changes:
- Add linopy_linearize.py: VectorizedLinopyBuilder visitor that maps
VariableNodes → linopy.Variable, ParameterNodes → xr.DataArray, and
handles time shifts, time sums, scenario expectations, and port fields
via vectorized xarray/linopy operations.
- Add linopy_problem.py: LinopyOptimizationProblem and build_problem()
using 4-phase construction (params, variables, ports, constraints).
- Fix time_shift for per-component shifts: masked accumulation over
unique shift values replaces scalar extraction (fixes d_min_down≠1).
- Fix _apply_time_shift: assign_coords after isel resets time coordinates
so subsequent xarray arithmetic does not silently re-align.
- Rewrite output_values.py to extract results from linopy_model.solution.
- Remove OR-Tools test files; update all e2e tests to linopy API.
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* chore: delete OR-Tools dead code after linopy migration
Remove all files that were part of the old scalar OR-Tools pipeline and
are no longer referenced by any active code:
Source:
- simulation/optimization.py (929 lines) — OR-Tools OptimizationProblem
- simulation/linearize.py (333 lines) — scalar expression linearizer
- simulation/linear_expression.py (438 lines) — LinearExpression/Term/TermKey
- simulation/benders_decomposed.py — depended on optimization.py; out of scope
Tests:
- tests/unittests/lib_parsing/test_objective_contribution.py — patched
optimization.py internals and used ortools directly
- tests/e2e/functional/test_investment_pathway.py — tested
build_benders_decomposed_problem (out of scope)
- tests/e2e/integration/test_benders_decomposed.py — was already fully
@pytest.mark.skip; benders is out of scope
Dependencies:
- Remove ortools from pyproject.toml dependencies
- Remove [mypy-ortools.*] stanza from mypy.ini
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* style: apply black 23.7.0 formatting to new/modified files
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* fix: resolve mypy errors in linopy_linearize.py and linopy_problem.py
linopy_linearize.py:
- time_shift: split the DataArray check so mypy narrows the type before
passing to _da_to_int; raise ValueError if shift evaluates to a linopy
Variable/LinearExpression (which should never happen for a valid model)
- time_sum slow path: assert both bounds are DataArrays before calling
.astype(int), replacing the unreachable else-int() branches that mypy
flagged as type errors
linopy_problem.py:
- remove unused get_solution(model_id: str, ...) which looked up
_linopy_vars keyed by (int, str) using a str key — wrong type and
logically broken; no callers existed
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* chore: remove dead imports and OR-Tools package dependencies
Python imports:
- simulation/__init__.py: drop re-exports of BendersSolution,
BendersRunner, MergeMPSRunner — nothing imports them anymore
- simulation/output_values.py: remove BendersSolution,
BendersMergedSolution, BendersDecomposedSolution classes (only
used by the deleted benders_decomposed.py) and the now-unused
`import math`
Dependencies:
- requirements.in: remove ortools==9.9.3963 direct dependency
- requirements.txt: remove ortools==9.9.3963 and its transitive-only
dependents absl-py, immutabledict, protobuf; scrub ortools from
comment annotations on numpy and pandas entries
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* docs: rewrite module docstrings to describe current design only
Remove comparative phrasing ("instead of", "replaces OR-Tools") from the
module docstrings of linopy_linearize.py and linopy_problem.py. Both
docstrings now describe the module's own behaviour and structure without
requiring the reader to know any prior implementation.
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* refactor: remove unused params from constraint/objective builders
Drop `components` and `total_obj` from `_create_constraints_for_model`
(neither was used), remove `components` from `_add_objectives_for_model`,
and iterate over `self.model_components.keys()` in the phase-4 build loop.
Closes tbittar/GemsPy#5
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Formatting
* Refacto
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Fix/merge conflict (#25)
* refactor: replace OR-Tools pipeline with vectorized linopy solver
Replace the scalar OR-Tools based build_problem pipeline with a fully
vectorized linopy/xarray pipeline. A single AST traversal now produces
linopy LinearExpressions covering all components × time steps × scenarios,
adding all constraints in one add_constraints() call per constraint type.
Key changes:
- Add linopy_linearize.py: VectorizedLinopyBuilder visitor that maps
VariableNodes → linopy.Variable, ParameterNodes → xr.DataArray, and
handles time shifts, time sums, scenario expectations, and port fields
via vectorized xarray/linopy operations.
- Add linopy_problem.py: LinopyOptimizationProblem and build_problem()
using 4-phase construction (params, variables, ports, constraints).
- Fix time_shift for per-component shifts: masked accumulation over
unique shift values replaces scalar extraction (fixes d_min_down≠1).
- Fix _apply_time_shift: assign_coords after isel resets time coordinates
so subsequent xarray arithmetic does not silently re-align.
- Rewrite output_values.py to extract results from linopy_model.solution.
- Remove OR-Tools test files; update all e2e tests to linopy API.
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* chore: delete OR-Tools dead code after linopy migration
Remove all files that were part of the old scalar OR-Tools pipeline and
are no longer referenced by any active code:
Source:
- simulation/optimization.py (929 lines) — OR-Tools OptimizationProblem
- simulation/linearize.py (333 lines) — scalar expression linearizer
- simulation/linear_expression.py (438 lines) — LinearExpression/Term/TermKey
- simulation/benders_decomposed.py — depended on optimization.py; out of scope
Tests:
- tests/unittests/lib_parsing/test_objective_contribution.py — patched
optimization.py internals and used ortools directly
- tests/e2e/functional/test_investment_pathway.py — tested
build_benders_decomposed_problem (out of scope)
- tests/e2e/integration/test_benders_decomposed.py — was already fully
@pytest.mark.skip; benders is out of scope
Dependencies:
- Remove ortools from pyproject.toml dependencies
- Remove [mypy-ortools.*] stanza from mypy.ini
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* style: apply black 23.7.0 formatting to new/modified files
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* fix: resolve mypy errors in linopy_linearize.py and linopy_problem.py
linopy_linearize.py:
- time_shift: split the DataArray check so mypy narrows the type before
passing to _da_to_int; raise ValueError if shift evaluates to a linopy
Variable/LinearExpression (which should never happen for a valid model)
- time_sum slow path: assert both bounds are DataArrays before calling
.astype(int), replacing the unreachable else-int() branches that mypy
flagged as type errors
linopy_problem.py:
- remove unused get_solution(model_id: str, ...) which looked up
_linopy_vars keyed by (int, str) using a str key — wrong type and
logically broken; no callers existed
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* chore: remove dead imports and OR-Tools package dependencies
Python imports:
- simulation/__init__.py: drop re-exports of BendersSolution,
BendersRunner, MergeMPSRunner — nothing imports them anymore
- simulation/output_values.py: remove BendersSolution,
BendersMergedSolution, BendersDecomposedSolution classes (only
used by the deleted benders_decomposed.py) and the now-unused
`import math`
Dependencies:
- requirements.in: remove ortools==9.9.3963 direct dependency
- requirements.txt: remove ortools==9.9.3963 and its transitive-only
dependents absl-py, immutabledict, protobuf; scrub ortools from
comment annotations on numpy and pandas entries
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* docs: rewrite module docstrings to describe current design only
Remove comparative phrasing ("instead of", "replaces OR-Tools") from the
module docstrings of linopy_linearize.py and linopy_problem.py. Both
docstrings now describe the module's own behaviour and structure without
requiring the reader to know any prior implementation.
https://claude.ai/code/session_01CXGhwjptqV25QGYb56CFdt
* refactor: remove unused params from constraint/objective builders
Drop `components` and `total_obj` from `_create_constraints_for_model`
(neither was used), remove `components` from `_add_objectives_for_model`,
and iterate over `self.model_components.keys()` in the phase-4 build loop.
Closes tbittar/GemsPy#5
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Formatting
* feat: evaluate extra outputs vectorized over xarray (issue #4)
Replace the scalar (time, scenario) loop in extra output evaluation with a
vectorized xarray-based approach, consistent with the Linopy refactoring.
Key changes:
- `extra_output.py`: add VectorizedExtraOutputBuilder (ExpressionVisitor
returning xr.DataArray), supporting all AST nodes including nonlinear ops
(floor/ceil/min/max, var*var) and sum_connections via port arrays;
add _build_port_arrays_xarray / _build_slave_port_array_xarray helpers
that replicate the incidence-matrix logic of _LinopyProblemBuilder but
with post-solve DataArrays; remove ExtraOutputValueProvider (obsolete).
- `linopy_problem.py`: expose param_arrays, model_components and models on
LinopyOptimizationProblem so OutputValues can access them post-solve;
remove expand_operators_for_extra_output (no longer needed).
- `output_values.py`: replace _evaluate_single_extra_output scalar loop with
a model-level vectorized pass using VectorizedExtraOutputBuilder; add
_fill_extra_output_from_da to unpack a DataArray into Dict storage.
- `test_output_values.py`: update mocks for new problem fields; add
test_extra_output_with_sum_connections and test_extra_output_nonlinear.
https://claude.ai/code/session_017jpeajLjMMyPuTeQpvWtBA
* refactor: extract shared port-connection helpers to avoid duplication
_group_port_connections_by_master and _build_incidence_matrix are now
module-level functions in linopy_problem.py. Both
_LinopyProblemBuilder._build_slave_port_array (linopy path) and
_build_slave_port_array_xarray (extra-output path) call them instead of
duplicating the connection-grouping + incidence-matrix logic.
_involves is also no longer copied in extra_output.py (was already in
linopy_problem.py).
https://claude.ai/code/session_017jpeajLjMMyPuTeQpvWtBA
* style: apply black 23.7.0 formatting to linopy_problem.py
https://claude.ai/code/session_017jpeajLjMMyPuTeQpvWtBA
* refactor: import _eval_int, _da_to_int, _has_dim from linopy_linearize
These three helpers were duplicated verbatim from linopy_linearize.py.
Remove the local copies in extra_output.py and import them directly.
Also drop the now-unused EvaluationContext/EvaluationVisitor imports.
https://claude.ai/code/session_017jpeajLjMMyPuTeQpvWtBA
* refactor: unify port-array building via build_port_arrays factory function
Both the linopy problem builder (pre-solve) and the extra-output evaluator
(post-solve) now share a single build_port_arrays(model, components,
all_models, all_model_components, network, make_builder) function in
linopy_problem.py. The caller supplies a make_builder(model_key, model)
factory that returns the appropriate ExpressionVisitor:
- _LinopyProblemBuilder._build_port_arrays_for_model → one-liner calling
build_port_arrays with a VectorizedLinopyBuilder factory.
- OutputValues._evaluate_extra_outputs → calls build_port_arrays with a
VectorizedExtraOutputBuilder factory (lambda capturing var_solution_arrays
and problem after Optional narrowing).
Consequently, _build_port_arrays_xarray and _build_slave_port_array_xarray
are removed from extra_output.py, together with the now-unused imports of
_build_incidence_matrix and _group_port_connections_by_master.
_build_slave_port_array is removed from _LinopyProblemBuilder.
https://claude.ai/code/session_017jpeajLjMMyPuTeQpvWtBA
* refactor: extract shared time-operator logic to module-level functions
_apply_time_shift, _eval_int_expr, _time_shift, _time_eval, _time_sum,
and _all_time_sum are now module-level functions in linopy_linearize.py.
Both VectorizedLinopyBuilder and VectorizedExtraOutputBuilder delegate
their time_* methods to these shared functions, reducing each to a
one-liner. _linopy_add is also moved from linopy_problem.py to
linopy_linearize.py (where LinopyExpression is defined) and imported
back. Using _linopy_add for accumulation in the shared functions makes
them work correctly for both linopy and pure-xarray visitors.
https://claude.ai/code/session_017jpeajLjMMyPuTeQpvWtBA
* Vectorize OutputVariable and ExtraOutput in OutputModel
Replace per-component scalar Dict[TimeScenarioIndex, float] storage with
vectorized xr.DataArray[component, time, scenario] held in a new
OutputModel class (one per GEMS model, covering all its components).
- output_values_base.py: replace BaseOutputValue with two independent
dataclasses OutputVariable and ExtraOutput, each storing an
Optional[xr.DataArray]; remove _value, _size, _set, get.
- extra_output.py: remove ExtraOutput(BaseOutputValue) subclass and its
_set() method; import ExtraOutput from output_values_base.
- output_values.py: introduce OutputModel, ComponentOutputView,
VarOutputView and ExtraOutputView; update OutputValues to use
_models/comp_to_model_key; drop _fill_extra_output_from_da and the
scalar unpacking loops; preserve the component().var().value API for
all 51 existing e2e call sites via thin backward-compat views.
- simulation_table.py: iterate _models DataArrays (isel over component,
time, scenario) instead of _components scalar dicts.
- Tests updated to the new internal API; all 63 tests pass.
https://claude.ai/code/session_017jpeajLjMMyPuTeQpvWtBA
* Refacto
* Formatting
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Added performance tests of optimization problem building with simple problems from standard library and a pypsa-eur based test case (data not included)
* Remove test for time-dependent bounds in model behaviour
Removed the test for model behaviour with time-dependent bounds, which verified complex bound expressions under varying scenarios.
* Build simulation table from linopy problem
* Fix tests
* Remove dead dependencies: antares_craft and scipy
Neither antares_craft nor scipy are imported anywhere in the codebase.
Removes them from pyproject.toml and requirements.in, and cleans up
their transitive-only packages (antares-study-version,
antares-timeseries-generation, psutil, requests, charset-normalizer,
idna, urllib3) from the generated requirements files.
scipy remains as a transitive dep via linopy.
https://claude.ai/code/session_01HZtmQxUDepFWW4TxWBZyv4
* Remove dead code: ProblemVar/Param nodes and operators_expansion pipeline
The scalar expansion pipeline (OperatorsExpansion, ApplyTimeShift, ApplyTimeStep,
ApplyScenario) and its associated AST node types (ProblemVariableNode,
ProblemParameterNode, TimeIndex/ScenarioIndex subclasses) are unused since the
refactorization to vectorized linopy+xarray problem building. Remove them entirely:
- Delete src/gems/expression/operators_expansion.py
- Delete tests/unittests/expressions/visitor/test_operators_expansion.py
- Remove ProblemVariableNode, ProblemParameterNode, problem_var(), problem_param()
from expression.py
- Remove TimeIndex, NoTimeIndex, TimeShift, TimeStep, ScenarioIndex,
NoScenarioIndex, CurrentScenarioIndex, OneScenarioIndex from expression.py
- Remove pb_variable()/pb_parameter() abstract methods from ExpressionVisitor and
their dispatch from visit() in visitor.py
- Remove pb_variable()/pb_parameter() stub/error methods from all visitor
implementations: copy, context_adder, evaluate, indexing, degree, print,
equality, linopy_linearize, extra_output, model/port
https://claude.ai/code/session_019cUZinfENNLYbqw8cVeSTp
* Fix missing and undeclared production dependencies
- Add pandas and attrs to pyproject.toml dependencies (both are imported
in src/ but were absent from the declared deps)
- Add linopy, xarray, highspy, pandas, attrs to requirements.in to sync
it with pyproject.toml
- Remove pandas from requirements-dev.in (it belongs in production deps)
- Add comment on highspy clarifying it is the HiGHS solver backend used
indirectly via linopy
https://claude.ai/code/session_0163cQuKsfWRgmgXkNiRFtbE
* Replace OutputValues with SimulationTable across codebase
Remove the OutputValues class and all its helper classes (OutputModel,
ComponentOutputView, VarOutputView, ExtraOutputView). Migrate all usages
to SimulationTableBuilder.build(problem) which returns a flat pandas DataFrame
with columns: block, component, output, absolute-time-index, block-time-index,
scenario-index, value, basis-status.
- Delete src/gems/simulation/output_values.py
- Remove OutputVariable from output_values_base.py (keep ExtraOutput for extra_output.py)
- Export SimulationTableBuilder and SimulationColumns from gems.simulation.__init__
- Replace OutputValues usage in 9 test files with DataFrame queries
- Migrate extra output unit tests to test_simulation_table_extra_outputs.py
- Update docs/user-guide/outputs.md and AGENTS.md to reflect new API
https://claude.ai/code/session_012R79FcYYAwT4BRo6dAz5Zv
* Added vectorized call of param_data.get_value in _build_param_arrays_for_model in linopy_problem.py to speed up problem building
* Apply black==23.7.0 formatting to modified files
https://claude.ai/code/session_012R79FcYYAwT4BRo6dAz5Zv
* Fix/clear trajectory investment (#38)
* Delete reference to build strategy
* Remove unseless libs
* Remove ComponentParameterNode/ComponentVariableNode and output_values… (#40)
* Remove ComponentParameterNode/ComponentVariableNode and output_values_base.py
Delete ComponentParameterNode, ComponentVariableNode, comp_param(), comp_var()
from the expression AST — these were produced by ContextAdder (also deleted)
which was never called in production. All vectorized-pipeline visitors already
raised ValueError on encountering these nodes.
Cascading cleanup:
- Remove comp_parameter / comp_variable abstract methods and dispatch from
ExpressionVisitor / visit() in visitor.py
- Strip the method pairs and their imports from all 9 concrete visitors:
print, copy, evaluate, evaluate_parameters, equality, degree, indexing,
linopy_linearize, extra_output, port
- Remove get_component_{variable,parameter}_value from ValueProvider /
EvaluationContext (evaluate.py) and get_component_parameter_value from
ParameterValueProvider (evaluate_parameters.py)
- Remove get_component_{variable,parameter}_structure from
IndexingStructureProvider (indexing.py) and the NotImplementedError stubs
in model.py's Provider inner class
- Update 4 test files: drop test_comp_parameter(), remove ComponentParameterNode
from test_copy_ast(), remove comp_param/comp_var parametrize cases
Also inline ExtraOutput from output_values_base.py into extra_output.py
and delete output_values_base.py (its only remaining consumer).
https://claude.ai/code/session_01E1avUHPpjDTp3id3n7zyC5
* Apply black==23.7.0 formatting to modified files
https://claude.ai/code/session_01E1avUHPpjDTp3id3n7zyC5
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: resolve all mypy type errors across 5 files
- data.py: fix DataBase.get_value to wrap timestep in a list and broaden
return type to Union[float, ndarray]; change dataframe_to_time_series
to return pd.Series instead of Dict[TimeIndex, float] to match
TimeSeriesData.time_series field type
- linopy_linearize.py: add return-value to type: ignore on multiplication
(left * right can yield QuadraticExpression)
- linopy_problem.py: suppress spurious ndarray assignment errors on
ScenarioSeriesData and fallback get_value calls inside for-loop
- simulation_table.py: replace unavailable attr.dataclass with
dataclasses.dataclass
- Add CLAUDE.md with project commands and architecture overview
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fixed formatting issues in data.py
* Remove objective_operational_contribution and objective_investment_contribution (#43)
* Enforce model id uniqueness (#44)
* Enforce model id uniqueness
* Remove usage of id(model)
* Use lid_id.model_id (#46)
* fixed issue in linopy_problem.py when use_time and use_scenario are active
* fixed wrong comments anr removed CLAUDE.md
* Remove dead code in gems/expression and gems/simulation
Based on main (strategy.py already removed in #43).
- Delete evaluate_parameters.py (ParameterResolver, ParameterValueProvider,
resolve_parameters) and its test: scalar-parameter-resolution helpers only
used by the old non-vectorised pipeline, replaced by linopy.
- Remove PortOperator abstract base class from port_operator.py: PortAggregator
does not inherit from it and nothing imports it.
- Remove TimestepComponentVariableKey from time_block.py: not exported, not
used anywhere, accompanied by its own TODO comment.
- Add build_time_scalability.csv to .gitignore (benchmark artifact).
https://claude.ai/code/session_01YNdcK2hpLvE1CWyUNXhyvw
* refactor: extract VectorizedBuilderBase to eliminate duplicated visitor logic
VectorizedLinopyBuilder and VectorizedExtraOutputBuilder shared 18 of 19
ExpressionVisitor methods verbatim. This commit extracts the common logic
into an abstract VectorizedBuilderBase[T_expr] in a new module
(vectorized_builder.py), leaving only the single axis of variation:
- VectorizedLinopyBuilder.variable() returns a linopy.Variable (pre-solve)
- VectorizedExtraOutputBuilder.variable() returns an xr.DataArray (post-solve)
VectorizedLinopyBuilder additionally overrides addition() (linopy-aware operand
swap via _linopy_add) and floor/ceil/maximum/minimum (guard: raise when
operands contain linopy types, as nonlinear ops cannot be expressed in LP).
Time operators (time_shift, time_eval, time_sum, all_time_sum) and all
sub-helpers (_apply_time_shift, _eval_int, _da_to_int, _eval_int_expr,
_has_dim) are now private methods of VectorizedBuilderBase, removing the
module-level helper functions from linopy_linearize.py.
LinopyExpression and _linopy_add are re-exported from linopy_linearize.py
so linopy_problem.py requires no import changes.
https://claude.ai/code/session_014b9i1WkuVLBdHuYNZsKkUw
* style: apply black 23.7.0 formatting to vectorized_builder.py
Expand inline abstract method stub `-> T_expr: ...` to two-line form
as required by black 23.7.0 (the project's pinned formatter version).
https://claude.ai/code/session_014b9i1WkuVLBdHuYNZsKkUw
* Fix all mypy type errors across source and test files
- mypy.ini: add ignore_missing_imports for pytest and pydantic
- vectorized_builder.py: add missing error codes (attr-defined, call-overload,
operator) to existing type: ignore comments on generic T_expr operations
- test_data_consistency.py: wrap dict literals with pd.Series() for TimeSeriesData
- test_components_parsing.py: add None guards before len() on Optional fields
- test_simulation_table_mock.py: add Path type and -> None annotation
- test_libs_yaml_system_yaml.py: fix Callable[[str], ...] fixture return type
and add return annotation to inner _setup_test function
- test_scalability.py: add type annotations to build_for_horizon, use pd.Index
- perf_pypsa.py: add full type annotations to all functions, use pd.Index
- test_operators_v1.py: annotate request as pytest.FixtureRequest
Result: mypy src/ tests/ reports 0 errors (was 55 errors in 33 files)
https://claude.ai/code/session_015cPUUeFfm44JNMAWx8J4Pz
* formatting
* Drop Network class, keep System as unified mutable concept
Replace the redundant System (frozen) + Network (mutable) duality with a
single mutable System class. `build_network()` was a pure structural copy
with no transformation, so it is deleted; `resolve_system()` now returns
System directly.
Changes:
- Rename Network → System in network.py (class stays mutable with full
builder/query API)
- Remove frozen System dataclass, system() factory, and build_network()
from resolve_components.py
- resolve_system() returns System; consistency_check() takes System
- Update all callers (main.py, linopy_problem.py, data.py, extra_output.py)
- Update all test files; rename test_network.py → test_system.py
https://claude.ai/code/session_011m8dvniijsWARLHGYcfZ7d
* Apply black 23.7.0 formatting to resolve_components.py
https://claude.ai/code/session_011m8dvniijsWARLHGYcfZ7d
* fix: resolve mypy override errors by removing T_expr generic from VectorizedBuilderBase
Replace the unconstrained TypeVar T_expr with the concrete VectorizedExpr union
(Union[xr.DataArray, linopy.LinearExpression, linopy.Variable]), making
VectorizedBuilderBase non-generic. The covariant overrides of literal() and
parameter() returning xr.DataArray are now valid since DataArray is a member of
the union. LinopyExpression is kept as a backward-compatible alias.
https://claude.ai/code/session_011FoLmLbTXiGJzUEePFHJe6
* added test for systems with varying time shifts and generators with different down time constraints
* fixed import sort
* formatting
* added missing demand time series file for tests
* typing
* formatting
* fix: make VectorizedBuilderBase generic with T_expr bound to VectorizedExpr
Introduces `T_expr = TypeVar("T_expr", bound=VectorizedExpr)` and changes
`VectorizedBuilderBase` to inherit from both `ExpressionVisitor[VectorizedExpr]`
and `Generic[T_expr]`. This makes the class generic while keeping the visitor
interface concrete (all dispatched methods return `VectorizedExpr`).
The `T_expr` parameter narrows two things:
- `port_arrays: Dict[PortFieldId, T_expr]`
- `variable() -> T_expr` (covariant override of `-> VectorizedExpr`)
`VectorizedExtraOutputBuilder(VectorizedBuilderBase[xr.DataArray])` now
correctly expresses that it only manipulates `xr.DataArray`.
`VectorizedLinopyBuilder` is updated to `VectorizedBuilderBase[VectorizedExpr]`.
https://claude.ai/code/session_012aNnBbnfndxD3c37EFAK4a
* fix: cast visit() result to xr.DataArray in simulation_table.py
VectorizedBuilderBase inherits ExpressionVisitor[VectorizedExpr] (not
ExpressionVisitor[T_expr]), so visit() is inferred to return the broad
Union type. VectorizedExtraOutputBuilder is designed to always produce
xr.DataArray, so cast() is semantically correct and resolves the mypy
[assignment] error.
https://claude.ai/code/session_01Bn8ST7WGkNeKvfwuZYei8e
* Fix isort, black (23.7.0), and mypy type errors
- isort: fix import ordering in main.py, resolve_components.py, __init__.py
- black: reformat expression/evaluate.py, indexing.py, visitor.py, simulation/linopy_problem.py
- mypy: fix Dict[int, Model]/Dict[int, List[Component]]/Callable[[int, Model], Any]
annotations in build_port_arrays and _build_slave_port_array — model IDs are str, not int
https://claude.ai/code/session_01Q4JaUjVBEuWBku1k6g6J46
* Fix isort in tests/ directory
CI runs isort on both src/ and tests/ — previous commit only covered src/.
Fix import ordering in 16 test files (System sorted after Node/PortRef).
https://claude.ai/code/session_01Q4JaUjVBEuWBku1k6g6J46
* Fix black formatting: use black 23.7.0 (not 26.x)
Previous run used the wrong black binary (26.3.1 in PATH instead of
the pip-installed 23.7.0). Black 26.x keeps abstract stub methods as
one-liners (def f(): ...) while 23.7.0 expands them to two lines.
Also fixes type annotation wrapping in linopy_problem.py.
https://claude.ai/code/session_01Q4JaUjVBEuWBku1k6g6J46
* Rename network→system: fix terminology across codebase
- Fix docstrings/comments in network.py: replace "network" with "system"
and "links" with "connections" in module/class/node docstrings and
error messages
- Rename `network` parameter and attribute to `system` in
linopy_problem.py (build_port_arrays, _build_slave_port_array,
LinopyOptimizationProblem, _LinopyProblemBuilder, build_problem),
simulation_table.py, and data.py
- Fix broken Network import in extra_output.py (class no longer exists)
- Fix broken Network/build_network imports in perf_pypsa.py and
test_component_dependent_time_shift.py; replace with System
- Rename `network` local variables to `system` in all test files
https://claude.ai/code/session_0165bvEfgmMvk6zKgYhYX8BK
* Fix black 23.7.0 formatting (revert black 26.x style changes)
Black 26.x reformatted abstract method stubs to single-line form
(def foo(self) -> T: ...) and changed Dict assignment parenthesization.
Black 23.7.0 (required by CI) wants the two-line form instead.
https://claude.ai/code/session_0165bvEfgmMvk6zKgYhYX8BK
* fix: replace undefined Network type hint with System in test_varying_down_time
Network was never imported; System is already imported from gems.study.network
and is the correct type used throughout the file.
https://claude.ai/code/session_01ELCgeS6xjZmW5j3XzhBWFJ
* remove perf tests
* Add optim-config for decomposition (#53)
* Add optim-config for decomposition
* Add test
* Write correct structure
* Add checks
* Refacto structure writer
* Refacto parsing
* Remove if continue
* Remove useless file
* feat: Add study reader to load and run simulations from a folder
This commit introduces the `gems.study.folder` module, which provides an API to easily load and execute simulation studies directly from a directory structure.
Key additions:
- `load_study(study_dir)`: Reads the system definition (`input/system.yml`), model libraries (`input/model-libraries/*.yml`), and data series (`input/data-series/`). It resolves the components, performs consistency checks, and builds the simulation network and database.
- `run_study(study_dir, scenarios, time_block)`: A convenience wrapper that loads a study, builds the optimization problem, and solves it.
- Added corresponding E2E tests (`test_study_from_folder.py`) and mock study data (under `studies/7_4/`) to verify the new folder-loading functionality.
This provides a cleaner interface for running studies, aligning with the goal of providing an API similar to C++.
* Remove stale Network references, replace with System
Network no longer exists in the codebase; all references now use System.
- parsing.py: swap Network TYPE_CHECKING import → System, rename param
- linopy_problem.py: rename network param → system in build_decomposed_problems
- main.py: import System, replace undefined `network` variable with `study`
Fixes 8 mypy errors across 3 files.
https://claude.ai/code/session_01HEkeCSF4N6EiACPvBGhika
* Apply black 23.7 formatting
Reformat 5 files to comply with black 23.7 style:
- Collapse abstract stub methods to single-line (def foo(): ...)
- Reformat long dict type annotation in linopy_problem.py
https://claude.ai/code/session_01HEkeCSF4N6EiACPvBGhika
* merge of main and fix of path provided to load_optim_config
* Fix black formatting for abstract method stubs and type annotation
Convert inline ellipsis stubs (`def method() -> T: ...`) to the
two-line format required by black 23.7.0, and reformat a long type
annotation in linopy_problem.py.
https://claude.ai/code/session_01XXoMCKcpcNfymrixzeGwQQ
* fixed formatting and typing issues
* Fix stale build_network import in e2e test
Remove the deleted build_network import and rename study/network
variables to system, reflecting the refactor that dropped the Network
class in favour of System as the unified mutable concept.
https://claude.ai/code/session_018DXQwxDJkVYqnSAFkzcbGx
* Delete Node class and fully deprecate nodes as a distinct concept
Node was an empty frozen dataclass subclassing Component with no added
fields or methods. This removes the Node/Component distinction entirely:
- Remove Node class, create_node(), and node-specific System methods
(add_node, get_node, _nodes dict, nodes property) from network.py
- Remove Node from the public API (__init__.py)
- Remove nodes field from InputSystem (parsing.py)
- Simplify resolve_components.py to add former nodes as plain Components
- Move nodes: sections in all YAML system files into components:
- Update all test files to use Component instead of Node
https://claude.ai/code/session_01LFjwykkbWfw5Z89Y2c2QcW
* Fix isort: expand single-line gems.study imports to multi-line
https://claude.ai/code/session_01LFjwykkbWfw5Z89Y2c2QcW
* Add SimulationTable fluent accessor API (component / output / value)
Introduces SimulationTable, ComponentView and OutputView wrapper classes
so that simulation results can be accessed via a readable fluent API:
st.component("gen_1").output("p").value() # Time × Scenario DataFrame
st.component("gen_1").output("p").value(scenario_index=0) # Series over time
st.component("gen_1").output("p").value(time_index=3) # Series over scenarios
st.component("gen_1").output("p").value(time_index=3, scenario_index=1) # float
SimulationTableBuilder.build() now returns SimulationTable instead of a
raw pd.DataFrame; SimulationTableWriter updated accordingly. All 9 test
files that used verbose boolean-indexing patterns are migrated to the new
API and a dedicated unit-test suite is added.
https://claude.ai/code/session_01UnxHZrTk6sqDqW8aySxDJw
* Add uv.lock generated by adding pytest dev dependency
https://claude.ai/code/session_01UnxHZrTk6sqDqW8aySxDJw
* isort'
* Refactor load_study to use System instead of Network
* Refactor output file generation with timestamp
* Fix black formatting for CI
Apply black formatting to 5 files with lines exceeding the line length limit and a missing blank line.
https://claude.ai/code/session_01JVq1eB9VnaeHwxVqJ4PriB
* Fix mypy arg-type error for float() call on pandas .loc scalar
Cast the pandas scalar to Any before passing to float() so mypy
accepts it. pandas-stubs types .loc scalars as a broad union that
includes Timestamp/Timedelta/complex, which are not in float()'s
accepted SupportsFloat | SupportsIndex set.
https://claude.ai/code/session_01MBjviMu9XToiypdR8v2UcY
* Module cleaning: rename network.py to system.py, delete probability_law.py
- Rename src/gems/study/network.py → system.py (preserving git history)
- Update all import references from gems.study.network → gems.study.system
- Rename local variable `network` → `system` in folder.py and tests
- Update comments/variable names in optim_config/parsing.py
- Delete src/gems/model/probability_law.py (unused stub)
Closes #74
https://claude.ai/code/session_012yzcLueLUDoiF3Fnw9XDLU
* Add unit tests for VectorizedLinopyBuilder (closes #15)
69 focused, isolated tests covering every method of VectorizedLinopyBuilder
and its base class: leaf nodes, overridden arithmetic/nonlinear guards,
_linopy_add swap logic, time operators (shift/eval/sum), scenario operators,
port field lookup, and private helpers. All error paths (KeyError,
NotImplementedError) are verified, including boundary conditions such as
cyclic time wrap-around, shift by block_length, out-of-bound eval, and
linopy type guards on floor/ceil/max/min.
https://claude.ai/code/session_01V2XghYSMCxndqAC1diRK9q
* Fix isort import ordering in perf_pypsa.py and test_libs_yaml_system_yaml.py
Move `from gems.study.system import System` after `gems.study.resolve_components`
to match alphabetical (isort/black profile) ordering.
https://claude.ai/code/session_012yzcLueLUDoiF3Fnw9XDLU
* Apply black formatting to fix CI
https://claude.ai/code/session_01V2XghYSMCxndqAC1diRK9q
* Fix SimulationTable.to_csv mypy error in folder.py
Access the underlying DataFrame via .data before calling to_csv,
consistent with how SimulationTableWriter already does it.
https://claude.ai/code/session_01CCigNV7CZLfWa494aoftPQ
* refactor: rename linopy-specific files and classes (issue #82)
- linopy_problem.py → optimization.py
- linopy_linearize.py → linearize.py
- LinopyOptimizationProblem → OptimizationProblem
- _LinopyProblemBuilder → _OptimizationProblemBuilder
- VectorizedLinopyBuilder → VectorizedLinearExprBuilder
- test_vectorized_linopy_builder.py → test_vectorized_linear_expr_builder.py
https://claude.ai/code/session_01LpavdwbPoU27KorGpVCcyb
* style: fix black formatting for long function signatures
https://claude.ai/code/session_01LpavdwbPoU27KorGpVCcyb
* Rename components_input to system_input in examples (fixes #73)
Replace the variable name `components_input` with `system_input` in
documentation code blocks and e2e test files. The variable holds the
result of `resolve_system()` which returns a `System` object, so
`system_input` better reflects its purpose.
https://claude.ai/code/session_01EQRcsQub1W3Wx7L4qrQ8bh
* Vectorize SimulationTableBuilder and add Parquet/NetCDF export (Issue #30)
- Replace row-by-row _da_to_rows() with vectorized _da_to_df() using NumPy
index arrays; use pd.concat instead of building a list of dicts, eliminating
Python loops over (component × time × scenario)
- Add SimulationTable.to_dataset() returning an xr.Dataset where each output
variable is a DataArray(component, time, scenario) and scalar rows
(e.g. objective-value) are stored as zero-dimensional variables
- Add SimulationTableWriter.write_parquet() and .write_netcdf() export methods
- Refactor run_study() in folder.py to delegate CSV export to SimulationTableWriter
- Add test_simulation_table_export.py covering to_dataset() and both new writers
- Update test_simulation_table_mock.py with _to_object_dtype() helper to handle
Arrow vs object dtype null representation differences across pandas versions
https://claude.ai/code/session_01JJCDwYa9gfEUxdRX3xLEzt
* Rename _collect_solver_outputs to _collect_vars_outputs
https://claude.ai/code/session_01JJCDwYa9gfEUxdRX3xLEzt
* Use None for missing time/scenario indices in _da_to_df
When a variable or extra output lacks a time or scenario dimension, the
corresponding index columns (absolute-time-index, block-time-index,
scenario-index) are now set to None instead of a synthetic 0. This
signals that the output is independent of that dimension, consistent
with how objective-value rows are already represented.
Add three new tests covering time-independent, scenario-independent,
and fully scalar outputs.
https://claude.ai/code/session_01JJCDwYa9gfEUxdRX3xLEzt
* Fix isort: remove extra blank line after imports in test_simulation_table_export.py
https://claude.ai/code/session_01JJCDwYa9gfEUxdRX3xLEzt
* Apply black formatting fixes across 4 files
https://claude.ai/code/session_01JJCDwYa9gfEUxdRX3xLEzt
* Fix two CI failures: KeyError in accessor API and missing pyarrow
Fix 1 — accessor KeyError for dimension-independent outputs:
ComponentView.output() now calls .copy() on the filtered slice then
fills None values in the time and scenario index columns with 0 before
pivoting. pandas drops NaN from the pivot column axis, producing an
empty index and a subsequent KeyError when value(scenario_index=0) is
called on investment variables (e.g. p_max, nb_units) that have no
scenario dimension. The .data property is unaffected — None values are
preserved there. Add two new accessor tests for scenario-independent
and fully-scalar outputs.
Fix 2 — missing pyarrow in CI:
Add pyarrow to requirements-dev.in / requirements-dev.txt so the CI
install step picks it up. Guard all three parquet tests with
pytest.importorskip("pyarrow") as a belt-and-suspenders measure for
environments that still lack the package.
https://claude.ai/code/session_01JJCDwYa9gfEUxdRX3xLEzt
* Apply black formatting to test_simulation_table_accessor.py
https://claude.ai/code/session_01JJCDwYa9gfEUxdRX3xLEzt
* Introduce Study container class to gather System and DataBase (closes #78)
`System` (component topology) and `DataBase` (parameter values) were always
passed together as a pair throughout the codebase. This commit introduces a
`Study` dataclass that holds both as attributes and centralises the
cross-validation logic previously scattered across `DataBase.requirements_consistency`
and its call sites.
Changes:
- Add `gems.study.study.Study` dataclass with `check_consistency()` method
- Remove `DataBase.requirements_consistency(system)` (logic moved to `Study`)
- Update `build_problem()` and `build_decomposed_problems()` to accept `Study`
- Update `load_study()` to return `Study` instead of `tuple[System, DataBase]`
- Rename `input_study()` → `input_system()` in `main.py` to avoid name clash
- Update all call sites in tests (61 build_problem, 2 build_decomposed_problems,
8 requirements_consistency, 1 load_study unpack) and documentation
https://claude.ai/code/session_01Wmj8XYdCk1EPKCeGs7dHAq
* Fix mypy and isort issues in main.py and folder.py
- Restore return type annotations on input_database() and input_system()
- Re-order imports in folder.py and main.py to satisfy isort
https://claude.ai/code/session_01Wmj8XYdCk1EPKCeGs7dHAq
* Fix isort ordering in all updated test files
The bulk import-insertion script placed Study at the end of import blocks
and mangled the closing parenthesis. Apply isort --profile black to restore
correct alphabetical order and formatting across all affected test files.
https://claude.ai/code/session_01Wmj8XYdCk1EPKCeGs7dHAq
* Remove extra blank line at end of data.py to satisfy black
https://claude.ai/code/session_01Wmj8XYdCk1EPKCeGs7dHAq
* Refactor tests to use Study instead of System
* Refactor input system and library handling in tests
* Update Agents file
* fix module name in ci cov (#89)
* Update
* Remove problem context
* Refactor OptimizationProblem to take Study instead of System+Database
- OptimizationProblem.__init__ now accepts study: Study instead of
separate system: System, database: DataBase, model_components, and
models arguments. system, database, model_components, and models are
exposed as read-only properties delegating to self.study.
- _OptimizationProblemBuilder similarly takes study: Study; the manual
component-grouping loop is removed in favour of study.model_components
and study.models.
- Study gains two cached_property attributes: models (Dict[str, Model])
and model_components (Dict[str, List[Component]]), computed once from
system.all_components.
- build_problem() and build_decomposed_problems() pass study directly to
the builder instead of unpacking study.system / study.database.
No changes required in simulation_table.py, couplings.py, or test files
since all previously public attributes remain accessible as properties.
https://claude.ai/code/session_01SdZ3bcTZmSuuNYR4k8vvcb
* Tolerate absence of expec() in objective contributions (Issue #76) (#95)
* Tolerate absence of expec() in objective contributions (Issue #76)
When an objective contribution has a residual scenario dimension but
no explicit expec() operator, the model() factory now automatically
wraps the expression with expec() (average-over-scenarios semantics)
and emits a UserWarning pointing authors to add the explicit form.
Expressions that are already fully scalar, or already wrapped in
expec(), are left unchanged with no warning. Time dimension errors
are still rejected.
This achieves iso-format with Antares Simulator v10.0.0.
https://claude.ai/code/session_01SDhHG2UhKm1wjvhDKxKXLu
* Add TODO note: auto-wrapping of expec() is a temporary shim
Per review comment: clarify in the docstring that
_normalize_objective_contributions is a temporary compatibility measure
until Antares Simulator natively supports the expec() operator.
https://claude.ai/code/session_01SDhHG2UhKm1wjvhDKxKXLu
* Reduce provider duplication
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Thomas Bittar <thomas.bittar@rte-france.com>
* Further refactor
* remove useless attributes from OptimizationProblem
* formatting
* fix: add FakeStudy to test stubs so FakeProblem exposes problem.study
simulation_table.py now accesses problem.study.model_components and
problem.study.models, but the FakeProblem stubs in three test files
only exposed those as direct attributes. Add FakeStudy dataclass and
a study field to FakeProblem in each test file, then pass it from
every factory/inline constructor.
https://claude.ai/code/session_013e7p8jzM9WY74X7aYJqByK
* refactor: rename Input[X] Pydantic classes to [X]Schema (issue #75) (#98)
* refactor: rename Input[X] Pydantic classes to [X]Schema (issue #75)
Adopts the [X]Schema naming convention (e.g. ParameterSchema,
ComponentSchema) in place of the former Input[X] pattern across all
source and test files. The Schema suffix is standard in Pydantic/FastAPI
projects and immediately signals that these classes are validation
schemas for external YAML input, distinct from the domain objects they
produce.
https://claude.ai/code/session_01N7Zs1WhkuRpZuR3snk594N
* style: wrap long line in ModelSchema to satisfy black
https://claude.ai/code/session_01N7Zs1WhkuRpZuR3snk594N
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: add SimulationSession with new resolution modes
- Add ResolutionConfig to optim-config with FRONTAL, SEQUENTIAL_SUBPROBLEMS,
PARALLEL_SUBPROBLEMS and BENDERS_DECOMPOSITION modes (replaces flat
resolution_mode field); horizon required for windowed modes
- Add gems.session module with SimulationSession dataclass and load_session
factory; scenario_ids: List[int] drives per-scenario vs joint LP dispatch
- Add scenario_ids_remap to SimulationTableBuilder.build() so scenario indices
in the output table reflect actual IDs rather than 0-based linopy positions
- Add merge_simulation_tables() utility function
- Add initial_values carry-over constraints to build_problem / builder
- Simplify main_cli() to delegate resolution to SimulationSession
- Fix folder.load_study() spurious Warning raise when optim-config is present
https://claude.ai/code/session_01HTa79fVcbZmsLaNLXdG6ay
* style: fix black 23.7.0 formatting in optimization.py
Reformat per_master type annotation to match black 23.7.0's style.
https://claude.ai/code/session_01HTa79fVcbZmsLaNLXdG6ay
* feat: add ScenarioBuilder, propagate scenario_group, fix scenario data selection
Introduce ScenarioBuilder (pass-through placeholder) that maps MC scenario IDs
to time-series column indices per component scenario_group. The scenario
selection is now passed as List[int] all the way down to the LP builder so
that sequential and parallel runners actually fetch the correct database column
for each MC scenario instead of always using column 0.
Key changes:
- New ScenarioBuilder.resolve(scenario_group, scenario_id) -> int (identity)
- Component gains scenario_group: Optional[str]; propagated from ComponentSchema
- Study gains scenario_builder: ScenarioBuilder (loaded from scenariobuilder.dat)
- build_problem / build_decomposed_problems: scenarios: int -> scenario_ids: List[int]
- _build_param_arrays_for_model uses scenario_builder.resolve per (component, scenario)
- SimulationSession._run_block passes scenario_ids list directly
- All callers updated: list(range(N)) at each former build_problem(study, block, N)
https://claude.ai/code/session_01HTa79fVcbZmsLaNLXdG6ay
* refactor: MAP returns (problem, SimulationTable); fix absolute_time_offset
SimulationTableBuilder.build() now derives absolute_time_offset from
problem.block.timesteps[0] instead of (block.id - 1) * block_size.
The old formula was wrong for block_id=0 (frontal, sequential) and
for overlapping windows.
SimulationSession._run_block (MAP) now returns (OptimizationProblem,
SimulationTable): the table is built and the scenario index remap is
applied immediately. _reduce is simplified to a plain merge of already-
built SimulationTables. Problems are kept in lists alongside tables so
they remain accessible for carry-over extraction (sequential) and
post-run inspection.
FakeBlock in unit tests gains a timesteps attribute to match the updated
SimulationTableBuilder.build() interface.
https://claude.ai/code/session_01HTa79fVcbZmsLaNLXdG6ay
* Refactor session.py to simplify table handling
* fix: add timesteps to FakeBlock in test_simulation_table_export
FakeBlock was missing the timesteps attribute introduced when
absolute_time_offset was changed to use problem.block.timesteps[0].
Aligns with the same fix already applied to test_simulation_table_mock
and test_simulation_table_accessor.
https://claude.ai/code/session_01HTa79fVcbZmsLaNLXdG6ay
* Update session.py
* Add unit test for SimulationTable correctness on partial TimeBlock (issue #103)
- Add `lib_dict_unittest` fixture to conftest.py that loads and resolves
lib_unittest.yml, making its models available for Python-based tests.
- Add test_simtable_timeblock.py with test_simtable_on_partial_timeblock:
system of node + generator (pmax=200) + demand (demand[t]=t, 150 steps),
solved on TimeBlock [40, 90). Verifies that SimulationTable reports
absolute-time-index 40–89, block-time-index 0–49, generation values
equal to the absolute timestep, and the correct objective (3225).
https://claude.ai/code/session_01KjvZr5ATEFct69WuWYVN3s
* Apply black formatting to test_simtable_timeblock.py
https://claude.ai/code/session_01KjvZr5ATEFct69WuWYVN3s
* Add LinopyModel alias to distinguish from gems.model.Model
Closes #107. Introduces `LinopyModel = linopy.Model` in
`gems.simulation.optimization` and re-exports it from
`gems.simulation`, so callers can reference the solver-level
model without importing linopy directly and without confusing it
with the GEMS component model.
https://claude.ai/code/session_01EB5vY5goa2w4hnrUREVucx
* refactor: rename horizon/overlap/total_timesteps to clearer names
- ResolutionConfig.horizon → block_length (length of one optimization window)
- ResolutionConfig.overlap → block_overlap (timesteps shared between consecutive windows)
- SimulationSession.total_timesteps → study_length (overall study duration)
- load_session parameter renamed accordingly
block_length is now consistent with OptimizationProblem.block_length,
which already used that name for the same concept.
https://claude.ai/code/session_01HTa79fVcbZmsLaNLXdG6ay
* avoid silent error
* formatting
* more explicit vars
* remove redundant scenarios var
* remove redundant scenarios var
* remove scenarios count
* fix docstring
* remove scenarios count
* fix dim size
* Parse binary variable
* Feature/non cyclic constraints (#92)
* Support for out-of-bounds-processing
* Remove BlockBorderManagement
* Handle parametrized shifts within sums
* Cleaner implementation
* Remove unrelevant description
* Remove unused imports
* Remove unused imports
* Fix test
* Raise with time or scenario dependency in shift
* Harmonize the interactions between CLI, main.py, folder.py, SimulationSession, SimulationTable. (#110)
* Harmonize CLI/folder/session/table interactions (issue #106) + parameters.yml
- New src/gems/study/parameters.py: StudyParameters pydantic model reading
first-time-step, last-time-step, nb-scenarios, solver, solver-logs,
solver-parameters from study_dir/parameters.yml (optional, defaults to
frontal/highs/1-scenario if absent)
- folder.py: remove dead optim-config ref in load_study; rewrite run_study
to delegate entirely to SimulationSession via load_parameters, returns None
and auto-exports SimulationTable to study_dir/output/
- session.py: add first_timestep, solver_name, solver_logs, solver_parameters
fields; fix _run_frontal/_run_sequential/_run_parallel to start from
first_timestep; stamp run_id as table_id on all produced SimulationTables
- simulation_table.py: add table_id to SimulationTable; add to_csv/to_parquet/
to_netcdf methods with auto-naming from table_id; remove SimulationTableWriter
- runner.py: rename CommandRunner -> AntaresXpansionCommandRunner; add module
docstring clarifying this wraps AntaresXpansion external binaries
- parsing.py: simplify CLI to --study (required) + --optim-config (optional);
remove --duration and --scenario (now driven by parameters.yml)
- main.py: delegate main_cli to run_study; keep input_libs/input_system/
input_database/_write_structure_txt for E2E test compatibility
- Update tests to use new SimulationTable export API (to_csv/to_parquet/
to_netcdf) instead of deleted SimulationTableWriter
https://claude.ai/code/session_01HyPLW7cBK5kugdX8Tqi9YE
* Apply black 23.7.0 and isort 5.12.0 formatting
Fix formatting issues found by CI linters (same versions as remote repo).
https://claude.ai/code/session_01HyPLW7cBK5kugdX8Tqi9YE
* Fix mypy call-arg errors in StudyParameters
Remove redundant explicit alias= from Field() calls — ModifiedBaseModel
already applies _to_kebab via alias_generator, so re-declaring the same
hyphenated aliases caused the pydantic mypy plugin to treat the fields as
required even though they had defaults.
https://claude.ai/code/session_01HyPLW7cBK5kugdX8Tqi9YE
* Fix circular import between folder.py and session.py
folder.py imports SimulationSession from session.py at module level,
and session.py imported load_study from folder.py at module level,
creating a cycle. Move the load_study import inside load_session()
where it is used.
https://claude.ai/code/session_01HyPLW7cBK5kugdX8Tqi9YE
* Fix test_run_study and related issues for new run_study API
- StudyParameters: override extra="ignore" so unknown fields in
parameters.yml (e.g. no-output, export-mps from other tools) are
silently dropped instead of raising a validation error.
- folder.py: remove kwargs that SimulationSession does not accept
(first_timestep, solver_name, solver_logs, solver_parameters).
- test_study_from_folder: update test_run_study to call the new
run_study(study_dir) signature (no TimeBlock or scenario count args),
copy the study to tmp_path to avoid polluting the source tree, and
assert the output CSV is created with an objective-value row instead
of inspecting a returned OptimizationProblem.
https://claude.ai/code/session_01HyPLW7cBK5kugdX8Tqi9YE
* Remove low-level helpers comment section
Removed commented section for low-level helpers.
* harmonizing time
* Merge parameters.yml into optim-config: add time-scope, solver-options, scenario-scope sections
- Add TimeScopeConfig (start-timestep / end-timestep), SolverOptionsConfig
(solver, solver-logs, solver-parameters) and ScenarioScopeConfig
(nb-scenarios) to OptimConfig in optim_config/parsing.py.
- Update session.py to read time range from optim_config.time_scope and
solver settings from optim_config.solver_options; remove the now-redundant
solver_name / solver_logs / solver_parameters fields from SimulationSession.
- Update folder.py to drop parameters.yml loading entirely; scenario_ids are
now derived from optim_config.scenario_scope.
- Delete study/parameters.py (StudyParameters / load_parameters).
- Migrate all test parameters.yml values into the corresponding
input/optim-config.yml files and delete the parameters.yml files.
https://claude.ai/code/session_01G29xWf8E1XyzkMWQ7YPYXo
* style: apply black formatting to session.py
https://claude.ai/code/session_01G29xWf8E1XyzkMWQ7YPYXo
* Compute scenario_ids from optim_config instead of passing as constructor arg
scenario_ids is now a property on SimulationSession derived from
optim_config.scenario_scope.nb_scenarios, removing it as a required
constructor parameter. load_session() drops its scenario_ids argument
for the same reason.
https://claude.ai/code/session_01QaaRTupB2WQ528NDgKs4Mh
* Add e2e consistency test for frontal, parallel, and sequential resolution modes
Tests that run_study with three different optim-config files (frontal, parallel-
subproblems with block-length=168, and sequential-subproblems with block-length=168
and block-overlap=1) produces identical per-timestep simulation table values for
a fully time-separable LP problem (andromede_v1 DSR study, base028 variant without
thermal clusters).
The study uses 504 timesteps (end-timestep=503). Parallel mode runs 3 blocks of
168 timesteps; sequential mode also uses block-length=168 and block-overlap=1,
which produces 4 blocks (3 full + 1 partial). Duplicate rows from the sequential
overlap are deduplicated before comparison. The frontal vs parallel test also
asserts that the summed block objectives are equal.
Closes #105.
https://claude.ai/code/session_01TQZE7z7wJfDDq794Te37tR
* Add simple_generator LP model and restore gas/oil/coal to dsr_3_blocks study
Replaces the MIP antares-historic.thermal model with a new continuous LP
model (simple_generator) in the local andromede_v1_models library, and
adds gas_base_zone, oil_base_zone, and coal_base_zone to the test system.
This keeps the test system realistic (thermal dispatch in merit order) while
avoiding MIP degeneracy: with zero startup/fixed costs the MIP solver finds
multiple equivalent integer solutions across block boundaries, making
per-timestep comparison between resolution modes unreliable.
https://claude.ai/code/session_01TQZE7z7wJfDDq794Te37tR
* Merge study libraries into test_lib and simplify test_optim_modes
- Replace andromede_v1_models.yml + antares_historic.yml with a single
test_lib.yml (id: test-lib) containing only the 5 models used by the
dsr_3_blocks study: area, load, renewable, dsr, simple_generator
- Update system.yml model references to use test-lib.*
- Remove _DEGENERATE_OUTPUTS and the integer-variable comment from
test_optim_modes.py; simplify _per_timestep_df accordingly
https://claude.ai/code/session_01TQZE7z7wJfDDq794Te37tR
* Update system.yml model references to test-lib
Follows the library consolidation: replace antares-historic.* and
andromede-v1-models.* prefixes with test-lib.* throughout.
https://claude.ai/code/session_01TQZE7z7wJfDDq794Te37tR
* Apply black 23.7 formatting to test_optim_modes
https://claude.ai/code/session_01TQZE7z7wJfDDq794Te37tR
* Rename timescope keys: start/end-timestep → first/last-time-step
https://claude.ai/code/session_01WWT1fjYx11X3uUbzuXY3NT
* Rename solver-options fields: solver->name, solver_logs->logs, solver_parameters->parameters (#122)
https://claude.ai/code/session_01DJAihUc9MLUGXimU25Rnu3
Co-authored-by: Claude <noreply@anthropic.com>
* Remove load_session function from session.py
Removed the load_session function and its associated docstring.
* Remove load_session from __all__ exports
* refactor: move run_study to runner.py to eliminate circular dependency (#123)
* refactor: move run_study to runner.py to eliminate circular dependency
folder.py was importing SimulationSession from session.py, which in turn
needed to import load_study from folder.py (worked around with a local
import). Moving run_study to a new runner.py breaks the cycle: folder.py
now only handles loading, session.py can import load_study at module level,
and runner.py owns the orchestration between the two.
https://claude.ai/code/session_01GnAgJKNP5b8D8SzH1TXRMU
* style: apply black formatting to runner.py
https://claude.ai/code/session_01GnAgJKNP5b8D8SzH1TXRMU
* fix: update test imports to use gems.study.runner for run_study
https://claude.ai/code/session_01GnAgJKNP5b8D8SzH1TXRMU
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Remove load_session function to resolve circular dependency
Removed the load_session function to fix circular dependency issues.
* feat: use timestamp run_id and per-run output_dir in run_study (#124)
The session now receives a run_id (minute-granularity timestamp, e.g.
20260427T1430) and an output_dir of study_dir/output/{run_id}/, so each
run is isolated in its own subdirectory. Results are written via
session.output_dir instead of a hardcoded study_dir/output path.
https://claude.ai/code/session_01ShR8EaCPWNrspYqLNFfV4X
Co-authored-by: Claude <noreply@anthropic.com>
* fix: resolve mypy type error for Path | None passed to to_csv (#125)
Extract output_dir as a local Path variable before constructing the
SimulationSession so the non-optional Path is passed directly to
table.to_csv() instead of session.output_dir (typed Optional[Path]).
https://claude.ai/code/session_014B2YPGb79WZ9boNhFfaaKV
Co-authored-by: Claude <noreply@anthropic.com>
* fix: update output globs to search subdirectories after per-run output_dir change (#127)
runner.py now writes to output/{run_id}/simulation_table_*.csv (introduced in
#124), but the e2e tests were still globbing the flat output/ directory.
Switch to a recursive glob (**/) so tests find the file regardless of depth.
https://claude.ai/code/session_01PXs3nz8srGDfQPGHG8xp5m
Co-authored-by: Claude <noreply@anthropic.com>
* Integrate PR 92 (non-cyclic constraints) into gemspy-issue-106-strategy (#128)
* Integrate PR 92 (non-cyclic constraints) into gemspy-issue-106-strategy
Source changes:
- Rename ValueType.BOOLEAN → BINARY in model/common.py and variable.py
- Remove BlockBorderManagement enum from optimization.py and __init__.py;
replace with OutOfBoundsFilter that reads per-constraint mode from
optim-config out-of-bounds-processing section (cyclic is the implicit
default when a constraint is not listed)
- Add ShiftValidityVisitor and _ShiftAmountEvaluator to vectorized_builder.py
for computing per-(component, time) DROP validity masks
- Add export_lp() helper to OptimizationProblem
- Update build_problem() and build_decomposed_problems() signatures:
drop border_management param, accept optional optim_config instead
- Add OutOfBoundsMode / OutOfBoundsConstraintConfig /
OutOfBoundsProcessingConfig to optim_config/parsing.py alongside the
existing TimeScopeConfig / SolverOptionsConfig / ScenarioScopeConfig;
add _check_oob_constraint_ids validation; extend validate_optim_config
to check both model_decomposition and out_of_bounds_processing
Test / study changes:
- Drop border_management=BlockBorderManagement.CYCLE from all test call
sites (test_component_dependent_time_shift, test_libs_*, poc)
- Add test_out_of_bounds_processing.py and test_shift_validity_visitor.py
from PR 92
- Add 4 new study directories from PR 92 (simple_system_cyclic,
simple_system_drop, system_cyclic_with_param_in_shift,
system_drop_with_param_in_shift); update their optim-config.yml to
include time-scope (first-time-step: 0, last-time-step: 2)
https://claude.ai/code/session_01KdDTbQgHkM7DrB9nEZ8ZvR
* Delete out-of-bounds processing documentation
Removed out-of-bounds processing section from the documentation.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Fix mypy errors: duplicate function definition and duplicate keyword argument (#129)
- Remove duplicate `_check_oob_constraint_ids` definition in parsing.py (line 297 was identical to line 181)
- Remove duplicate `oob_filter` keyword argument in optimization.py `_OptimizationProblemBuilder` call
https://claude.ai/code/session_01YHkEmwVVZmYh5aZ4KJZitP
Co-authored-by: Claude <noreply@anthropic.com>
* Refactor simulation table writing in tests (remove TableWriter)
* Update study folder docstring
Removed optional optim-config.yml description from docstring.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Reorganize tests (#119)
Co-authored-by: OUSTRY Antoine <aoustry@gmail.com>
* Fix description, remove imports (#131)
* Add E2E test for rolling-horizon suboptimality (Issue #102) and fix s… (#132)
* Add E2E test for rolling-horizon suboptimality (Issue #102) and fix session optim_config propagation
Adds a new E2E test that demonstrat…
aoustry
approved these changes
Apr 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR is a major refactoring of GemsPy, touching nearly every layer of the codebase. The headline changes are:
xr.DataArrayoperations instead of scalar loops. OR-Tools and the oldoperators_expansionpipeline are removed.gems/study/package provides a folder-based entry point (similar to the C++ API):load_study(path)reads YAML system/optim-config files, resolves components, and returns a runnableSimulationSession.SimulationSession— replaces the ad-hoc runner wiring; it owns the full lifecycle of a simulation run and can be constructed either programmatically or via the study reader.SimulationTable— replacesOutputValuesacross the codebase. Exposes a fluent accessor API (.component(…).output(…).value(…)) for post-processing results, and is built directly from the linopy problem.Network→System— theNetworkclass is dropped in favour ofSystemas the single unified, mutable concept for describing the power system. All references (including tests and docs) are updated.Node,ComponentParameterNode,ComponentVariableNode, and the broader nodes abstraction are fully deleted; the expression layer now works directly on model objects.What was removed
antares_craft/scipydependenciesNetworkclassSystemNode/ComponentParameterNode/ComponentVariableNodeOutputValues/output_values.pySimulationTableProblemVar/ProblemParamexpression nodesoperators_expansionpipelineprobability_law.pymoduleTests
optest4— verifies complex variable bounds (min,ceil) with non-trivial runtime/scenario-dependent parameter values, across 2 scenarios x 168 timesteps.VectorizedLinopyBuilder.