GenX.jl v0.4.0

Note: user guide "Porting a module from GenX v0.3.6 to GenX v0.4.0" available at this link.

What's Changed

This present version of GenX.jl through this new release introduces a few new modules and technologies. It also introduces a new version of the folder structure for case studies and a reformatted version of the input files for demand, reserves and resources. The documentation pages have also been updated to reflect these new changes as well as we have a much more logically consistent and user-friendly structure of organization of the documentation pages. We hope, this will cater better and more effectively, all kinds of users, whether developers/contributors, or energy modelers.

Added

• Feature CO2 and fuel module (#536)
  Adds a fuel module which enables modeling of fuel usage via (1) a constant heat rate and (2)
  piecewise-linear approximation of heat rate curves.
  Adds a CO2 module that determines the CO2 emissions based on fuel consumption, CO2 capture
  fraction, and whether the feedstock is biomass.
• Enable thermal power plants to burn multiple fuels (#586)
• Feature electrolysis basic (#525)
  Adds hydrogen electrolyzer model which enables the addition of hydrogen electrolyzer
  demands along with optional clean supply constraints.
• Add ability of storage to contribute to capacity reserves (#475)
• Add Co-located VRE+Storage Module (#523)
• Add output for dual of capacity constraint (#473)
• Add PR template (#516)
• Validation ensures that resource flags (THERM, HYDRO, LDS etc) are self-consistent (#513).
• Maintenance formulation for thermal-commit plants (#556).
• Add new tests for GenX: three-zone, multi-stage, electrolyzer, VRE+storage,
  piecewise_fuel+CO2, and TDR (#563 and #578).
• Added a DC OPF method (#543) to calculate power flows across all lines
• Added write_operating_reserve_price_revenue.jl to compute annual operating reserve and regulation revenue.
  and added the operating reserve and regulation revenue to net revenue (#611)
• Add functions to compute conflicting constraints when model is infeasible if supported by the solver (#624).
• New settings parameter, VirtualChargeDischargeCost to test script and VREStor example case. The PR 608 attempts to
  introduce this parameter as cost of virtual charging and discharging to avoid unusual results (#608).
• New settings parameter, StorageVirtualDischarge, to turn storage virtual charging and discharging off if desired by the user (#638).
• Add module to retrofit existing resources with new technologies (#600).
• Formatted the code and added a format check to the CI pipeline (#673).
• Add check when capital recovery period is zero and investment costs are non-zero in multi-stage GenX (#666)
• Added condition number scaling added to objective function (#667)
• Added versioned doc-pages for v0.3.6 and v0.4.0
• Added a warning message in write_costs_multistage mentioning the approximate value of costs currently. (#680)

Fixed

• Reserves have been renamed as OperationReserves in order to distinguish it from Capacity Reserve Margin (#641)
• Generators_data.csv is not split into multiple different input files grouped together according to different resources (#612)
• Assign correct investment cost per stage in objective function initialization for multi-stage runs (#530)
• Fix name of Fixed_OM_Cost_Charge_per_MWyr when reading from Generators_data in multistage code. (#533)
  Previously there was a typo in this in the multistage code that led to a silent bug, which affects outputs,
  for anyone running non-myopic multistage GenX with asymmetric storage.
• Fix computation of cumulative minimum capacity retirements in multistage GenX (#514)
• Fix access of eELOSSByZone expr before initialization (#541)
• Correctly write unmet reserves (in reg_dn.csv) (#575)
• Correctly scale total reserves column (in reg_dn.csv) (#594)
• Add validation for Reg_Max and Rsv_Max columns in Generators_data.csv when MUST_RUN is set to 1 (#576)
• Fix scaling of transmission losses in write_transmission_losses.jl (#621)
• Fix cost assignment to virtual storage charge/discharge - issue #604 (#608)
• Fix modeling of hydro reservoir with long duration storage (#572).
• Fix update of starting transmission capacity in multistage GenX
• Fix write_status with UCommit = WriteShadowPrices = 1 (#645)
• Fixed outputting capital recovery cost to 0 if the remaining number of years is 0 (#666)
• Updated the docstring for the initialize_cost_to_go function and adjusted the formula for the discount factor to reflect the code
  implementation (#672).
• Fix write_multi_stage_cost.jl: add discount with OPEX multipliers to cUnmetPolicyPenalty (#679)
• Fix DF calculation in DDP to make it more generic for variable length stages (#680)
• Fix write_power_balance.jl: add additional two columns ("VRE_Storage_Discharge" and "VRE_Storage_Charge") for VRE_STOR

Changed

• Use add_to_expression! instead of the += and -= operators for memory performance improvements (#498).
• Generally, 'demand' is now used where (electrical) 'load' was used previously (#397).
• Load_data.csv is being renamed to Demand_data.csv. (#397)
• Load_MW_z* columns in that file are renamed to Demand_MW_z*. (#397)
• In Reserves.csv, Reg_Req_Percent_Load and Rsv_Req_Percent_Load are renamed to ..._Demand. (#397)
• Load and LoadWeight keys in the time_domain_reduction_settings.yml file are renamed to Demand, DemandWeight. (#397)
• The New_Build column in Generators_data.csv has been separated into two: New_Build and Can_Retire (#392).
  Values in each column are {0,1}.
• Separate proprietary JuMP solvers from the GenX package.
  This allows users of Gurobi or CPLEX to use them without modifying
  the source code of the GenX package directly. This is a key step in publishing
  GenX as a proper Julia package. This does require change to the Run.jl files,
  to specify the solver. (#531)
• In the examples, change Reg_Max and Rsv_Max of any MUST_RUN generators to 0.
  This mitigates but does not fully fix (#576).
• Expressions of virtual charging and discharging costs in storage_all.jl and vre_stor.jl
• The input file Generators_data.csv has been split into different files, one for each type of generator.
  The new files are: Thermal.csv, Hydro.csv, Vre.csv, Storage.csv, Flex_demand.csv, Must_run.csv,
  Electrolyzer.csv, and Vre_stor.csv. The examples have been updated, and new tests have been added to
  check the new data format (#612).
• The settings parameter Reserves has been renamed to OperationalReserves, Reserves.csv to
  Operational_reserves.csv, and the .jl files contain the word reserves have been renamed to
  operational_reserves (#641).
• New folder structure for a GenX case. The input files are now organized in the following folders: settings,
  policies, resources and system. The examples and tests have been updated to reflect this change.
• New folder structure implemented for example_system. This folder now consists of nine separate folders each pertaining to a different
  case study example, ranging from the ISONE three zones, with singlestage, multistage, electrolyzers, all the way to the 9 bus IEEE
  case for running DC-OPF.
• Pruned HiGHS solver settings to the necessary minimum (#668)
• Changed deploydoc URL to GenX.jl (#662)

Deprecated

• The above load keys, which generally refer to electrical demand, are being deprecated.
  Users should update their case files.
  For now this is done in a backward-compatible manner, and @info reminders are written to the log to prompt the user to update.
  "Load" now typically refers only to the transferrence of data from files to memory,
  except for a few places such as the common term "value of lost load" which refers to non-served demand (#397).
• New_Build = -1 in Generators_data.csv: instead, use New_Build = 0 and Can_Retire = 0.
• The matrix-style input of the grid for Network.csv is deprecated in favor a column-style input.
  Instead of columns z1, z2, ... with entries -1, 0, 1, use two columns: Start_Zone, End_Zone (#591).

Full Changelog: v0.3.6...v0.4.0-rc

Full List of Changes from GenX v0.3.6 to prerelease v0.4.0-rc

• Add output for dual of capacity constraint by @gmantegna in #473
• Remove Network.csv files from single-zone cases by @cfe316 in #505
• Fix error on write_costs re: COMMIT_ZONE #506 by @cfe316 in #508
• Feature issue template by @cfe316 in #511
• add PR template by @gmantegna in #516
• Deduplicate 'multi-stage' overview docs by @cfe316 in #515
• Return path from write_outputs by @cfe316 in #509
• Set example small_hydro plants MUST_RUN=1 by @cfe316 in #517
• Validate resources in generators_data.csv by @cfe316 in #513
• Remove coveralls and coverage by @cfe316 in #518
• Fix typo in logs by @gmantegna in #521
• Update PULL_REQUEST_TEMPLATE.md by @gmantegna in #520
• Fix bug in reserves by @gmantegna in #524
• add_to_expression! by @pshlime in #498
• Add ability for storage to contribute to capacity reserves without needing to generate by @gmantegna in h...

GenX Patch Version v0.3.7

What's Changed

Fixed

• write_costs: add check on COMMIT_ZONE set (#507)
• Access to eELOSSByZone expr before initialization (#569)
• Scaling of transmission losses in tlosses.csv (#622)
• Correct tracking of current investment stage by SDDP (#628)
• Computation of cumulative minimum capacity retirements in multistage GenX (#631)
• Modeling of hydro reservoir with long duration storage (#632)
• Update of starting transmission capacity in multistage GenX (#633)

Full Changelog: v0.3.6...v0.3.7

GenX v0.4.0-rc

Note: user guide "Porting a module from GenX v0.3.6 to GenX v0.4.0" available at this link.

What's Changed

Added

• Feature CO2 and fuel module (#536)
  Adds a fuel module which enables modeling of fuel usage via (1) a constant heat rate and (2)
  piecewise-linear approximation of heat rate curves.
  Adds a CO2 module that determines the CO2 emissions based on fuel consumption, CO2 capture
  fraction, and whether the feedstock is biomass.
• Enable thermal power plants to burn multiple fuels (#586)
• Feature electrolysis basic (#525)
  Adds hydrogen electrolyzer model which enables the addition of hydrogen electrolyzer
  demands along with optional clean supply constraints.
• Add ability of storage to contribute to capacity reserves (#475)
• Add Co-located VRE+Storage Module (#523)
• Add output for dual of capacity constraint (#473)
• Add PR template (#516)
• Validation ensures that resource flags (THERM, HYDRO, LDS etc) are self-consistent (#513).
• Maintenance formulation for thermal-commit plants (#556).
• Add new tests for GenX: three-zone, multi-stage, electrolyzer, VRE+storage,
  piecewise_fuel+CO2, and TDR (#563 and #578).
• Added a DC OPF method (#543) to calculate power flows across all lines
• Added write_operating_reserve_price_revenue.jl to compute annual operating reserve and regulation revenue.
  Added the operating reserve and regulation revenue to net revenue (PR # 611)
• Add functions to compute conflicting constraints when model is infeasible if supported by the solver (#624).
• New settings parameter, VirtualChargeDischargeCost to test script and VREStor example case. The PR 608 attempts to
  introduce this parameter as cost of virtual charging and discharging to avoid unusual results (#608).
• New settings parameter, StorageVirtualDischarge, to turn storage virtual charging and discharging off if desired by the user (#638).
• Add module to retrofit existing resources with new technologies (#600).

Fixed

• Set MUST_RUN=1 for RealSystemExample/small_hydro plants (#517).
  Previously these plants had no resource flag set, and so they did not contribute to the power balance.
  As these plants are now useful, the objective in these cases is slightly lower.
• Assign correct investment cost per stage in objective function initialization for multi-stage runs (#530)
• Fix name of Fixed_OM_Cost_Charge_per_MWyr when reading from Generators_data in multistage code. (#533)
  Previously there was a typo in this in the multistage code that led to a silent bug, which affects outputs,
  for anyone running non-myopic multistage GenX with asymmetric storage.
• Fix computation of cumulative minimum capacity retirements in multistage GenX (#514)
• Fix access of eELOSSByZone expr before initialization (#541)
• Correctly write unmet reserves (in reg_dn.csv) (#575)
• Correctly scale total reserves column (in reg_dn.csv) (#594)
• Add validation for Reg_Max and Rsv_Max columns in Generators_data.csv when MUST_RUN is set to 1 (#576)
• Fix scaling of transmission losses in write_transmission_losses.jl (#621)
• Fix cost assignment to virtual storage charge/discharge - issue #604 (#608)
• Fix modeling of hydro reservoir with long duration storage (#572).
• Fix update of starting transmission capacity in multistage GenX
• Fix write_status with UCommit = WriteShadowPrices = 1 (#645)

Changed

• Use add_to_expression! instead of the += and -= operators for memory performance improvements (#498).
• Generally, 'demand' is now used where (electrical) 'load' was used previously (#397).
• Load_data.csv is being renamed to Demand_data.csv.
• Load_MW_z* columns in that file are renamed to Demand_MW_z*.
• In Reserves.csv, Reg_Req_Percent_Load and Rsv_Req_Percent_Load are renamed to ..._Demand.
• Load and LoadWeight keys in the time_domain_reduction_settings.yml file are renamed to Demand, DemandWeight.
• The New_Build column in Generators_data.csv has been separated into two: New_Build and Can_Retire (#392).
  Values in each column are {0,1}.
• Separate proprietary JuMP solvers from the GenX package.
  This allows users of Gurobi or CPLEX to use them without modifying
  the source code of the GenX package directly. This is a key step in publishing
  GenX as a proper Julia package. This does require change to the Run.jl files,
  to specify the solver. (#531)
• In the examples, change Reg_Max and Rsv_Max of any MUST_RUN generators to 0.
  This mitigates but does not fully fix (#576).
• Expressions of virtual charging and discharging costs in storage_all.jl and vre_stor.jl
• The input file Generators_data.csv has been split into different files, one for each type of generator.
  The new files are: Thermal.csv, Hydro.csv, Vre.csv, Storage.csv, Flex_demand.csv, Must_run.csv,
  Electrolyzer.csv, and Vre_stor.csv. The examples have been updated, and new tests have been added to
  check the new data format (#612).
• The settings parameter Reserves has been renamed to OperationalReserves, Reserves.csv to
  Operational_reserves.csv, and the .jl files contain the word reserves have been renamed to
  operational_reserves (#641).
• New folder structure for a GenX case. The input files are now organized in the following folders: settings,
  policies, resources and system. The examples and tests have been updated to reflect this change.

Deprecated

• The above load keys, which generally refer to electrical demand, are being deprecated.
  Users should update their case files.
  For now this is done in a backward-compatible manner, and @info reminders are written to the log to prompt the user to update.
  "Load" now typically refers only to the transferrence of data from files to memory,
  except for a few places such as the common term "value of lost load" which refers to non-served demand (#397).
• New_Build = -1 in Generators_data.csv: instead, use New_Build = 0 and Can_Retire = 0.
• The matrix-style input of the grid for Network.csv is deprecated in favor a column-style input.
  Instead of columns z1, z2, ... with entries -1, 0, 1, use two columns: Start_Zone, End_Zone (#591).

Full Changelog: v0.3.6...v0.4.0-rc

GenX Patch Version v0.3.6

What's Changed

Fixed

• Order of slack policy constraint declarations (#464)
• Sign error in the Maximum Capacity Requirement slack constraint term (#461)
• Remove outdated HiGHS setting simplex_dualise_strategy (#489)
• Fix bug in LDES outputs (#472)
• Updated README with new instructions for running GenX through Julia REPL terminal (#492)
• Fix factor of 0.5 when writing out transmission losses. (#480)
• Fix summation error when a set of hours is empty (in thermal_commit.jl).

Changed

• Eliminate 'Axis contains one element' warning (seen when LDS is used) by combining two constraints. (#496)

Full Changelog: v0.3.5...v0.3.6

GenX Patch Version v0.3.5

Release Notes for GenX v0.3.5 (Released on: 18th May, 2023)

Added

• Added ability to apply run_timedomainreduction to multistage problems (#443).
• Added a description of how to use time domain reduction (#426).
• Validation: against trying to perform time domain reduction (clustering)
  on data which has already been clustered.
• This changelog (#424).

Fixed

• run_timedomainreduction did not support multi-stage problems (#441).
• Not having a changelog (#423).

Changed

• The columns Rep_Periods, Timesteps_per_Rep_Period, and Sub_Weights are now required in Load_data.csv
  for all cases (#426).

Removed

• The settings key OperationsWrapping. Its functionality has now been folded into the
  TimeDomainReduction setting. Using the key now will print a gentle warning (#426).

Full Changelog: v0.3.4...v0.3.5

GenX Patch Version v0.3.4

[0.3.4] - 2023-04-28

Added

• Validation of the time basis in load_data.csv (#413).
• Arbitrary option keys can be passed to Solvers (#356).
• Validation for OperationWrapping and TimeDomainReduction settings (#337).
• Ability to use 'slack variables' to violate policy constraints---at a cost (#328).
• Documented minimum up & down-time constraints (#324).
• Validation preventing two columns of input data with the same name (#309).
• Fuel type None is no longer need to be listed as a column in fuels_data.csv, e.g. for solar panels (#303).
• Non-varying generators (e.g. thermal generators) no longer need to be listed in generators_variability.csv (#303).
• Ability to load the transmission network representation from lists rather than a matrix (#292).
• Maximum Capacity Requirement policy constraint.
• New run_genx_case! function for use in scripts.
• New run_timedomainreduction! function for pre-clustering a case.
• Improved documentation.

Fixed

• Corrected the interaction of Reserves and Regulation policies with ramp rates (#415).
• Removed the useless MinCapTag column from examples (#380).
• Removed invalid BarObjRng key from gurobi settings (#374).
• Default crossover or run_crossover settings (#363).
• HYDRO resources now allow the period map to be loaded (#362).
• Numbering in documentation (#330).
• Correct scaling of emission outputs (#322).
• Add transmission losses to ESR constraints (#320).
• Author's name spelling in docs (#317).
• Unset executable bits on files (#297).
• Morris method example now runs.
• Various other settings issues with example cases.

Changed

• Simplified the Simple_Test_Case example (#414).
• SmallNewEngland/Onezone example now uses linearized unit committment by default (#404).
• Removed the unused dependency BenchmarkTools (#381).

Removed

• The unmaintained MonteCarlo code (#357).
• License blocks from most file headers (#353).
• Extra LDS columns from several examples (#312).
• SCIP from the Project and from documentation.

What's Changed

• Slightly squashed Weekly-Develop branch by @cfe316 in #377
• Remove scary warning about CapResPeriodLength by @cfe316 in #419
• Rebase Develop onto main to prepare for GenX 0.3.4 release by @cfe316 in #421

Full Changelog: v0.3.3...v0.3.4

v0.3.3

What's Changed

• Added docs.yml to enable doc page versioning by @sambuddhac in #287
• PR to merge Develop with main for version 0.3.3 release by @sambuddhac in #291
• Deleting doc.yml by @sambuddhac in #296

Full Changelog: v0.3.2...v0.3.3

GenX patch version v0.3.2

What's Changed

• Fix indexing in setting eTotalCapCharge max and min by @RuaridhMacd in #269
• Feature updates and bug fixes for v0.3.2 patch release by @sambuddhac in #270
• CRM Zone Membership feature by @xuqingyu
• Cleaner output file writing and bud fixes for DDP method by @jf-morris

Full Changelog: v0.3.1...v0.3.2

v0.3.1

What's Changed

The major updates to the v0.3.1 patch release are the following:

1. Addition of HiGHS as the default solver (HiGHS is open-source, free/non-commercial, and a fast solver as long as LP problems are being solved)
2. Significant changes to the way in which GenX is run (We decided to stick to what we have been calling Mathod 1 till date and decided to get rid of Method 2. Please read the README carefully to run GenX)
3. Revised and refined DDP model
4. Gotten rid of unused package dependencies to expedite the loading and running of GenX.

The details of the patch release as as follows:

• Use joinpath rather than manual path construction from strings by @cfe316 in #191
• Simplify load_period_map by @cfe316 in #192
• Simplify dataframe accesses by @cfe316 in #196
• Reformatted the documentation Welcome page to sectionalize it and condense the left pane by @sambuddhac in #199
• Simplify model generation function calls by @cfe316 in #197
• Simplify flexible demand formulation by @cfe316 in #195
• Fix bug in endogenous_retirement by @cfe316 in #203
• Remove extraneous method_of_morris files by @cfe316 in #207
• Modifications in thermal_no_commit and versioned doc pages by @sambuddhac in #206
• Dev by @sambuddhac in #193
• Prevent writing_storagedual if there's no STOR > 0 by @cfe316 in #226
• GenX Patch 0.3.1 Upgrade: Addition of HiGHS Solver, DDP bug-fixes, and documentation upgrade by @sambuddhac in #236
• Fix images in how_to_run_genx.md by @cfe316 in #260
• Rm five unused files from / by @cfe316 in #261
• Rm Documenter[Tools] from main project by @cfe316 in #262

Full Changelog: v0.3.0...v0.3.1

GenX version v0.3.0

GenX version 0.3.0 Release Notes

In this version, we have upgraded GenX to version v0.3.0. To run GenX 0.3.0, we recommend Julia language version 1.6.x series along with JuMP 0.21.x series (all the environment dependencies have been updated to the most recent compatible versions as well). The following are the salient features that have been added in this version release:

• Added Dual Dynamic Programming (DDP) Multi-stage capacity expansion feature to GenX: This is the major highlight and most important new feature added to GenX in version 0.3.0. Addition of the DDP now allows GenX to carry out multi stage and multi year look-ahead capacity expansion planning into the future in a most rigorous mathematical framework.
• Vectorization of output writing to enhance the speed of generating results files.
• Revised and updated "Method of Morris" for sophisticated sensitivity analyses studies (along with previously created "Modeling to Generate Alternatives (MGS)" analysis).
• Bug-fixes and refinement of modularization of the different policy modules like co2_cap, cap_reserve_margin, minimum_capacity_requirement etc., for ease and convenience for adding new technologies by the users.

Below are the details of the changes in this release:

What's Changed

• Merging Dev into Method of Morris by @nspatank in #66
• modularizing policies: capacity reserve margin, minimum requirement capacity, ESR by @aneeshamanocha in #68
• Multi-Stage GenX by @jf-morris in #78
• Make 'solver' entry case insensitive; document this by @cfe316 in #103
• Simplify loading scripts: use joinpath by @cfe316 in #97
• Use joinpath rather than passing string separator by @cfe316 in #104
• Remove debugging info from flexible demand module by @cfe316 in #107
• Remove service for unsupported Julia versions by @cfe316 in #105
• Add cplex.log file to gitignore. by @cfe316 in #88
• Shorten the .gitignore file by @cfe316 in #110
• Add a default for MultiStage to configure_settings by @cfe316 in #90
• Policies (added must run for capacity reserve margins) by @aneeshamanocha in #69
• Fix bug on main which prevents writing many outputs by @cfe316 in #123
• Refactor period map loading logic by @cfe316 in #130
• Remove unused code requiring CapRes_* cols by @gschivley in #120
• Remove references to LongDurationStorage key by @cfe316 in #124
• Call write_costs exactly once by @cfe316 in #136
• Resolving first-stage-only network expansion issue in multi-stage GenX by @jf-morris in #131
• Add github action test workflow by @gschivley in #142
• Annualizing costs in myopic multi-stage GenX by @jf-morris in #144
• Vectorize the simpler writing procedures by @cfe316 in #137
• Vectorize write_storagedual by @cfe316 in #139
• Documentation for Multi-Stage GenX by @jf-morris in #160
• Vectorize write_costs by @cfe316 in #145
• Vectorize the three slowest outputs by @cfe316 in #147
• Annualize outputs for reserves and ucommit by @cfe316 in #155
• Vectorize writing of outputs by @cfe316 in #157
• DDP Documentation Fix by @jf-morris in #163
• Correcting multi-stage TDR indexing error by @jf-morris in #162
• Include STOR=2 techs in STOR_LONG_DURATION and STOR_SHORT_DURATION sets by @jf-morris in #159
• Fix mismatched arguments in write_outputs by @cfe316 in #175
• Fix and simplify Up&Down_Time constraints by @cfe316 in #151
• Scale all generator Cap_Size, not just those with > 1 MW by @cfe316 in #178
• Gate for accessing Hydro_Energy_to_Power_Ratio by @cfe316 in #174
• DDP Post-convergence extra forward pass by @jf-morris in #182
• Aaron: Reformatting DDP Directory by @jf-morris in #183
• Fix zonal cost summations by @cfe316 in #185
• Dev by @sambuddhac in #179
• Dev by @sambuddhac in #188

New Contributors

• @aneeshamanocha made their first contribution in #68
• @jf-morris made their first contribution in #78

Full Changelog: v0.2.0...v0.3.0