Merge pull request #59 from lanl-ansi/v0.4-dev

UPD: New multi-infrastructure conventions, examples, and documentation.
lanl-ansi · Apr 15, 2021 · 3d7b8c9 · 3d7b8c9
2 parents d9b130f + ecee21c
commit 3d7b8c9
Show file tree

Hide file tree

Showing 120 changed files with 24,490 additions and 8,938 deletions.
diff --git a/Project.toml b/Project.toml
@@ -2,28 +2,31 @@ name = "GasPowerModels"
 uuid = "1fcaf345-95e3-5b8b-91d6-2145eb844b9d"
 authors = ["Russell Bent"]
 repo = "https://github.com/lanl-ansi/GasPowerModels.jl"
-version = "0.3.0"
+version = "0.4.0"
 
 [deps]
 GasModels = "5e113713-6c35-5477-b766-e1109486666f"
+InfrastructureModels = "2030c09a-7f63-5d83-885d-db604e0e9cc0"
 JSON = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
 JuMP = "4076af6c-e467-56ae-b986-b466b2749572"
 Memento = "f28f55f0-a522-5efc-85c2-fe41dfb9b2d9"
-PowerModels = "c36e90e8-916a-50a6-bd94-075b64ef4655"
+PowerModelsRestoration = "23adbb12-a187-11e9-26a2-eb4d4e6e68fb"
 
 [compat]
-GasModels = "~0.8"
-JSON = "~0.18, ~0.19, ~0.20, ~0.21"
+GasModels = "~0.9"
+InfrastructureModels = "~0.6"
+JSON = "~0.21"
 JuMP = "~0.21"
 Memento = "~1.0, ~1.1"
-PowerModels = "~0.17"
+PowerModelsRestoration = "~0.6"
 julia = "^1"
 
 [extras]
 Cbc = "9961bab8-2fa3-5c5a-9d89-47fab24efd76"
 Ipopt = "b6b21f68-93f8-5de0-b562-5493be1d77c9"
 Juniper = "2ddba703-00a4-53a7-87a5-e8b9971dde84"
+Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
 
 [targets]
-test = ["Test", "Cbc", "Juniper", "Ipopt"]
+test = ["Cbc", "Juniper", "Ipopt", "Logging", "Test"]
diff --git a/README.md b/README.md
@@ -7,19 +7,17 @@
 GasPowerModels.jl is a Julia/JuMP package for the joint optimization of steady state natural gas and power transmission networks.
 It provides utilities for modeling problems that combine elements of natural gas and electric power systems.
 It is designed to enable the computational evaluation of historical and emerging gas-power network optimization formulations and algorithms using a common platform.
-The code is engineered to decouple problem specifications (e.g., gas-power flow, network expansion planning) from network formulations (e.g., mixed-integer linear, mixed-integer nonlinear).
+The code is engineered to decouple problem specifications (e.g., gas-power flow, network expansion planning) from network formulations (e.g., mixed-integer convex, mixed-integer nonconvex).
 This decoupling enables the definition of a variety of optimization formulations and their comparison on common problem specifications.
 
 **Core Problem Specifications**
 * Gas-Power Flow (`gpf`)
+* Maximum Load Delivery (`mld`)
+* Maximum Load Delivery - Unit Commitment  (`mld_uc`)
 * Optimal Power Flow (`opf`)
 * Network Expansion Planning (`ne`)
 * Optimal Power Flow with Network Expansion Planning (`opf_ne`)
 
-**Core Network Formulations**
-* Directed flow, mixed-integer nonconvex formulation (`D`)
-* Convexly relaxed, directed flow mixed-integer formulation (`CRD`)
-
 ## Documentation
 The package [documentation](https://lanl-ansi.github.io/GasPowerModels.jl/stable/) includes a [quick start guide](https://lanl-ansi.github.io/GasPowerModels.jl/stable/quickguide).
 
@@ -50,44 +48,45 @@ Installation of the JuMP interfaces to Juniper, Ipopt, and Cbc can be performed
 ] add JuMP Juniper Ipopt Cbc
 ```
 
-After installation of the required solvers, an example gas-power flow feasibility problem (whose file inputs can be found in the `examples` directory within the [GasPowerModels repository](https://github.com/lanl-ansi/GasPowerModels.jl)) can be solved via
+After installation of the required solvers, an example gas-power flow feasibility problem (whose file inputs can be found in the `test` directory within the [GasPowerModels repository](https://github.com/lanl-ansi/GasPowerModels.jl)) can be solved via
 ```julia
 using JuMP, Juniper, Ipopt, Cbc
 using GasPowerModels
 
 # Set up the optimization solvers.
-ipopt = JuMP.optimizer_with_attributes(Ipopt.Optimizer, "print_level"=>0, "sb"=>"yes")
-cbc = JuMP.optimizer_with_attributes(Cbc.Optimizer, "logLevel"=>0)
-juniper = JuMP.optimizer_with_attributes(Juniper.Optimizer, "nl_solver"=>ipopt, "mip_solver"=>cbc)
-
-# Specify paths to the gas and power network files.
-g_file = "examples/data/matgas/belgian.m" # Gas network.
-p_file = "examples/data/matpower/case14.m" # Power network.
-
-# Specify the gas and power formulation types separately.
-g_type, p_type = CRDWPGasModel, SOCWRPowerModel
-
-# Solve the gas-power flow feasibility problem.
-result = run_gpf(g_file, p_file, g_type, p_type, juniper;
-    gm_solution_processors=[GasPowerModels._GM.sol_psqr_to_p!],
-    pm_solution_processors=[GasPowerModels._PM.sol_data_model!])
+ipopt = JuMP.optimizer_with_attributes(Ipopt.Optimizer, "print_level" => 0, "sb" => "yes")
+cbc = JuMP.optimizer_with_attributes(Cbc.Optimizer, "logLevel" => 0)
+juniper = JuMP.optimizer_with_attributes(Juniper.Optimizer, "nl_solver" => ipopt, "mip_solver" => cbc)
+
+# Specify paths to the gas, power, and linking files.
+g_file = "test/data/matgas/GasLib-11-GPF.m" # Gas network.
+p_file = "test/data/matpower/case5-GPF.m" # Power network.
+link_file = "test/data/json/GasLib-11-case5.json" # Linking data.
+
+# Specify the gas-power formulation type.
+gpm_type = GasPowerModel{CRDWPGasModel, SOCWRPowerModel}
+
+# Solve the gas-power flow (gpf) feasibility problem.
+result = run_gpf(g_file, p_file, link_file, gpm_type, juniper;
+    solution_processors = [GasPowerModels._GM.sol_psqr_to_p!,
+    GasPowerModels._PM.sol_data_model!])
 ```
 
 After solving the problem, results can then be analyzed, e.g.,
 ```julia
 # The termination status of the optimization solver.
 result["termination_status"]
 
-# Generator 1's real power generation.
-result["solution"]["gen"]["1"]["pg"]
+# Generator 1's active (real) power generation.
+result["solution"]["it"]["pm"]["gen"]["1"]["pg"]
 
 # Junction 1's pressure.
-result["solution"]["junction"]["1"]["p"]
+result["solution"]["it"]["gm"]["junction"]["1"]["p"]
 ```
 
 ## Acknowledgments
-The primary developers are Russell Bent and Kaarthik Sundar.
-Significant contributions on the technical model were made by Conrado Borraz-Sanchez, Pascal van Hentenryck, and Seth Blumsack.
+The primary developers are Russell Bent, Kaarthik Sundar, and Byron Tasseff.
+Significant contributions on the technical model were made by Conrado Borraz-Sanchez, Pascal Van Hentenryck, and Seth Blumsack.
 Special thanks to Miles Lubin and Carleton Coffrin for their assistance in integrating with Julia/JuMP and PowerModels.jl.
 
 ## License

diff --git a/docs/make.jl b/docs/make.jl
@@ -17,14 +17,14 @@ makedocs(
             "Network Formulations" => "formulations.md",
             "Problem Specifications" => "specifications.md",
             "Modeling Components" => [
-                "GasGridModel" => "model.md",
                 "Objective" => "objective.md",
                 "Variables" => "variables.md",
                 "Constraints" => "constraints.md"
             ],
             "File IO" => "parser.md"
         ],
-        "Developer" => "developer.md"
+        "Developer" => "developer.md",
+        "Examples" => "examples.md"
     ]
 )
 

diff --git a/docs/src/constraints.md b/docs/src/constraints.md
@@ -1,4 +1,8 @@
 # Constraints
+We define the following methods to provide a compositional approach toward defining common constraints used in coupled gas-power models.
+These methods should always be defined over `AbstractGasPowerModel`.
+
+
 ```@autodocs
 Modules = [GasPowerModels]
 Pages   = ["core/constraint_template.jl"]

diff --git a/docs/src/developer.md b/docs/src/developer.md
@@ -1 +1,74 @@
 # Developer Documentation
+
+The GasPowerModels data format allows the user to specify gas network data, power network data, and data related to the interdependencies between gas and power systems.
+
+## Data Processing functions
+
+`GasPowerModels` relies on the automated data processing routines of `GasModels` and `PowerModels`, which include capabilities for status propagation, nondimensionalization, topology correction, etc.
+However, these capabilities are typically used on independent infrastructure data, whereas `GasPowerModels` must join these data.
+Thus, in preprocessing routines, it is recommended that capabilities be invoked explictly so that external dependencies are accounted for.
+For example, the core data parsing function `parse_files` performs the following operations:
+
+```julia
+function parse_files(gas_path::String, power_path::String, link_path::String)
+    joint_network_data = parse_link_file(link_path)
+    _IM.update_data!(joint_network_data, parse_gas_file(gas_path))
+    _IM.update_data!(joint_network_data, parse_power_file(power_path))
+
+    # Store whether or not each network uses per-unit data.
+    g_per_unit = get(joint_network_data["it"][_GM.gm_it_name], "is_per_unit", 0) != 0
+    p_per_unit = get(joint_network_data["it"][_PM.pm_it_name], "per_unit", false)
+
+    # Correct the network data.
+    correct_network_data!(joint_network_data)
+
+    # Ensure all datasets use the same units for power.
+    resolve_units!(joint_network_data, g_per_unit, p_per_unit)
+
+    # Return the network dictionary.
+    return joint_network_data
+end
+```
+
+Here, the `parse_gas_file` and `parse_power_file` routines skip their respective data correction steps, i.e.,
+
+```julia
+function parse_gas_file(file_path::String; skip_correct::Bool = true)
+    data = _GM.parse_file(file_path; skip_correct = skip_correct)
+    ...
+end
+
+function parse_power_file(file_path::String; skip_correct::Bool = true)
+    data = _PM.parse_file(file_path; validate = !skip_correct)
+    ...
+end
+```
+
+This ensures the per-unit statuses within source files are preserved so that `GasPowerModels` can determine if interdependency data requires nondimensionalization.
+After these routines are called, `correct_network_data!` executes various data and topology correction routines on gas, power, and linking data.
+Then, `resolve_units` ensures that linking data is correctly dimensionalized with respect to the initial gas and power dimensionalizations.
+
+
+## Compositional Problems
+
+A best practice is to adopt a compositional approach for building problems in `GasPowerModels`, leveraging problem definitions of `PowerModels` and `GasModels`.
+This helps lessen the impact of breaking changes across independent infrastructure packages.
+For example, the joint network expansion planning problem invokes the network expansion planning problems of `GasModels` and `PowerModels` directly with routines like
+
+```julia
+# Gas-only variables and constraints
+_GM.build_nels(_get_gasmodel_from_gaspowermodel(gpm))
+
+# Power-only variables and constraints
+_PM.build_tnep(_get_powermodel_from_gaspowermodel(gpm))
+
+# Gas-power related parts of the problem formulation.
+for i in _get_interdependent_deliveries(gpm)
+    constraint_heat_rate(gpm, i)
+end
+
+# Objective minimizes cost of network expansion.
+objective_min_ne_cost(gpm)
+```
+
+Compared to the `GasModels` (`_GM`) and `PowerModels` (`_PM`) routines, the `GasPowerModels` routines only specify interdependency constraints and the joint objective.
diff --git a/docs/src/examples.md b/docs/src/examples.md
@@ -0,0 +1,30 @@
+# Examples Documentation
+
+The `examples` folder in the root directory contains a library of gas-power network instances which have been developed in the literature.
+
+Many of the examples can be run using the `run_examples.jl` script which executes various problems and formulations on the library of instances and verifies that `GasPowerModels` returns solutions which were reported in the literature.
+Some results, especially those based on relaxations, have departed from those reported in the literature due to advances that have tightened these relaxations since those papers have been published.
+
+Long term, the plan is to move the examples out of the `GasPowerModels` repository and maintain a special `GasPowerModelsLib` repository specifically for warehousing models developed in the literature.
+
+
+| Problems                  | Source                    |
+| -----------------------   | ------------------------  |
+| case36                    | [1] (base model)          |
+| case36-ne-*               | [1] (network expansion)   |
+| case14-ne                 | [2] (0% stress case)      |
+| case14-ne-100             | [2] (100% stress case)    |
+| northeast                 | [1] (base model)          |
+| northeast-ne-*            | [1] (network expansion)   |
+| northeast-ne-C            | [1] (section IV-C)        |
+| northeast-ne-D-*          | [1] (section IV-D)        |
+| northeast-ne-E-*          | [1] (section IV-E)        |
+| belgian-ne                | [2] (0% stress case)      |
+| belgian-ne-100            | [2] (100% stress case)    |
+
+
+## Sources
+
+[1] Russell Bent, Seth Blumsack, Pascal Van Hentenryck, Conrado Borraz-Sánchez, Mehdi Shahriari. Joint Electricity and Natural Gas Transmission Planning With Endogenous Market Feedbacks. IEEE Transactions on Power Systems. 33 (6): 6397-6409, 2018.
+
+[2] C. Borraz-Sanchez, R. Bent, S. Backhaus, S. Blumsack, H. Hijazi, and P. van Hentenryck. Convex Optimization for Joint Expansion Planning of Natural Gas and Power Systems. Proceedings of the 49th Hawaii International Conference on System Sciences (HICSS-49) (HICSS 2016), Jan. 2016, Grand Hyatt, Kauai.
diff --git a/docs/src/formulations.md b/docs/src/formulations.md
@@ -1,2 +1,32 @@
 # Network Formulations
 The network formulations for joint gas-power modeling use the formulations defined in GasModels.jl and PowerModels.jl.
+
+
+# GasPowerModels Types
+```@meta
+CurrentModule = GasPowerModels
+```
+
+Specification of a `GasPowerModel` requires the specification of both a `GasModels.AbstractGasModel` and a `PowerModels.AbstractPowerModel`, respectively.
+For example, to specify a formulation that leverages the `CRDWPGasModel` and `SOCWRPowerModel` types, the corresponding `GasPowerModel` type would be
+```julia
+GasPowerModel{CRDWPGasModel, SOCWRPowerModel}
+```
+
+GasPowerModels then utilizes the following (internal) function to construct a `GasPowerModel` object:
+```@docs
+instantiate_model
+```
+
+## Type Hierarchy
+
+GasPowerModels inherits the type hierarchies of GasModels and PowerModels.
+Constraint and objective functions are then dispatched based on the choice of types for each of the models.
+An example is seen in the function
+```@docs
+constraint_heat_rate
+constraint_heat_rate_on_off
+```
+
+The convention is that, if a relaxation or approximation of a nonconvex constraint is used in a natural gas and/or electric power model, the linking constraint will also be similarly relaxed or approximated according to the most "complex" independent infrastructure modeling formulation.
+For example, if the natural gas formulation uses a linear representation, and the electric power model uses a quadratic representation, then the linking constraint uses the tightest possible relaxation using linear and quadratic interdependency equations.
diff --git a/docs/src/index.md b/docs/src/index.md
@@ -8,7 +8,7 @@ CurrentModule = GasPowerModels
 GasPowerModels.jl is a Julia/JuMP package for the joint optimization of steady state natural gas and power transmission networks.
 It provides utilities for modeling problems that combine elements of natural gas and electric power systems.
 It is designed to enable the computational evaluation of historical and emerging gas-power network optimization formulations and algorithms using a common platform.
-The code is engineered to decouple [Problem Specifications](@ref) (e.g., gas-power flow, network expansion planning) from [Network Formulations](@ref) (e.g., mixed-integer linear, mixed-integer nonlinear).
+The code is engineered to decouple [Problem Specifications](@ref) (e.g., gas-power flow, network expansion planning) from [Network Formulations](@ref) (e.g., mixed-integer convex, mixed-integer nonconvex).
 This decoupling enables the definition of a variety of optimization formulations and their comparison on common problem specifications.
 
 ## Installation
@@ -32,43 +32,44 @@ At least one optimization solver is required to run GasPowerModels.
 The solver selected typically depends on the type of problem formulation being employed.
 As an example, the mixed-integer nonlinear programming solver [Juniper](https://github.com/lanl-ansi/Juniper.jl) can be used for testing any of the problem formulations considered in this package.
 Juniper itself depends on the installation of a nonlinear programming solver (e.g., [Ipopt](https://github.com/jump-dev/Ipopt.jl)) and a mixed-integer linear programming solver (e.g., [CBC](https://github.com/jump-dev/Cbc.jl)).
-Installation of the JuMP interfaces to Juniper, Ipopt, and Cbc can be performed via the Julia package manager, i.e.,
+Installation of the JuMP interfaces to Juniper, Ipopt, and CBC can be performed via the Julia package manager, i.e.,
 
 ```julia
 ] add JuMP Juniper Ipopt Cbc
 ```
 
-After installation of the required solvers, an example gas-power flow feasibility problem (whose file inputs can be found in the `examples` directory within the [GasPowerModels repository](https://github.com/lanl-ansi/GasPowerModels.jl)) can be solved via
+After installation of the required solvers, an example gas-power flow feasibility problem (whose file inputs can be found in the `test` directory within the [GasPowerModels repository](https://github.com/lanl-ansi/GasPowerModels.jl)) can be solved via
 ```julia
 using JuMP, Juniper, Ipopt, Cbc
 using GasPowerModels
 
 # Set up the optimization solvers.
-ipopt = JuMP.optimizer_with_attributes(Ipopt.Optimizer, "print_level"=>0, "sb"=>"yes")
-cbc = JuMP.optimizer_with_attributes(Cbc.Optimizer, "logLevel"=>0)
-juniper = JuMP.optimizer_with_attributes(Juniper.Optimizer, "nl_solver"=>ipopt, "mip_solver"=>cbc)
+ipopt = JuMP.optimizer_with_attributes(Ipopt.Optimizer, "print_level" => 0, "sb" => "yes")
+cbc = JuMP.optimizer_with_attributes(Cbc.Optimizer, "logLevel" => 0)
+juniper = JuMP.optimizer_with_attributes(Juniper.Optimizer, "nl_solver" => ipopt, "mip_solver" => cbc)
 
-# Specify paths to the gas and power network files.
-g_file = "examples/data/matgas/belgian.m" # Gas network.
-p_file = "examples/data/matpower/case14.m" # Power network.
+# Specify paths to the gas, power, and linking files.
+g_file = "test/data/matgas/GasLib-11-GPF.m" # Gas network.
+p_file = "test/data/matpower/case5-GPF.m" # Power network.
+link_file = "test/data/json/GasLib-11-case5.json" # Linking data.
 
-# Specify the gas and power formulation types separately.
-g_type, p_type = CRDWPGasModel, SOCWRPowerModel
+# Specify the gas-power formulation type.
+gpm_type = GasPowerModel{CRDWPGasModel, SOCWRPowerModel}
 
-# Solve the gas-power flow feasibility problem.
-result = run_gpf(g_file, p_file, g_type, p_type, juniper;
-    gm_solution_processors=[GasPowerModels._GM.sol_psqr_to_p!],
-    pm_solution_processors=[GasPowerModels._PM.sol_data_model!])
+# Solve the gas-power flow (gpf) feasibility problem.
+result = run_gpf(g_file, p_file, link_file, gpm_type, juniper;
+    solution_processors = [GasPowerModels._GM.sol_psqr_to_p!,
+    GasPowerModels._PM.sol_data_model!])
 ```
 
 After solving the problem, results can then be analyzed, e.g.,
 ```julia
 # The termination status of the optimization solver.
 result["termination_status"]
 
-# Generator 1's real power generation.
-result["solution"]["gen"]["1"]["pg"]
+# Generator 1's active (real) power generation.
+result["solution"]["it"]["pm"]["gen"]["1"]["pg"]
 
 # Junction 1's pressure.
-result["solution"]["junction"]["1"]["p"]
+result["solution"]["it"]["gm"]["junction"]["1"]["p"]
 ```