Skip to content

Commit

Permalink
paper updates
Browse files Browse the repository at this point in the history
  • Loading branch information
trevorb1 committed Sep 26, 2023
1 parent c41d3dc commit 845782f
Showing 1 changed file with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions paper/paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@ Data handling for energy system optimisation models is a necessary but tedious t

# Statement of need

The Open Source energy MOdelling SYStem, or OSeMOSYS [@howells2011], is a highly cited and widely used open-source framework for conducting long-term energy system planning studies. The original, and still highly used, implementation of OSeMOSYS is formulated in the mathematical programming language, [GNU MathProg](https://github.com/OSeMOSYS/OSeMOSYS_GNU_MathProg). While MathProg [@GNULinearProgramming2012] is open-source, fitting with the ethos of OSeMOSYS, it requires all data to be stored in a single large and unwieldy text file. Data edits to this file are tedious, error-prone, and difficult to integrate into automated workflows. Furthermore, model result files are often difficult to interface with unless processing work is first performed. In addition to the MathProg implementation, OSeMOSYS has also been formulated in [PuLP]( https://github.com/OSeMOSYS/OSeMOSYS_PuLP), [Pyomo](https://github.com/OSeMOSYS/OSeMOSYS_Pyomo), [Julia JUMP](https://github.com/sei-international/NemoMod.jl) and [GAMS](https://github.com/OSeMOSYS/OSeMOSYS_GAMS). A software gap exists to provide OSeMOSYS modellers with an easy way to work with different input data formats and process result files.
The Open Source energy MOdelling SYStem, or OSeMOSYS [@howells2011], is a highly cited and widely used open-source framework for conducting long-term energy system planning studies [@groissbock2019]. The original, and still highly used, implementation of OSeMOSYS is formulated in the mathematical programming language, [GNU MathProg](https://github.com/OSeMOSYS/OSeMOSYS_GNU_MathProg). While MathProg [@GNULinearProgramming2012] is open-source, fitting with the ethos of OSeMOSYS, it requires all data to be stored in a single large and unwieldy text file. Data edits to this file are tedious, error-prone, and difficult to integrate into automated workflows. Furthermore, model result files are often difficult to interface with unless processing work is first performed. In addition to the MathProg implementation, OSeMOSYS has also been formulated in [PuLP]( https://github.com/OSeMOSYS/OSeMOSYS_PuLP), [Pyomo](https://github.com/OSeMOSYS/OSeMOSYS_Pyomo), [Julia JUMP](https://github.com/sei-international/NemoMod.jl) and [GAMS](https://github.com/OSeMOSYS/OSeMOSYS_GAMS). A software gap exists to provide OSeMOSYS modellers with an easy way to work with different input data formats and process result files.

Compared to other open energy modelling frameworks that are based around a single (Python) implementation, such as PyPSA[@pypsa2019] or Calliope[@Pfenninger2018], the OSeMOSYS framework does not have standard data processing functions. For example, in PyPSA, users instantiate a `Network` object which has methods to import data (from various formats such as CSV, netCDF, and pypower), manipulate data, build and solve the model, and export results. In contrast, users of OSeMOSYS are required to prepare the input data and process results data themselves. otoole addresses the need in the OSeMOSYS community to standardize data handling functions across multiple different model implementations.

otoole supports three different input file formats: wide-format Excel files (pivoted on the year index), long-format CSV files, and GNU MathProg files. With otoole, users can convert between any of these formats to meet their skill level and workflow requirements. Furthermore, otoole can process result solution files from the open-source solvers GLPK [@GNULinearProgramming2012] and CBC [@forrest2022], and the commercial solvers Gurobi [@gurobioptimizationllcGurobiOptimizerReference2022] and CPLEX [@ibmCPLEXUsersManual2022], into tabulated CSV results.

Expand All @@ -41,13 +43,13 @@ Additionally, otoole can visualize and validate input data. Through the visualiz

# Publications

The development of otoole was originally mentioned in [@niet2021] through a discussion on consolidating and standardising scripts and data formats within the OSeMOSYS community. Since then, otoole has grown and been implemented in various published research projects. A pedagogical paper on conducting global sensitivity analysis in the context of energy systems [@usher2023] uses otoole to convert data for hundreds of simultaneous OSeMOSYS model runs. OSeMOSYS Global [@barnes2022], an open-source global electricity system model generator, uses otoole in its workflow to programmatically create unique OSeMOSYS models based on user inputs. Finally, [@Ramos2022] mentions otoole in a paper discussing how to use OSeMOSYS to perform climate, land, energy, and water system modelling.
The development of otoole was originally mentioned in [@niet2021] through a discussion on consolidating and standardising scripts and data formats within the OSeMOSYS community. Since then, otoole has grown and been implemented in various published research projects. A paper on conducting global sensitivity analysis in the context of energy systems [@usher2023] uses otoole to convert data for hundreds of simultaneous OSeMOSYS model runs. OSeMOSYS Global [@barnes2022], an open-source global electricity system model generator, uses otoole in its workflow to programmatically create unique OSeMOSYS models based on user inputs. Finally, [@Ramos2022] mentions otoole in a paper discussing how to use OSeMOSYS to perform climate, land, energy, and water system modelling.

# Extensibility

While otoole was originally created to assist MathProg OSeMOSYS modellers, its functionality can easily extend to other implementations of OSeMOSYS, and any workflow that uses non-OSeMOSYS MathProg data files.

otoole follows a strategy-based software design pattern. Two abstract base classes define how otoole reads and writes data, named ReadStrategy and WriteStrategy respectively. The ReadStrategy class reads model data or result data into an internal data structure (currently a dictionary of pandas DataFrames). While the WriteStrategy class writes out data to a format specified by the user. The ReadResults abstract class is responsible for calculating intermediate results if required. A schematic of this design pattern is shown in \autoref{fig:design-pattern}.
otoole follows a strategy design pattern[@gamma1994, p.315]. Two abstract base classes define how otoole reads and writes data, named ReadStrategy and WriteStrategy respectively. The ReadStrategy class reads model data or result data into an internal data structure (currently a dictionary of pandas DataFrames). While the WriteStrategy class writes out data to a format specified by the user. The ReadResults abstract class is responsible for calculating intermediate results if required. A schematic of this design pattern is shown in \autoref{fig:design-pattern}.

![otoole Design Pattern \label{fig:design-pattern}](images/class-diagram.png)

Expand All @@ -57,7 +59,7 @@ When converting between formats, the user supplies otoole with a configuration f

# Installation and Example

otoole is [deployed](https://pypi.org/project/otoole/) to the Python Packaging Index (PyPI) and can be installed via pip
otoole is [published](https://pypi.org/project/otoole/) to the Python Packaging Index (PyPI) and can be installed via pip

```bash
pip install otoole
Expand Down

0 comments on commit 845782f

Please sign in to comment.