updating install requirements

AyrtonB · May 26, 2021 · ec68037 · ec68037
1 parent a69cb5d
commit ec68037
Show file tree

Hide file tree

Showing 3 changed files with 45 additions and 23 deletions.
diff --git a/README.md b/README.md
@@ -2,32 +2,62 @@
 
 # MOE-Py
 
-This repository outlines the development and usage of code and analysis used in calculating the Merit-Order-Effect (MOE) of renewables on price and carbon intensity of electricity markets. Beyond MOE analysis the `moepy` library can be used more generally for standard, quantile, and bootstrapped LOWESS estimation. The particular implementation of LOWESS in this software has been extended to significantly reduce the computational resource required. 
+### Installation
 
 To reduce dependency bloat the `moepy` library can be installed in two ways. If you just wish to use the LOWESS curve fitting aspects of the library then you can install it using:
 
 ```bash
 pip install moepy
 ```
 
-If you wish to reproduce the analysis used for estimation of the Merit Order Effect please run :
+If you wish to reproduce the analysis used for estimation of the Merit Order Effect in [this paper](https://github.com/AyrtonB/Merit-Order-Effect/blob/main/paper/Quantifying%20the%20Merit%20Order%20Effect%20of%20Renewables%20on%20Price%20and%20Carbon%20Intensity%20of%20the%20British%20and%20Germany%20Energy%20Systems.pdf) please run :
 
 ```bash
 pip install moepy[research]
 ```
 
-`moepy` makes it simple to fit a LOWESS curve, a quick start example to generate the plot below can be found [here](https://ayrtonb.github.io/Merit-Order-Effect/ug-04-gb-mcc/).
+<br>
+
+### Overview
+
+This repository outlines the development and usage of code and analysis used in calculating the Merit-Order-Effect (MOE) of renewables on price and carbon intensity of electricity markets. Beyond MOE analysis the `moepy` library can be used more generally for standard, quantile, and bootstrapped LOWESS estimation. The particular implementation of LOWESS in this software has been extended to significantly reduce the computational resource required. 
+
+The original LOWESS model was developed by W. S. Cleveland in 1979 and the FORTRAN code for it can be found [here](https://www.netlib.org/go/lowess). LOWESS is an acronym for Locally Weighted Scatterplot Smoothing, whereby multiple regressions are fitted over different regions of the data domain and then combined based on weightings linked to the distance between the prediction point and the data used to fit each of the local regressions.
+
+`moepy` exposes several variants on the traditional LOWESS,  including estimation of confidence and prediction intervals, as well as the robust LOWESS proposed by Cleveland in 1988. Examples  of how to use each of these are shown in the [Quick-Start notebook](#). An example of a LOWESS curve estimated by `moepy` can be seen below, the code to generate the plot can be found [here](https://ayrtonb.github.io/Merit-Order-Effect/ug-04-gb-mcc/).
 
 ![](img/latest_gb_mcc.png)
 
 <br>
 
-The library also includes the option to ensemble LOWESS models together and smooth them over time, an example is shown below for the marginal cost curve of dispatchable generation in Great Britain. For ease of use this has been made to work directly with Pandas datetime indexes and is exposed through a standard sklearn model API.
+The library also includes the option to ensemble LOWESS models together (very meta) and smooth them over time without including time within the regression itself. This is particularly useful when there are many exogenous factors changing over time but you don't have data for them, meaning it is therefore not easy to capture them within the regression. An example fit from the ensembled date-smoothing model is shown below, depicting the marginal price curve of dispatchable generation in Great Britain. For ease of use this model has been made to work directly with Pandas datetime indexes and is again exposed through a standard sklearn model API.
 
 ![](img/UK_price_MOE_heatmap.png)
 
 <br>
 
+### Examples
+
+Several notebooks have been created to show examples of how LOWESS estimations can be made using various data sources and for different analysis purposes, these include:
+* Quantile estimation of hydro-power production in Portgual
+* Confidence interval estimation of gravitational wave observations
+* Cleaning of wind power curves
+* Estimation of electricity price curves
+
+Key plots from each of these can be seen below.
+
+![](img/lowess_fit_examples.png)
+
+Get involved! If you have used `moepy` for something cool and want to share it with others please create a pull request containing a notebook with your self-contained example.
+
+<br>
+
+### Why MOE-Py?
+
+LOWESS implementation in Python is relatively sparse compared to most regression methods. Statsmodels provides a [LOWESS model](https://www.statsmodels.org/dev/_modules/statsmodels/nonparametric/smoothers_lowess.html), however it exposes no predict method, forcing users to add wrappers on top that linearly interpolate between the results Statsmodels does return (e.g. [here](https://stackoverflow.com/a/37060980/8035710) and [here](https://towardsdatascience.com/lowess-regression-in-python-how-to-discover-clear-patterns-in-your-data-f26e523d7a35)). In contrast the `moepy` library is made specifically for LOWESS estimation and provides it through an `sklearn` compatible API, meaning that it works within the broader `sklearn` eco-system for model ensembling and tuning as well as reducing the learning barrier. 
+
+<br>
+
 ### Paper
 
 The `moepy` library was developed to enable new research into the Merit-Order-Effect of renewables in the British and German power systems. The full paper can be found [here](https://ayrtonb.github.io/Merit-Order-Effect/assets/Quantifying%20the%20MOE%20in%20Britain%20&%20Germany.pdf), the abstract is shown below:
@@ -38,7 +68,7 @@ The key premise behind the analysis is that intermittent renewables with no fuel
 
 ![](img/MOE_diagram_supply_shift.png)
 
-In this work a time-adaptive LOWESS was used to estimate the marginal price curve, then simulate the MOE. We calculated significant CO2 emission and electricity price savings for Britain and Germany, results for 2019 are shown in the table below.
+In this work a time-adaptive LOWESS was used to estimate the marginal price curve, then simulate the MOE. We calculated significant CO2 emission and electricity price savings for Britain and Germany. Results for 2019 are shown in the table below.
 
 |                       |   Germany |   Great Britain |
 |:----------------------|----------:|----------------:|
@@ -53,25 +83,9 @@ We identified a strong relationship between increasing renewable penetration and
 
 <br>
 
-### Examples
-
-Several notebooks have been created to show examples of how LOWESS estimations can be made using various data sources and for different analysis purposes, these include:
-* Quantile estimation of hydro-power production in Portgual
-* Confidence interval estimation of gravitational wave observations
-* Cleaning of wind power curves
-* Estimation of electricity price curves
-
-Key plots from each of these can be seen below.
-
-![](img/lowess_fit_examples.png)
-
-If you have used `moepy` for something cool and want to share it with others please create a pull request containing a notebook with your self-contained example.
-
-<br>
-
 ### Referencing
 
-If you use this software please cite it using the following:
+If you wish to reference a specific version of this software please use the following citation, changing the DOI to reflect the specific version you used (these are available through the Zenodo link at the top of this page):
 
 ```
 @software{bourn_moepy_2021,

diff --git a/settings.ini b/settings.ini
@@ -21,4 +21,6 @@ title = moepy
 doc_path = docs
 doc_host = https://AyrtonB.github.io
 doc_baseurl = /Merit-Order-Effect/
-requirements = pandas>=1.2.0 numpy>=1.19.5 matplotlib>=3.3.3 lxml>=4.6.2 tqdm>=4.59.0 scikit-learn>=0.24.0 scipy>=1.6.0
+requirements = pandas>=1.2.0 numpy>=1.19.5 matplotlib>=3.3.3 tqdm>=4.59.0 scikit-learn>=0.24.0 scipy>=1.6.0
+extra_cicd_requirements = configparser>=5.0.0 typer>=0.3.2
+extra_research_requirements = lxml>=4.6.2 xmltodict>=0.12.0 entsoe-py>=0.3.2 python-dotenv>=0.15.0
diff --git a/setup.py b/setup.py
@@ -11,6 +11,11 @@
 config.read('settings.ini')
 setup_kwargs = dict(config['DEFAULT'])
 
+extra_requirements = {
+   'cicd': setup_kwargs['extra_cicd_requirements'].split(' '),
+   'research': setup_kwargs['extra_research_requirements'].split(' ')
+}
+
 setuptools.setup(
     name=setup_kwargs['lib_name'],
     version=setup_kwargs['version'],
@@ -22,6 +27,7 @@
     url=setup_kwargs['git_url'],
     packages=setuptools.find_packages(),
     install_requires=setup_kwargs['requirements'].split(' '),
+    extras_require=extra_requirements,
     classifiers=[
         "Programming Language :: Python :: 3",
     ],