Merge pull request #335 from rl-institut/features/update_tabular

Features/update tabular

.github/workflows/main.yml

@@ -23,7 +23,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [ 3.8 ]
+        python-version: [ '3.8', '3.9', '3.10']
 
     # Steps represent a sequence of tasks that will be executed as part of the job
     steps:

README.md

@@ -29,15 +29,33 @@ In the documentation you will find instructions on how to run the scenarios with
 
 ### Installation
 
-Currently, oemof-B3 needs python 3.7 or 3.8 (newer versions may be supported, but installation can take very long).
+Currently, oemof-B3 needs python 3.8, 3.9 or 3.10 (newer versions may be supported, but installation can take very long).
 
-In order to install oemof-B3, proceed with the following steps:
+Additionally, you need to install the python dependency manager [poetry](https://python-poetry.org/).
+It is recommended to install poetry system-wide via the command below or
+[pipx](https://python-poetry.org/docs/#installing-with-pipx):
 
-- git-clone oemof-B3 into local folder: `git clone https://github.com/rl-institut/oemof-B3.git`
-- enter folder
-- create virtual environment using conda: `conda env create environment.yml`
-- activate environment: `conda activate oemof-B3`
-- install oemof-B3 package using poetry, via: `poetry install`
+    curl -sSL https://install.python-poetry.org | python3 -
+    poetry install
+
+
+**In order to install oemof-B3, proceed with the following steps:**
+
+1. Clone oemof-B3 into local folder:
+
+       git clone git@github.com:rl-institut/oemof-B3.git
+2. Enter folder
+
+       cd oemof-B3
+3. Create virtual environment using conda:
+
+       conda env create environment.yml
+4. Activate environment:
+
+       conda activate oemof-B3
+5. Install oemof-B3 package using poetry, via:
+
+       poetry install
 
 Alternatively, you can create a virtual environment using other approaches, such as `virtualenv`.
 
@@ -46,7 +64,7 @@ If environment is build otherwise, pandoc must be installed manually. It can be
 
 For the optimization, oemof-B3 needs a solver. Check out the [oemof.solph](https://oemof-solph.readthedocs.io/en/latest/readme.html#installing-a-solver) documentation for installation notes.
 
-To test if everything works, you can run the examples (cf. :ref:`How to run the model`).
+To test if everything works, you can run the [examples](https://oemof-b3.readthedocs.io/en/latest/examples.html).
 
 For developers: Please activate pre-commit hooks (via `pre-commit install`) in order to follow our coding styles.
 

docs/examples.rst

@@ -27,7 +27,7 @@ The examples are limited to the electricity sector, the heat sector is not model
 
 Data for capacities of wind, photovoltaic and biomass as well as electricity demand have been taken
 from the grid development plan
-(`Netzentwicklungsplan <https://www.netzentwicklungsplan.de/sites/default/files/paragraphs-files/NEP_2035_V2021_1_Entwurf_Teil1.pdf>`_, p. 41 ff.)
+(`Netzentwicklungsplan <https://www.netzentwicklungsplan.de/sites/default/files/2022-11/NEP_2035_V2021_1_Entwurf_Teil1_1.pdf>`_, p. 41 ff.)
 :cite:`NEP2021_Entwurf_1`.
 Data for efficiencies and costs (specific annuity, fuel costs) are based on different sources as well
 as on own assumptions.

docs/getting_started.rst

@@ -11,15 +11,49 @@ Using oemof-B3
 Installation
 ------------
 
-Currently, oemof-B3 needs python 3.7 or 3.8 (newer versions may be supported, but installation can take very long).
+Currently, oemof-B3 needs python 3.8, 3.9 or 3.10 (newer versions may be supported, but installation can take very long).
 
-In order to install oemof-B3, proceed with the following steps:
+Additionally, you need to install the python dependency manager `poetry <https://python-poetry.org/>`_.
+It is recommended to install poetry system-wide via the command below or
+`pipx <https://python-poetry.org/docs/#installing-with-pipx>`_:
 
-- git-clone oemof-B3 into local folder: `git clone https://github.com/rl-institut/oemof-B3.git`
-- enter folder
-- create virtual environment using conda: `conda env create environment.yml`
-- activate environment: `conda activate oemof-B3`
-- install oemof-B3 package using poetry, via: `poetry install`
+::
+
+    curl -sSL https://install.python-poetry.org | python3 -
+    poetry install
+
+
+**In order to install oemof-B3, proceed with the following steps:**
+
+1. Clone oemof-B3 into local folder:
+
+::
+
+    git clone git@github.com:rl-institut/oemof-B3.git
+
+2. Enter folder
+
+::
+
+    cd oemof-B3
+
+3. Create virtual environment using conda:
+
+::
+
+    conda env create environment.yml
+
+4. Activate environment:
+
+::
+
+    conda activate oemof-B3
+
+5. Install oemof-B3 package using poetry, via:
+
+::
+
+    poetry install
 
 Alternatively, you can create a virtual environment using other approaches, such as `virtualenv`.
 
@@ -163,7 +197,7 @@ of the componentes in the subdirectory oemof_b3 can be done.
 But you can also modify oemof_b3 to define your own energy system of another city or district.
 For all these use cases, the data in the raw directory
 must be adapted. For this purpose, it is advisable to have energy system-specific empty scalar data
-and time series created for each scenario.
+and time series created for each scenario. See further information in :ref:`How to customize oemof-B3`.
 
 Executing the rule
 

docs/model_pipeline.rst

@@ -100,7 +100,7 @@ A few more conventions are important to know:
 * The capacity of specific components of a scenario can be expanded during an optimization, if the facade attribute `expandable` is set on `True`.
 
   * To adjust this setting, the attribute needs to be specified in the file :attr:`costs_efficiencies.csv`.
-  * An overview of all facades, which includes `expandable` can be found in the facade attributes overview <https://oemof-tabular.readthedocs.io/en/latest/facades.html#>`_ provided by oemof.tabular.
+  * An overview of all facades, which includes `expandable`, can be found in the `facade attributes overview <https://oemof-tabular.readthedocs.io/en/latest/facades.html#>`_ provided by oemof.tabular.
 
 * There is a distinction between the maximum capacity and the already installed capacity of specific components.
   Both of the two attributes can be set individually:

environment.yml

@@ -3,7 +3,7 @@ channels:
   - defaults
   - conda-forge
 dependencies:
-  - python=3.8
+  - python=3.10
   - conda-forge::pandoc>=2.17
   - pip
   - pip:

examples/example_base/preprocessed/data/elements/electricity-shortage.csv

@@ -1,3 +1,3 @@
 region,name,type,carrier,tech,bus,capacity,marginal_cost,output_parameters
-BE,BE-electricity-shortage,shortage,electricity,shortage,BE-electricity,,100000,{}
-BB,BB-electricity-shortage,shortage,electricity,shortage,BB-electricity,,100000,{}
+BE,BE-electricity-shortage,shortage,electricity,shortage,BE-electricity,10000000,410,{}
+BB,BB-electricity-shortage,shortage,electricity,shortage,BB-electricity,10000000,410,{}

examples/example_more_re/preprocessed/data/elements/electricity-shortage.csv

@@ -1,3 +1,3 @@
 region,name,type,carrier,tech,bus,capacity,marginal_cost,output_parameters
-BE,BE-electricity-shortage,shortage,electricity,shortage,BE-electricity,,100000,{}
-BB,BB-electricity-shortage,shortage,electricity,shortage,BB-electricity,,100000,{}
+BE,BE-electricity-shortage,shortage,electricity,shortage,BE-electricity,10000000,410,{}
+BB,BB-electricity-shortage,shortage,electricity,shortage,BB-electricity,10000000,410,{}

examples/example_more_re_less_fossil/preprocessed/data/elements/electricity-shortage.csv

@@ -1,3 +1,3 @@
 region,name,type,carrier,tech,bus,capacity,marginal_cost,output_parameters
-BE,BE-electricity-shortage,shortage,electricity,shortage,BE-electricity,,100000,{}
-BB,BB-electricity-shortage,shortage,electricity,shortage,BB-electricity,,100000,{}
+BE,BE-electricity-shortage,shortage,electricity,shortage,BE-electricity,10000000,410,{}
+BB,BB-electricity-shortage,shortage,electricity,shortage,BB-electricity,10000000,410,{}

oemof_b3/schema/__init__.py

@@ -41,7 +41,7 @@ def write_schema_to_metadata(schema, metadata, num_resource=0):
         "description"
     ] = schema.index["description"]
 
-    for i, (name, (type, description)) in enumerate(schema.columns.iteritems()):
+    for i, (name, (type, description)) in enumerate(schema.columns.items()):
         _metadata["resources"][num_resource]["schema"]["fields"][i + 1]["name"] = name
         _metadata["resources"][num_resource]["schema"]["fields"][i + 1]["type"] = type
         _metadata["resources"][num_resource]["schema"]["fields"][i + 1][

oemof_b3/tools/data_processing.py

@@ -248,7 +248,12 @@ def save_df(df, path):
         Path to save the csv file
     """
     # Save scalars to csv file
-    df.to_csv(path, index=True, sep=config.settings.general.separator)
+    df.to_csv(
+        path,
+        index=True,
+        sep=config.settings.general.separator,
+        date_format="%Y-%m-%d %H:%M:%S",
+    )
 
     # Print user info
     logger.info(f"The DataFrame has been saved to: {path}.")
@@ -517,7 +522,7 @@ def aggregate_scalars(df, columns_to_aggregate, agg_method=None):
     if not agg_method:
         agg_method = {
             "var_value": sum,
-            "name": lambda x: "None",
+            "name": lambda x: np.nan,
             "var_unit": aggregate_units,
         }
 
@@ -1254,7 +1259,10 @@ def _get_direction(oemof_tuple):
 
 def _get_region_carrier_tech_from_component(component, delimiter="-"):
 
-    if isinstance(component, oemof.tabular.facades.Facade):
+    typemap_facades = oemof.tabular.facades.TYPEMAP
+    typemap_values = list(typemap_facades.values())
+
+    if isinstance(component, classmethod) and (component in typemap_values):
         region = component.region
         carrier = component.carrier
         tech = component.tech