Merge pull request PyPSA#640 from PyPSA/configs-in-one-folder

Configs in one folder

.github/workflows/ci.yaml

@@ -103,9 +103,9 @@ jobs:
       run: |
         conda activate pypsa-eur
         conda list
-        snakemake -call solve_elec_networks --configfile test/config.electricity.yaml --rerun-triggers=mtime
-        snakemake -call all --configfile test/config.overnight.yaml --rerun-triggers=mtime
-        snakemake -call all --configfile test/config.myopic.yaml --rerun-triggers=mtime
+        snakemake -call solve_elec_networks --configfile config/test/config.electricity.yaml --rerun-triggers=mtime
+        snakemake -call all --configfile config/test/config.overnight.yaml --rerun-triggers=mtime
+        snakemake -call all --configfile config/test/config.myopic.yaml --rerun-triggers=mtime
 
     - name: Upload artifacts
       uses: actions/upload-artifact@v3

.syncignore-receive

@@ -18,3 +18,4 @@ cutouts
 data
 benchmarks
 *.nc
+configs

Snakefile

@@ -14,11 +14,11 @@ from snakemake.utils import min_version
 min_version("7.7")
 
 
-if not exists("config.yaml"):
-    copyfile("config.default.yaml", "config.yaml")
+if not exists("config/config.yaml"):
+    copyfile("config/config.default.yaml", "config/config.yaml")
 
 
-configfile: "config.yaml"
+configfile: "config/config.yaml"
 
 
 COSTS = f"data/costs_{config['costs']['year']}.csv"

doc/configuration.rst

@@ -9,14 +9,14 @@
 Configuration
 ##########################################
 
-PyPSA-Eur has several configuration options which are documented in this section and are collected in a ``config.yaml`` file located in the root directory. Users should copy the provided default configuration (``config.default.yaml``) and amend their own modifications and assumptions in the user-specific configuration file (``config.yaml``); confer installation instructions at :ref:`defaultconfig`.
+PyPSA-Eur has several configuration options which are documented in this section and are collected in a ``config/config.yaml`` file located in the root directory. Users should copy the provided default configuration (``config/config.default.yaml``) and amend their own modifications and assumptions in the user-specific configuration file (``config/config.yaml``); confer installation instructions at :ref:`defaultconfig`.
 
 .. _toplevel_cf:
 
 Top-level configuration
 =======================
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :lines: 5-11,18-19,62,80-90
 
@@ -37,7 +37,7 @@ investment changes as more ambitious greenhouse-gas emission reduction targets a
 
 The ``run`` section is used for running and storing scenarios with different configurations which are not covered by :ref:`wildcards`. It determines the path at which resources, networks and results are stored. Therefore the user can run different configurations within the same directory. If a run with a non-empty name should use cutouts shared across runs, set ``shared_cutouts`` to `true`.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: run:
    :end-before: foresight:
@@ -76,7 +76,7 @@ An exemplary dependency graph (starting from the simplification rules) then look
 
 .. image:: img/scenarios.png
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: scenario:
    :end-before: countries:
@@ -93,7 +93,7 @@ An exemplary dependency graph (starting from the simplification rules) then look
 
 Specifies the temporal range to build an energy system model for as arguments to `pandas.date_range <https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.date_range.html>`_
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: snapshots:
    :end-before: enable:
@@ -110,7 +110,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 Switches for some rules and optional features.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: enable:
    :end-before: co2_budget:
@@ -125,7 +125,7 @@ Switches for some rules and optional features.
 ``electricity``
 ===============
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: electricity:
    :end-before: atlite:
@@ -142,7 +142,7 @@ Switches for some rules and optional features.
 
 Define and specify the ``atlite.Cutout`` used for calculating renewable potentials and time-series. All options except for ``features`` are directly used as `cutout parameters <https://atlite.readthedocs.io/en/latest/ref_api.html#cutout>`_.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: atlite:
    :end-before: renewable:
@@ -160,7 +160,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
 ``onwind``
 ----------
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: renewable:
    :end-before:   offwind-ac:
@@ -173,7 +173,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
 ``offwind-ac``
 --------------
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at:   offwind-ac:
    :end-before:   offwind-dc:
@@ -186,7 +186,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
 ``offwind-dc``
 ---------------
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at:   offwind-dc:
    :end-before:   solar:
@@ -199,7 +199,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
 ``solar``
 ---------------
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at:   solar:
    :end-before:   hydro:
@@ -212,7 +212,7 @@ Define and specify the ``atlite.Cutout`` used for calculating renewable potentia
 ``hydro``
 ---------------
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at:   hydro:
    :end-before: conventional:
@@ -234,7 +234,7 @@ with country specific values. Then, the values are read in and applied to all
 generators of the given carrier in the given country. Note that the value(s)
 overwrite the existing values.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at:   conventional:
    :end-before: lines:
@@ -247,7 +247,7 @@ overwrite the existing values.
 ``lines``
 =============
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: lines:
    :end-before: links:
@@ -262,7 +262,7 @@ overwrite the existing values.
 ``links``
 =============
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: links:
    :end-before: transformers:
@@ -277,7 +277,7 @@ overwrite the existing values.
 ``transformers``
 ================
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: transformers:
    :end-before: load:
@@ -292,7 +292,7 @@ overwrite the existing values.
 ``load``
 =============
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-after:   type:
    :end-at:   scaling_factor:
@@ -307,7 +307,7 @@ overwrite the existing values.
 ``costs``
 =============
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: costs:
    :end-before: clustering:
@@ -323,7 +323,7 @@ overwrite the existing values.
 ``clustering``
 ==============
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: clustering:
    :end-before: solving:
@@ -345,7 +345,7 @@ overwrite the existing values.
 .. warning::
    More comprehensive documentation for this segment will be released soon.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: energy:
    :end-before: biomass:
@@ -362,7 +362,7 @@ overwrite the existing values.
 .. warning::
    More comprehensive documentation for this segment will be released soon.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: biomass:
    :end-before: solar_thermal:
@@ -378,7 +378,7 @@ overwrite the existing values.
 .. warning::
    More comprehensive documentation for this segment will be released soon.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: solar_thermal:
    :end-before: existing_capacities:
@@ -394,7 +394,7 @@ overwrite the existing values.
 .. warning::
    More comprehensive documentation for this segment will be released soon.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: existing_capacities:
    :end-before: sector:
@@ -410,7 +410,7 @@ overwrite the existing values.
 .. warning::
    More comprehensive documentation for this segment will be released soon.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: sector:
    :end-before: industry:
@@ -426,7 +426,7 @@ overwrite the existing values.
 .. warning::
    More comprehensive documentation for this segment will be released soon.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: industry:
    :end-before: costs:
@@ -436,7 +436,7 @@ overwrite the existing values.
 ``solving``
 =============
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: solving:
    :end-before: plotting:
@@ -459,7 +459,7 @@ overwrite the existing values.
 .. warning::
    More comprehensive documentation for this segment will be released soon.
 
-.. literalinclude:: ../config.default.yaml
+.. literalinclude:: ../config/config.default.yaml
    :language: yaml
    :start-at: plotting:
 

doc/costs.rst

@@ -9,7 +9,7 @@ Techno-Economic Assumptions
 
 The database of cost assumptions is retrieved from the repository
 `PyPSA/technology-data <https://github.com/pypsa/technology-data>`_ and then
-saved to a file ``data/costs_{year}.csv``. The ``config.yaml`` provides options
+saved to a file ``data/costs_{year}.csv``. The ``config/config.yaml`` provides options
 to choose a reference year and use a specific version of the repository.
 
 .. literalinclude:: ../config.default.yaml
@@ -48,7 +48,7 @@ Modifying Assumptions
 =====================
 
 Some cost assumptions (e.g. marginal cost and capital cost) can be directly
-set in the ``config.yaml`` (cf. Section  :ref:`costs_cf`  in
+set in the ``config/config.yaml`` (cf. Section  :ref:`costs_cf`  in
 :ref:`config`). To change cost assumptions in more detail, make a copy of
 ``data/costs_{year}.csv`` and reference the new cost file in the ``Snakefile``:
 

doc/foresight.rst

@@ -28,7 +28,7 @@ It does not affect the year for cost and technology assumptions, which is set se
   costs:
     year: 2030
 
-For running overnight scenarios, use in the ``config.yaml``:
+For running overnight scenarios, use in the ``config/config.yaml``:
 
 .. code:: yaml
 
@@ -44,7 +44,7 @@ Perfect foresight scenarios
   Perfect foresight is currently under development and not yet implemented.
 
 For running perfect foresight scenarios, in future versions you will be able to
-set in the ``config.yaml``:
+set in the ``config/config.yaml``:
 
 .. code:: yaml
 
@@ -94,13 +94,13 @@ evolve with the myopic approach:
 Configuration
 --------------
 
-For running myopic foresight transition scenarios, set in ``config.yaml``:
+For running myopic foresight transition scenarios, set in ``config/config.yaml``:
 
 .. code:: yaml
 
   foresight: myopic
 
-The following options included in the config.yaml file  are relevant for the
+The following options included in the ``config/config.yaml`` file  are relevant for the
 myopic code.
 
 The ``{planning_horizons}`` wildcard indicates the year in which the network is
@@ -163,7 +163,7 @@ Options
 The total carbon budget for the entire transition path can be indicated in the
 `sector_opts
 <https://github.com/PyPSA/pypsa-eur-sec/blob/f13902510010b734c510c38c4cae99356f683058/config.default.yaml#L25>`_
-in ``config.yaml``. The carbon budget can be split among the
+in ``config/config.yaml``. The carbon budget can be split among the
 ``planning_horizons`` following an exponential or beta decay. E.g. ``'cb40ex0'``
 splits a carbon budget equal to 40 Gt :math:`_{CO_2}` following an exponential
 decay whose initial linear growth rate r is zero. They can also follow some
@@ -218,7 +218,7 @@ add_brownfield for the remaining planning_horizons.
    ``results/run_name/networks/prenetworks-brownfield``.
 
 Steps 2 and 3 are solved recursively for all the planning_horizons included in
-``config.yaml``.
+``config/config.yaml``.
 
 Rule overview
 --------------

doc/installation.rst

@@ -119,19 +119,19 @@ Handling Configuration Files
 ============================
 
 PyPSA-Eur has several configuration options that must be specified in a
-``config.yaml`` file located in the root directory. An example configuration
-``config.default.yaml`` is maintained in the repository, which will be used to
-automatically create your customisable ``config.yaml`` on first use. More
+``config/config.yaml`` file located in the root directory. An example configuration
+``config/config.default.yaml`` is maintained in the repository, which will be used to
+automatically create your customisable ``config/config.yaml`` on first use. More
 details on the configuration options are in :ref:`config`.
 
 You can also use ``snakemake`` to specify another file, e.g.
-``config.mymodifications.yaml``, to update the settings of the ``config.yaml``.
+``config/config.mymodifications.yaml``, to update the settings of the ``config/config.yaml``.
 
 .. code:: bash
 
-    .../pypsa-eur % snakemake -call --configfile config.mymodifications.yaml
+    .../pypsa-eur % snakemake -call --configfile config/config.mymodifications.yaml
 
 .. warning::
-    Users are advised to regularly check their own ``config.yaml`` against changes
-    in the ``config.default.yaml`` when pulling a new version from the remote
+    Users are advised to regularly check their own ``config/config.yaml`` against changes
+    in the ``config/config.default.yaml`` when pulling a new version from the remote
     repository.

doc/introduction.rst

@@ -74,7 +74,7 @@ what data to retrieve and what files to produce. Details are explained in
 :ref:`wildcards` and :ref:`scenario`.
 
 The model also has several further configuration options collected in the
-``config.yaml`` file located in the root directory, which that are not part of
+``config/config.yaml`` file located in the root directory, which that are not part of
 the scenarios. Options are explained in :ref:`config`.
 
 Folder Structure