diff --git a/.github/workflows/pytest.yaml b/.github/workflows/pytest.yaml index 4abe232f3..0c020f100 100644 --- a/.github/workflows/pytest.yaml +++ b/.github/workflows/pytest.yaml @@ -209,4 +209,4 @@ jobs: run: gh cache list -L 999 | cut -f2 | grep pre-commit | xargs -I{} gh cache delete "{}" || true env: { GH_TOKEN: "${{ github.token }}" } - - uses: pre-commit/action@v3.0.0 + - uses: pre-commit/action@v3.0.1 diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 1d326c7ed..43d6663db 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -10,7 +10,7 @@ repos: language: python entry: bash -c ". ${PRE_COMMIT_MYPY_VENV:-/dev/null}/bin/activate 2>/dev/null; mypy $0 $@" additional_dependencies: - - mypy >= 1.8.0 + - mypy >= 1.9.0 - asyncssh - git+https://github.com/iiasa/ixmp.git@main - importlib_resources @@ -20,7 +20,7 @@ repos: - types-requests args: ["."] - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.3.0 + rev: v0.3.2 hooks: - id: ruff - id: ruff-format diff --git a/RELEASE_NOTES.rst b/RELEASE_NOTES.rst index 98f22413c..c9b30a43e 100644 --- a/RELEASE_NOTES.rst +++ b/RELEASE_NOTES.rst @@ -4,6 +4,7 @@ Next release All changes ----------- +- Update tutorial Westeros multinode to include code-based hints for in-depth questions (:pull:`798`). - :func:`.make_df` can now create partly-filled :class:`DataFrames ` for indexed sets; not only parameters (:pull:`784`). - New function :func:`.util.copy_model` that exposes the behaviour of the :program:`message-ix copy-model` CLI command to other Python code (:pull:`784`). - New test fixture :func:`.tmp_model_dir` (:pull:`784`). diff --git a/message_ix/tests/test_tutorials.py b/message_ix/tests/test_tutorials.py index 6c14546a2..edf6facc2 100644 --- a/message_ix/tests/test_tutorials.py +++ b/message_ix/tests/test_tutorials.py @@ -85,7 +85,7 @@ def _t(group: Union[str, None], basename: str, *, check=None, marks=None): _t("w0", f"{W}_soft_constraints"), _t("w0", f"{W}_addon_technologies"), _t("w0", f"{W}_historical_new_capacity"), - _t("w0", f"{W}_multinode"), + _t("w0", f"{W}_multinode_energy_trade"), # NB this is the same value as in test_reporter() _t(None, f"{W}_report", check=[("len-rep-graph", 13724)]), _t("at0", "austria", check=[("solve-objective-value", 206321.90625)]), diff --git a/tutorial/README.rst b/tutorial/README.rst index fd01615d1..5ea76bdf0 100644 --- a/tutorial/README.rst +++ b/tutorial/README.rst @@ -156,7 +156,7 @@ framework, such as used in global research applications of |MESSAGEix|. system (:tut:`westeros/westeros_historical_new_capacity.ipynb`). #. Modeling of a multi-node energy system and representing trade between nodes - (:tut:`westeros/westeros_multinode.ipynb`). + (:tut:`westeros/westeros_multinode_energy_trade.ipynb`). #. Use other features of :mod:`message_ix` and :mod:`ixmp`: diff --git a/tutorial/westeros/westeros_multinode.ipynb b/tutorial/westeros/westeros_multinode_energy_trade.ipynb similarity index 91% rename from tutorial/westeros/westeros_multinode.ipynb rename to tutorial/westeros/westeros_multinode_energy_trade.ipynb index 6c3497f1b..fa2548617 100644 --- a/tutorial/westeros/westeros_multinode.ipynb +++ b/tutorial/westeros/westeros_multinode_energy_trade.ipynb @@ -1,7 +1,6 @@ { "cells": [ { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -39,29 +38,22 @@ } ], "source": [ - "# Importing required software packages\n", - "import pandas as pd\n", - "from itertools import product\n", - "import ixmp\n", - "import message_ix\n", - "\n", - "from message_ix.util import make_df\n", - "\n", "%matplotlib inline" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ + "import ixmp\n", + "\n", "# Loading the modeling platform\n", "mp = ixmp.Platform()" ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -74,8 +66,18 @@ "cell_type": "code", "execution_count": 3, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "This Scenario has a solution, use `Scenario.remove_solution()` or `Scenario.clone(..., keep_solution=False)`\n" + ] + } + ], "source": [ + "import message_ix\n", + "\n", "# Loading baseline scenario\n", "model = \"Westeros Electrified\"\n", "scenario = \"baseline\"\n", @@ -169,17 +171,11 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### 1.2. Populating data for the new nodes\n", - "By now, the model has no data related to these nodes, e.g., no technology parameter is defined for them. Here, we use a simple function to copy all the data of \"Westeros\" to new nodes. This way, we do not have to specify all the input data as we did for \"Westeros\" in the ``westeros_baseline.ipynb`` tutorial.\n", - "\n", - "\n", - "> **NOTE:** There is an [open issue #601](https://github.com/iiasa/message_ix/issues/601) related to `message_ix.Scenario.rename()`. After solving this issue, you can simply use the follwoing line for copying data of \"Westeros\" to \"Essos\", instead of the code in the next cell.\n", - "> \n", - "> `scen.rename(\"node\", {\"Westeros\": \"Essos\"}, keep=True)` " + "By now, the model has no data related to these new nodes, e.g., no technology parameter is defined for them. Here, we can copy the data of \"Westeros\" to the new nodes. This way, we do not have to repeat all the steps for adding input data as we did for \"Westeros\" in ``westeros_baseline.ipynb`` for each new node. We can simply use `message_ix.rename()` feature and keep the original node (\"Westeros\") by using `keep=True`." ] }, { @@ -188,21 +184,9 @@ "metadata": {}, "outputs": [], "source": [ - "# Copying data of \"Westeros\" to other nodes\n", - "for parname, node in product(scen.par_list(), nodes_new):\n", - " \n", - " # If there is no \"node\" specified for this parameter, ignore copying data\n", - " if \"node\" not in scen.idx_sets(parname):\n", - " continue\n", - " \n", - " # Finding indexes related to \"node\" in this parameter\n", - " node_columns = [x for x in scen.idx_names(parname) if \"node\" in x]\n", - " \n", - " # Replace \"Westeros\" with new node names in columns related to \"node\"\n", - " df = scen.par(parname)\n", - " for node_column in node_columns:\n", - " df[node_column] = df[node_column].replace({\"Westeros\": node}) \n", - " scen.add_par(parname, df)" + "# Copying data from \"Westeros\" to the new nodes\n", + "for node in nodes_new:\n", + " scen.rename(\"node\", {\"Westeros\": node}, keep=True)" ] }, { @@ -307,7 +291,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 10, "metadata": {}, "outputs": [], "source": [ @@ -457,7 +441,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -477,13 +460,12 @@ "metadata": {}, "outputs": [], "source": [ - "# We clone a new scenario and call it \"scen2\" \n", + "# We clone a new scenario and call it \"scen2\"\n", "scen2 = scen.clone(scenario=\"multinode_hub\", keep_solution=False)\n", "scen2.check_out()" ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -516,7 +498,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -581,7 +562,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -595,6 +575,8 @@ "metadata": {}, "outputs": [], "source": [ + "from message_ix.util import make_df\n", + "\n", "# Parametrization of \"input\" for import technologies\n", "# The origin of import is the level of \"trade\" from hub\n", "model_years = list(scen2.set(\"year\"))\n", @@ -646,7 +628,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -662,7 +643,9 @@ "source": [ "# The origin of export is the level of \"secondary\" in each country\n", "for country in [\"Westeros\", \"Essos\", \"Stepstones\"]:\n", - " base_input.update(dict(technology=\"elec_exp\", node_origin=country, level=\"secondary\", value=1.05))\n", + " base_input.update(\n", + " dict(technology=\"elec_exp\", node_origin=country, level=\"secondary\", value=1.05)\n", + " )\n", " inp = make_df(\"input\", **base_input, node_loc=country)\n", " scen2.add_par(\"input\", inp)" ] @@ -700,20 +683,21 @@ " # Loading data of \"demand\"\n", " df = scen2.par(\"demand\", {\"node\": node})\n", " # Multiplying by the value\n", - " df[\"value\"] *= (1 + value)\n", + " df[\"value\"] *= 1 + value\n", " # Adding new demand to the scenario\n", " scen2.add_par(\"demand\", df)" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "# Committing and solving\n", "scen2.commit(\"new nodes added\")\n", - "scen2.solve()" + "scen2.solve()\n", + "scen2.set_as_default()" ] }, { @@ -859,7 +843,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -873,7 +856,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1119,7 +1101,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1137,7 +1118,7 @@ "metadata": {}, "outputs": [], "source": [ - "# We clone a new scenario and call it \"scen3\" \n", + "# We clone a new scenario and call it \"scen3\"\n", "scen3 = scen2.clone(scenario=\"multinode_bilateral\", keep_solution=False)\n", "scen3.check_out()" ] @@ -1179,7 +1160,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1203,7 +1183,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1292,7 +1271,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1310,7 +1288,9 @@ "# Origin is Westeros\n", "# We assign the \"mode\" of \"standard\" as there is only one direction\n", "scen3.add_set(\"technology\", \"elec_export\")\n", - "base_input.update(dict(node_origin=\"Westeros\", technology=\"elec_export\", mode=\"standard\"))\n", + "base_input.update(\n", + " dict(node_origin=\"Westeros\", technology=\"elec_export\", mode=\"standard\")\n", + ")\n", "inp = make_df(\"input\", **base_input, node_loc=\"hub\")\n", "scen3.add_par(\"input\", inp)" ] @@ -1330,7 +1310,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 35, "metadata": {}, "outputs": [], "source": [ @@ -1539,7 +1519,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -1548,11 +1527,73 @@ "\n", "\n", "\n", - "In this exercise, you should establish trade with an external market in the Westeros multi-node model. This trade link can be through a hub or directly from a node. You can specify the price of electricity ($/kWa) in the external market in 700-720 as follows: {700: 97, 710: 93, 720: 87}.\n", + "In this exercise, you should establish trade with an external market in the Westeros multi-node model. This trade link can be through a hub or directly from a node. You can specify the price of electricity ($/kWa) in the external market as follows: {690: 100, 700: 97, 710: 93, 720: 87}." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
\n", + " Click for hints\n", + " \n", + " * We can model this by adding an electricity importing technology that imports electricity to the node \"hub\". This technology, however, has no `input` parameter (because we do not model the commodity balance outside of the boundary of our model).\n", + " However, we can define `var_cost` for this trade link to represent the price of commodity in the external market.\n", + " \n", + "
\n", + " Click for the code\n", "\n", - "(Hint: we can model this with an importing technology that imports electricity to the node \"hub\". This technology, however, has no `input` parameter (because we do not model the commodity balance outside of the boundary of our model). However, we can define `var_cost` for this trade link to represent the price of commodity in the external market.)\n", + "```python\n", + "# We clone a new scenario from the scenario with a central hub.\n", + "scen2 = message_ix.Scenario(mp, model, \"multinode_hub\")\n", + "scen4 = scen2.clone(scenario=\"multinode_hub_external_trade\", keep_solution=False)\n", + "scen4.check_out()\n", + "# We link the central hub to an external electricity market.\n", + "# First, we add a new technology:\n", + "scen4.add_set(\"technology\", \"elec_external_imp\")\n", + "# Then, we define parameter \"output\" for importing electricity to \"hub\"\n", + "# Here, we don't need to define \"input\", as this is considered outside the boundary\n", + "# conditions of our model\n", + "base_output = {\n", + " \"node_dest\": \"hub\",\n", + " \"node_loc\": \"hub\",\n", + " \"technology\": \"elec_external_imp\",\n", + " \"commodity\": \"electricity\",\n", + " \"level\": \"trade\",\n", + " \"year_vtg\": model_years,\n", + " \"year_act\": model_years,\n", + " \"mode\": \"standard\",\n", + " \"time\": \"year\",\n", + " \"time_dest\": \"year\",\n", + " \"value\": 1,\n", + " \"unit\": \"-\",\n", + "}\n", + "# We add this data to the scenario\n", + "scen4.add_par(\"output\", make_df(\"output\", **base_output))\n", + "# At this stage, imported electricity has no cost. We define variable cost for that as\n", + "# asked in the question.\n", + "base_output.update({\"value\": [100, 97, 93, 87], \"unit\": \"USD/kWa\"})\n", + "scen4.add_par(\"var_cost\", make_df(\"var_cost\", **base_output))\n", + "scen4.commit(\"\")\n", + "# Finally, we can solve the scenario\n", + "scen4.solve()\n", + "# Let's check the price of electricity\n", + "print(\n", + " \"Price of electricity:\\n\",\n", + " scen4.var(\"PRICE_COMMODITY\", {\"commodity\": \"electricity\", \"node\": \"hub\"}),\n", + ")\n", + "# We observe that the price of electricity in years 710 and 720 is set by the cost of\n", + "# importing electricity. But in 700 the cost of trading electricity at the internal hub\n", + "# is lower than the cost of importing. Hence, if we check the activity of the import\n", + "# technology, it should not show any import in 700 (see column \"lvl\"):\n", + "print(\n", + " \"Activity of the electricity importing technology:\\n\",\n", + " scen4.var(\"ACT\", {\"technology\": \"elec_external_imp\"}),\n", + ")\n", + "```\n", "\n", - "Self-supervised students who would like to see a solution for this exercise: please contact the author." + "
\n", + "
" ] }, { @@ -1568,7 +1609,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -1582,7 +1623,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.6" + "version": "3.10.12" } }, "nbformat": 4,