getting_started/10_StorageUnits.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# In this notebook you will learn about the storage units in grid2op\n",
    "\n",
    "Try this notebook out interactively with: [![Binder](./img/badge_logo.svg)](https://mybinder.org/v2/gh/rte-france/Grid2Op/master)\n",
    "\n",
    "**Objectives**\n",
    "\n",
    "The objective of this notebook is to describe the action on storage units that are modeled as continuous actions in grid2op, how these actions work, what they do, and how you can apply them.\n",
    "\n",
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" width=\"200\">\n",
    "Execute the cell below by removing the # character if you use google colab !\n",
    "\n",
    "Cell will look like:\n",
    "```python\n",
    "!pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)\n",
    "```\n",
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" width=\"200\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I) Compatible environments\n",
    "\n",
    "First, in order to do action on storage units, storage units must be present on the grid. This is not the case for most grid2op environments. So you might want to check if there are storage units, as shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "import grid2op\n",
    "from tqdm.notebook import tqdm  # for easy progress bar\n",
    "display_tqdm = False  # this is set to False for ease with the unitt test, feel free to set it to True\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "env_name1 = \"l2rpn_case14_sandbox\"\n",
    "env_nok = grid2op.make(env_name1, test=True)\n",
    "print(f\"Can I use action on storage units in environment \\\"{env_name1}\\\": {env_nok.n_storage > 0}\")\n",
    "\n",
    "env_name2 = \"educ_case14_storage\"\n",
    "env = grid2op.make(env_name2, test=True)\n",
    "print(f\"Can I use action on storage units in environment \\\"{env_name2}\\\": {env.n_storage > 0}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## II) What are storage units ?\n",
    "\n",
    "### A) Description\n",
    "Storage units are \"elements\" of a grid that can act sometimes as generators, sometimes as load (depending on what they are told to do). Storage units can basically store a certain quantity of energy (then acting as load) and release this energy to the power system when asked (then acting as generators)\n",
    "\n",
    "The two main types of storage units we can think of are:\n",
    "\n",
    "- \"pumped storage\": they store electric power by pumping it in an upward reservoir and can produce it again by letting the water through a turbine when going downhill (see this [wikipedia article](https://en.wikipedia.org/wiki/Pumped-storage_hydroelectricity) for more information)\n",
    "- \"batteries\": they store energy in a chemical form and can charge/discharge similarly to the battery of a cellphone, but in (way, way) bigger.\n",
    "\n",
    "In grid2op a storage unit is defined by different parameters. The main ones are:\n",
    "- `storage_Emax`: the maximum energy (expressed in MWh) the storage unit can contain. \n",
    "- `storage_Emin`: the minimum energy (in MWh) allowed in the unit (for example some batteries should not be \"emptied\" entirely)\n",
    "- `storage_loss`: the loss (in MW) in the storage unit. This corresponds to the loss of energy that happens continuously. In reality, for example, this can model the \"self discharge\" of a battery or the evaporation of the upper lake in pumped storage. It should not be mixed with the following two attributes.\n",
    "- `storage_charging_efficiency`: this is the efficiency when the storage unit is charged. This efficiency corresponds to the amount of energy that will be stored in the unit if a power of 1MW is taken from the grid to charge it during 1 hour. It has no unit and should be between 0 and 1.\n",
    "- `storage_discharging_efficiency`: this is the efficiency when the storage unit is discharged. This efficiency corresponds to the amount of energy that will be subtracted from the unit if a power of 1MW is injected into the powergrid during 1 hour. It has no unit and should be between 0 and 1.\n",
    "- `storage_max_p_prod`: the maximum value (still seen from the grid) that a storage unit can inject into the grid. It is expressed in MW.\n",
    "- `storage_max_p_absorb`: the maximum value (still from the grid point of view) that a storage unit can absorb from the grid, expressed also in MW.\n",
    "\n",
    "The [official documentation](https://grid2op.readthedocs.io/en/latest/modeled_elements.html#storage-units-optional) gives more details about all these attributes and some others too. In the following cell, we give an example of how to access such attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.storage_Emax\n",
    "env.storage_Emin\n",
    "env.storage_loss\n",
    "env.storage_charging_efficiency\n",
    "env.storage_discharging_efficiency"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main usage for storage units in grid2op is to assign them a setpoint of power they will absorb (or produce) during a time step.\n",
    "\n",
    "Grid2op handles the conversion of this power (seen from the grid) into the energy stored in the unit taking into account the losses and inefficiencies.\n",
    "\n",
    "\n",
    "### B) Convention\n",
    "\n",
    "There are different conventions to model power grid elements.\n",
    "\n",
    "For the storage unit, we adopted the \"load convention\". In short, this means that:\n",
    "- if a positive power setpoint is given, then the storage will behave like a load. It will absorb power from the grid. It will recharge.\n",
    "- if this same power is negative, then the storage will behave like a generator. It will inject power into the grid. It will discharge.\n",
    "\n",
    "## III) Actions on storage units\n",
    "\n",
    "Like any other grid2op object, storage units are modified through actions. The only modification you can do with storage units is to give a setpoint of how much power you want the storage unit to absorb/produce. This action is done with the \"*storage_p*\" keys.\n",
    "\n",
    "In the next cell, we will ask the storage unit 0 to inject 2.7MW into the grid and the storage unit 1  to absorb 3.14 MW from the grid."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create the action described above\n",
    "# method 1, with the \"dictionnary\" action comprehension\n",
    "storage_act1 = env.action_space({\"set_storage\": [(0, -2.7), (1, 3.14)]})\n",
    "\n",
    "# method 2, with the \"property\"\n",
    "storage_act2 = env.action_space()\n",
    "storage_act2.storage_p = [(0, -2.7), (1, 3.14)]\n",
    "\n",
    "# or alternatively, you can pass it a full vector:\n",
    "storage_setpoint = np.zeros(env.n_storage, dtype=float)\n",
    "storage_setpoint[0] = -2.7\n",
    "storage_setpoint[1] = 3.14\n",
    "storage_act3 = env.action_space({\"set_storage\": storage_setpoint})\n",
    "\n",
    "# the same things with the property:\n",
    "storage_act4 = env.action_space()\n",
    "storage_act4.storage_p = storage_setpoint\n",
    "\n",
    "# all the above actions are equivalent. And you can print them:\n",
    "print(storage_act4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IV) Storage units in the observation\n",
    "There exist a lot of information given in the observation concerning the storage units, the complete list of attributes you can retrieve is explained in the [official documentation](https://grid2op.readthedocs.io/en/latest/observation.html#main-observation-attributes).\n",
    "\n",
    "The most important information:\n",
    "- `storage_charge`: the current \"charge\" of each storage units (given in MWh)\n",
    "- `storage_power_target`: the setpoint given by the observation to the storage units (in MW)\n",
    "- `storage_power`: the actual power produced / absorbed (still seen from the grid) for every storage units\n",
    "\n",
    "The following \"properties\" are met:\n",
    "- `storage_charge` is decreasing (due to `storage_loss`) if the storage units are not charged\n",
    "- `storage_power_target` corresponds to the storage action given in the previous step by the agent\n",
    "- `storage_power` may be different than the target, for example, if the storage units are totally discharged, and you ask the storage unit to continue producing power\n",
    "- `storage_power` and `storage_power_target` are both vectors containing only 0 if no actions are performed on the storage units.\n",
    "\n",
    "A simple example is given below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I do not do any action, storage power, and storage_power_target are all 0\n",
    "obs_init = env.reset()\n",
    "obs1, reward1, done1, info1 = env.step(env.action_space())\n",
    "print(f\"The `storage_power` when no actions on storage units is performed is {obs1.storage_power}\")\n",
    "\n",
    "# I perform the action described above\n",
    "obs2, reward2, done2, info2 = env.step(storage_act1)\n",
    "print(f\"The `storage_power` after the action described above is {obs2.storage_power}\")\n",
    "\n",
    "# Computing the amount of energy stored in the unit is not trivial. Indeed, each step is (for this environment)\n",
    "# the equivalent of 5 mins. And if you ask 3.14 MW for 5mins, the charge will not\n",
    "# be reduced by 3.14 MWh but rather by 3.14 / 60 * 5 MWh as can be seen here:\n",
    "print(f\"The initial charge in the storage unit 1 was: {obs_init.storage_charge[1]:.3f}MWh\")\n",
    "print(f\"And after the action on this storage, it is: {obs2.storage_charge[1]:.3f}\")\n",
    "print(f\"And we have: 3.14/60*5={3.14/60*5:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Oh, what is happening here? We should have a charge of 3.50MWh + 0.26 MWh = 7.76 MWh. Why do we get only 3.74 MWh?\n",
    "\n",
    "This is because the storage has some losses: even if you did nothing with the storage unit, it will dissipate 0.1MW each time. See section [II) What are storage units ?](#II\\)-What-are-storage-units-?) for more details.\n",
    "\n",
    "This means that, every 5 minutes, the storage unit will dissipate 0.1 / 60 * 5 = 0.00833333... MWh of energy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"The initial energy storage in the unit 0 was: {obs_init.storage_charge[1]:.6f}MWh\")\n",
    "print(f\"After doing nothing on this unit, it is: {obs1.storage_charge[1]:.6f}MWh\")\n",
    "print(f\"As you see, the energy stored decrease of 0.1 / 60 * 5 = {0.1 / 60 * 5:.6f}MWh / step\")\n",
    "print(f\"This explains that after doing nothing, then absorbing 3.14MW, the charge of the storage unit 1 is:\\n \"\n",
    "      f\"\\t\\t 3.50 - (0.1/60*5) + ((3.14/60*5) -(0.1/60*5)) = {obs2.storage_charge[1]:.3f} MWh \"\n",
    "     )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB** The formula above is true as for storage 1, the charging efficiency is 1.0. This would have been slightly modified (and be more complicated) with a charging efficiency different from 1.0. More details about this more complex case are given [in the documentation](https://grid2op.readthedocs.io/en/latest/modeled_elements.html#storage-units-optional) especially the sub section [Satisfied equations](https://grid2op.readthedocs.io/en/latest/modeled_elements.html#modeled_elements.html#id19) (of the description of the storage units).\n",
    "\n",
    "**NB** As opposed to curtailment or redispatching, storage unit actions do not \"cumulate\". An action that you do at a given step will affect only the next step. Therefore, storage unit actions do not last with time. An example is given in the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"I do a storage action\")\n",
    "obs3, reward3, done3, info3 = env.step(storage_act1)\n",
    "print(f\"The setpoint for storage unit 1 is indeed: {obs3.storage_power_target[1]:.2f}MW\")\n",
    "print(f\"And the charge of this unit is {obs3.storage_charge[1]:.2f}MWh\")\n",
    "print(\"Then I do nothing\")\n",
    "obs4, reward4, done4, info4 = env.step(env.action_space())\n",
    "print(f\"The setpoint for storage unit 1 is indeed: {obs4.storage_power_target[1]:.2f}MW\")\n",
    "print(f\"And the charge of this unit is {obs4.storage_charge[1]:.2f}MWh\")\n",
    "print(\"(the difference in the charge is due to the losses in the storage units)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### V) Side effects of using storage units\n",
    "\n",
    "As always with grid2op, we assume that the market (or central authority) already adjusted the energy generation to the load to reach a balance. Therefore, at each step, if nothing is done, the total load can be powered by the total generation without the intervention of any \"agent\" in grid2op.\n",
    "\n",
    "This fact above has some implications when using storage units.  For example, if you decide to act on the storage unit, then you will either increase the load (if you charge the units) or the generation (if you discharge them). With this action, you will affect the above balance which in response becomes inbalanced.\n",
    "\n",
    "To restore the balance between total generation, and total demand, as in the case of curtailment, dispatchable generators are used. In fact, if you ask for a storage action that does not sum to 0, then automatically, the environment will perform some dispatch on the generators. \n",
    "\n",
    "This behavior is explained below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"I do a storage action that sums in total to {storage_act1.storage_p.sum():.2f} MW\")\n",
    "obs5, reward5, done5, info5 = env.step(storage_act1)\n",
    "print(f\"And the sum of redispatching at this step is {obs5.actual_dispatch.sum():.2f} MW\")\n",
    "\n",
    "print(\"\\nBut if I do nothing the next step, then we will have:\")\n",
    "obs6, reward6, done6, info6 = env.step(env.action_space())\n",
    "print(f\"the sum of redispatching at this step is {obs6.actual_dispatch.sum():.2f} MW\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}