tutorial/westeros/westeros_baseline_using_xlsx_import_part1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Westeros Tutorial - Introducing the use of Excel files: Building a scenario by importing data from a single \".xlsx\" file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Scope of this tutorial\n",
    "\n",
    "This tutorial is the first of two tutorials, which explains how the Westeros Baseline scenario can be built using data imported from Excel (.xlsx) files.The figure below compares the workflows of the original Westeros baseline tutorial, where data was added using the [add_par()](https://docs.messageix.org/projects/ixmp/en/stable/api.html?highlight=add_set()#ixmp.Scenario.add_par) and [add_set()](https://docs.messageix.org/projects/ixmp/en/stable/api.html?highlight=add_set()#ixmp.Scenario.add_set) functions. The same scenario can also be created by importing data from Excel files, using [ixmp.Scenario.read_excel()](https://docs.messageix.org/projects/ixmp/en/latest/api.html#ixmp.Scenario.read_excel) as shown in the figure below.\n",
    "\n",
    "**Pre-requisites**\n",
    "- You have the *MESSAGEix* framework installed and working\n",
    "- You have run Westeros baseline scenario (``westeros_baseline.ipynb``) and solved it successfully\n",
    "\n",
    "In this tutorial, we will generate a single data file, which we require to generate a new scenario, by exporting the scenario data of the `westeros_baseline.ipynb` to an Excel file.  Subsequently, we will import this data into a new scenario, therefore creating an exact replica of the original baseline scenario.\n",
    "\n",
    "<img src='_static/westeros_baseline_xlsx_workflow_part1.jpg'>\n",
    "\n",
    "In the second tutorial, we will then import data from multiple Excel files. This will provide a more detailed look at the underlying structure of the Excel files.\n",
    "\n",
    "### Online documentation\n",
    "\n",
    "The documentation of the MESSAGEix framework is available at [https://docs.messageix.org](https://docs.messageix.org)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<div class=\"alert alert-block alert-success\">\n",
    "    \n",
    "**Please Note:**\n",
    "    \n",
    "At this point, it should be emphasized that the use of this method to build complex models is not recommended. There are multiple reasons for this.  Most importantly, experience has shown that relying on Excel files as the only source of data often results in issues of data management. While at the beginning of the model building exercise, there are only a few files, their number will increase over time.  This can result in possible confusions as to which dataset was used for which scenario. Further, the `read_excel()` function makes use of many other MESSAGEix functionalities. Hence, the modeler will rely on the use of a single function instead of the underlying functions. This can result in the development of workflows that are more complex than required.\n",
    "\n",
    "<u>An example:</u>\n",
    "\n",
    "Assume that the 'baseline' scenario is built from an Excel file. It is easy to duplicate the excel file, change/add one or two parameters in the new excel file in order to create a new scenario. But for each additional scenario, a new excel file is created, which requires manual intervention.\n",
    "    \n",
    "The method which we advise in such cases would be to `clone()` the 'baseline' scenario. Functions such as `par()` can then be used to retrieve existing parameters from a scenario.  The data can be modified and added back to the scenario using `add_par()`. Setting up clear workflows is key to also ensuring the reproducibility and transparency of your work."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "As in the \"original\" tutorial, we start by importing all the packages we need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import ixmp\n",
    "import message_ix\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Loading Modeling platform\n",
    "mp = ixmp.Platform()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1. Loading the Westeros baseline scenario"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "base = message_ix.Scenario(mp, model='Westeros Electrified', scenario='baseline')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2. Export the data to a file called `westeros_baseline_data.xlsx`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_file = 'westeros_baseline_data.xlsx'\n",
    "base.to_excel(data_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3. Create a new scenario"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Creating a new, empty scenario\n",
    "scenario = message_ix.Scenario(mp, model='Westeros Electrified', \n",
    "                               scenario='baseline_xlsx', version='new')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Step 4: Importing data from Excel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Instead of using the `message_ix.Scenario.add_par()` for adding data to a MESSAGEix parameter, we import data from an Excel file. The arguments `add_units` has been set to `True`, so that any units which have not yet been specified in the modeling platform will be defined automatically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scenario.read_excel(data_file, add_units=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Time to Solve the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "scenario.set_as_default()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "scenario.solve()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check the objective value\n",
    "Finally, we can check the objective value of the original baseline scenario with that of the newly created version."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "name": "solve-objective-value"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Objective value of the original 'baseline' scenario.\n",
    "base.var('OBJ')['lvl']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Objective value of scenario built using the Excel file.\n",
    "scenario.var('OBJ')['lvl']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mp.close_db()"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}