doc/source/tutorial/tutorial_presenting_larray_objects.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Presenting LArray objects (Axis, Groups, LArray, Session)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "nbsphinx": "hidden"
   },
   "outputs": [],
   "source": [
    "# run this cell to avoid annoying warnings\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\", message=r'.*numpy.dtype size changed*')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the LArray library:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from larray import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the version of LArray:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from larray import __version__\n",
    "__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Axis\n",
    "\n",
    "An ``Axis`` represents a dimension of an LArray object.\n",
    "It consists of a name and a list of labels. \n",
    "\n",
    "They are several ways to create an axis:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a wildcard axis\n",
    "age = Axis(3, 'age')\n",
    "# labels given as a list\n",
    "time = Axis([2007, 2008, 2009], 'time')\n",
    "# create an axis using one string\n",
    "sex = Axis('sex=M,F')\n",
    "# labels generated using a special syntax\n",
    "other = Axis('other=A01..C03')\n",
    "\n",
    "age, sex, time, other"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See the [Axis](../api.rst#axis) section of the API Reference to explore all methods of Axis objects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Groups\n",
    "\n",
    "A ``Group`` represents a selection of labels from an Axis. It can optionally have a name (using operator ``>>``). \n",
    "Groups can be used when selecting a subset of an array and in aggregations. \n",
    "\n",
    "Group objects are created as follow:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define an Axis object 'age'\n",
    "age = Axis('age=0..100')\n",
    "\n",
    "# create an anonymous Group object 'teens'\n",
    "teens = age[10:20]\n",
    "# create a Group object 'pensioners' with a name \n",
    "pensioners = age[67:] >> 'pensioners'\n",
    "\n",
    "teens"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is possible to set a name or to rename a group after its declaration: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# method 'named' returns a new group with the given name\n",
    "teens = teens.named('teens')\n",
    "\n",
    "# operator >> is just a shortcut for the call of the method named\n",
    "teens = teens >> 'teens'\n",
    "\n",
    "teens"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See the [Group](../api.rst#group) section of the API Reference to explore all methods of Group objects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LArray\n",
    "\n",
    "A ``LArray`` object represents a multidimensional array with labeled axes.\n",
    "\n",
    "### Create an array from scratch\n",
    "\n",
    "To create an array from scratch, you need to provide the data and a list\n",
    "of axes. Optionally, metadata (title, description, creation date, authors, ...) can be associated to the array:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# list of the axes\n",
    "axes = [age, sex, time, other]\n",
    "# data (the shape of data array must match axes lengths)\n",
    "data = np.random.randint(100, size=[len(axis) for axis in axes])\n",
    "# metadata\n",
    "meta = [('title', 'random array')]\n",
    "\n",
    "arr = LArray(data, axes, meta=meta)\n",
    "arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Metadata can be added to an array at any time using: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.meta.description = 'array containing random values between 0 and 100'\n",
    "\n",
    "arr.meta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "\n",
    "**Warning:**  \n",
    "  <ul>\n",
    "    <li>Currently, only the HDF (.h5) file format supports saving and loading array metadata.</li>\n",
    "    <li>Metadata is not kept when actions or methods are applied on an array\n",
    "        except for operations modifying the object in-place, such as `pop[age < 10] = 0`,\n",
    "        and when the method `copy()` is called. Do not add metadata to an array if you know\n",
    "        you will apply actions or methods on it before dumping it.</li>\n",
    "  </ul>\n",
    "\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Array creation functions\n",
    "\n",
    "Arrays can also be generated in an easier way through [creation functions](../api.rst#array-creation-functions):\n",
    "\n",
    "-  `ndtest` : creates a test array with increasing numbers as data\n",
    "-  `empty` : creates an array but leaves its allocated memory\n",
    "   unchanged (i.e., it contains \"garbage\". Be careful !)\n",
    "-  `zeros`: fills an array with 0\n",
    "-  `ones` : fills an array with 1\n",
    "-  `full` : fills an array with a given value\n",
    "-  `sequence` : creates an array from an axis by iteratively applying a function to a given initial value.\n",
    "\n",
    "Except for ndtest, a list of axes must be provided.\n",
    "Axes can be passed in different ways:\n",
    "\n",
    "-  as Axis objects\n",
    "-  as integers defining the lengths of auto-generated wildcard axes\n",
    "-  as a string : 'sex=M,F;time=2007,2008,2009' (name is optional)\n",
    "-  as pairs (name, labels)\n",
    "\n",
    "Optionally, the type of data stored by the array can be specified using argument dtype.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# start defines the starting value of data\n",
    "ndtest(['age=0..2', 'sex=M,F', 'time=2007..2009'], start=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# start defines the starting value of data\n",
    "# label_start defines the starting index of labels\n",
    "ndtest((3, 3), start=-1, label_start=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# empty generates uninitialised array with correct axes\n",
    "# (much faster but use with care!).\n",
    "# This not really random either, it just reuses a portion\n",
    "# of memory that is available, with whatever content is there.\n",
    "# Use it only if performance matters and make sure all data\n",
    "# will be overridden.\n",
    "empty(['age=0..2', 'sex=M,F', 'time=2007..2009'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# example with anonymous axes\n",
    "zeros(['0..2', 'M,F', '2007..2009'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dtype=int forces to store int data instead of default float\n",
    "ones(['age=0..2', 'sex=M,F', 'time=2007..2009'], dtype=int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "full(['age=0..2', 'sex=M,F', 'time=2007..2009'], 1.23)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the above functions exist in *(func)_like* variants which take axes from another array\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ones_like(arr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create an array using the special ``sequence`` function (see link to documention of ``sequence`` in API reference for more examples):\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# With initial=1.0 and inc=0.5, we generate the sequence 1.0, 1.5, 2.0, 2.5, 3.0, ...\n",
    "sequence('sex=M,F', initial=1.0, inc=0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inspecting LArray objects\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a test array\n",
    "arr = ndtest([age, sex, time, other])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get array summary : metadata + dimensions + description of axes + dtype + size in memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get axes\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.axes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get number of dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.ndim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get length of each dimension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get total number of elements of the array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get type of internal data (int, float, ...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get size in memory\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.memory_used"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display the array in the viewer (graphical user interface) in read-only mode.\n",
    "This will open a new window and block execution of the rest of code until the windows is closed! Required PyQt installed.\n",
    "\n",
    "```python\n",
    "view(arr)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or load it in Excel:\n",
    "\n",
    "```python\n",
    "arr.to_excel()\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More on LArray objects\n",
    "\n",
    "To know how to save and load arrays in CSV, Excel or HDF format, please refer to the [Loading and Dumping Arrays](tutorial_IO.ipynb#Loading-and-Dumping-Arrays) section of the tutorial.\n",
    "\n",
    "See the [LArray](../api.rst#larray) section of the API Reference to explore all methods of LArray objects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Session\n",
    "\n",
    "A ``Session`` object is a dictionary-like object used to gather several arrays, axes and groups. \n",
    "A session is particularly adapted to gather all input objects of a model or to gather the output arrays from different scenarios. Like with arrays, it is possible to associate metadata to sessions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating Sessions\n",
    "\n",
    "To create a session, you can first create an empty session and then populate it with arrays, axes and groups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create an empty session\n",
    "s_pop = Session()\n",
    "\n",
    "# add axes to the session\n",
    "gender = Axis(\"gender=Male,Female\")\n",
    "s_pop.gender = gender\n",
    "time = Axis(\"time=2013,2014,2015\")\n",
    "s_pop.time = time\n",
    "\n",
    "# add arrays to the session\n",
    "s_pop.pop = zeros((gender, time))\n",
    "s_pop.births = zeros((gender, time))\n",
    "s_pop.deaths = zeros((gender, time))\n",
    "\n",
    "# add metadata after creation\n",
    "s_pop.meta.title = 'Demographic Model of Belgium'\n",
    "s_pop.meta.description = 'Models the demography of Belgium'\n",
    "\n",
    "# print content of the session\n",
    "print(s_pop.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or you can create and populate a session in one step:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gender = Axis(\"gender=Male,Female\")\n",
    "time = Axis(\"time=2013,2014,2015\")\n",
    "\n",
    "# create and populate a new session in one step\n",
    "# Python <= 3.5\n",
    "s_pop = Session([('gender', gender), ('time', time), ('pop', zeros((gender, time))), \n",
    "                ('births', zeros((gender, time))), ('deaths', zeros((gender, time)))], \n",
    "                meta=[('title', 'Demographic Model of Belgium'),('description', 'Modelize the demography of Belgium')])\n",
    "# Python 3.6+\n",
    "s_pop = Session(gender=gender, time=time, pop=zeros((gender, time)), \n",
    "                births=zeros((gender, time)), deaths=zeros((gender, time)), \n",
    "                meta=Metadata(title='Demographic Model of Belgium', description='Modelize the demography of Belgium'))\n",
    "\n",
    "# print content of the session\n",
    "print(s_pop.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "\n",
    "**Warning:**\n",
    "  <ul>\n",
    "    <li>Contrary to array metadata, saving and loading session metadata is supported for\n",
    "        all current session file formats: Excel, CSV and HDF (.h5).</li>\n",
    "    <li>Metadata is not kept when actions or methods are applied on a session\n",
    "        except for operations modifying a session in-place, such as: `s.arr1 = 0`.\n",
    "        Do not add metadata to a session if you know you will apply actions or methods\n",
    "        on it before dumping it.</li>\n",
    "  </ul>\n",
    "\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More on Session objects\n",
    "\n",
    "To know how to save and load sessions in CSV, Excel or HDF format, please refer to the [Loading and Dumping Sessions](tutorial_IO.ipynb#Loading-and-Dumping-Sessions) section of the tutorial.\n",
    "\n",
    "To see how to work with sessions, please read the [Working With Sessions](tutorial_sessions.ipynb#Working-With-Sessions) section of the tutorial.\n",
    "\n",
    "Finally, see the [Session](../api.rst#session) section of the API Reference to explore all methods of Session objects."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "livereveal": {
   "autolaunch": false,
   "scroll": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}