2_Efficient_representations_of_tensors.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Efficient representation of multidimensional arrays.\n",
    "### Last modification (05.06.2018)\n",
    "\n",
    "![storage_complexity](./images/storage_complexity.png)\n",
    "\n",
    "\n",
    "In this tutorial we provide a theoretical backgound on efficient representation of multidimensional arrays and show how these data structures are integrated into [hottbox](https://github.com/hottbox/hottbox) through **TensorCPD**, **TensorTKD** and **TensorTT** classes.\n",
    "\n",
    "More details on **TensorCPD**, **TensorTKD** and **TensorTT** classes can be found on our [documentation page](https://hottbox.github.io/stable/api/hottbox.core.html#module-hottbox.core).\n",
    "\n",
    "**Note:** this tutorial assumes that you are familiar with the basics of tensor algebra and the corresponding conventional notation.  If you are new to this area, the required background is covered in our [introductory notebook](https://github.com/hottbox/hottbox-tutorials/blob/master/1_N-dimensional_arrays_and_Tensor_class.ipynb).\n",
    "\n",
    "**Requirements:** ``hottbox==0.1.3``\n",
    "\n",
    "**Authors:** \n",
    "Ilya Kisil (ilyakisil@gmail.com); \n",
    "Giuseppe G. Calvi (ggc115@ic.ac.uk)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from hottbox.core import Tensor, TensorCPD, TensorTKD, TensorTT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Outer product, rank-1 tensor and definitions of rank of a multi-dimensional array.\n",
    "\n",
    "\n",
    "The central operator in tensor analysis is the outer product (sometimes refered to as the tensor product). \n",
    "Consider tensors $\\mathbf{\\underline{A}} \\in \\mathbb{R}^{I_1 \\times \\cdots \\times I_N}$ and $\\mathbf{\\underline{B}}  \\in \\mathbb{R}^{J_1 \\times \\cdots \\times J_M}$, then  their outer product yeilds a tensor of higher order then both of them:\n",
    "\n",
    "$$\n",
    "\\begin{equation}\n",
    "\\begin{aligned}\n",
    "    \\mathbf{\\underline{A}} \\circ \\mathbf{\\underline{B}} &= \\mathbf{\\underline{C}} \\in \\mathbb{R}^{I_1 \\times \\cdots \\times I_N \\times J_1 \\times \\cdots \\times J_M} \\\\\n",
    "    a_{i_1,\\dots,i_N}b_{j_1,\\dots,j_M} &= c_{i_1,\\dots,i_N,j_1,\\dots,j_M} \n",
    "\\end{aligned}    \n",
    "\\end{equation}\n",
    "$$\n",
    "\n",
    "Most of the time we deal with the outer product of vectors, which significanlty simplifies the general form expressed above and establishes one the of the most fundamenatal definitions. A tensor of order $N$ is said to be of **rank-1** if it can be represented as an outer product of $N$ vectors. The figure below illustrates an example of rank-1 tensor $\\mathbf{\\underline{X}}$ and provides intuition of how operation of outer product is computed:\n",
    "\n",
    "![outerproduct](./images/outerproduct_3.png)\n",
    "\n",
    "There are several forms of the rank of N-dimensional arrays each of which is accosiated with a representation of a tensor in a particular form:\n",
    "\n",
    "1. Kruskal rank $\\rightarrow$ canonical polyadic form.\n",
    "\n",
    "- Multi-linear rank $\\rightarrow$ tucker form.\n",
    "\n",
    "- TT rank $\\rightarrow$ tensor train form.\n",
    "\n",
    "Each of this representations has the correposing class: **``TensorCPD``**, **``TensorTKD``**, **``TensorTT``**. All of them come with almost identical API except for obejct creation and, as a result, the names for some attributes. But before, we can proceed, it is crucial to get acquainted with the following definitions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Canonical Polydiac representation (CP), Kruskal rank and TensorCPD class\n",
    "\n",
    "![cpd_as_rank_one](./images/cpd_as_rank_one.png)\n",
    "\n",
    "## Kryskal rank\n",
    "This figure illustrates a tensor $\\mathbf{\\underline{X}}$ of rank $R$. The **rank** of a tensor $\\mathbf{\\underline{X}}$ is defined as the smallest number of rank-one tensors that produce $\\mathbf{\\underline{X}}$ as their linear combination. This definition of a tensor rank is also known as the **Kruskal rank**.\n",
    "\n",
    "## CP representation\n",
    "For a third order tensor or rank $R$ it can be expressed as follows:\n",
    "\n",
    "$$\\mathbf{\\underline{X}} = \\sum_{r=1}^R \\mathbf{\\underline{X}}_r = \\sum_{r=1}^R \\lambda_{r} \\cdot \\mathbf{a}_r \\circ \\mathbf{b}_r \\circ \\mathbf{c}_r$$\n",
    "\n",
    "The vectors $\\mathbf{a}_r, \\mathbf{b}_r$ and $\\mathbf{c}_r$ are oftentime combined into corresponding **factor matrices**:\n",
    "\n",
    "$$\n",
    "\\mathbf{A} = \\Big[ \\mathbf{a}_1 \\cdots \\mathbf{a}_R \\Big] \\quad\n",
    "\\mathbf{B} = \\Big[ \\mathbf{b}_1 \\cdots \\mathbf{b}_R \\Big] \\quad\n",
    "\\mathbf{C} = \\Big[ \\mathbf{c}_1 \\cdots \\mathbf{c}_R \\Big] \\quad\n",
    "$$\n",
    "\n",
    "Thus, if we employ the mode-$n$ product, the canonical polyadic representation takes form:\n",
    "\n",
    "$$\n",
    "\\mathbf{\\underline{X}} = \\mathbf{\\underline{\\Lambda}} \\times_1 \\mathbf{A} \\times_2 \\mathbf{B} \\times_3 \\mathbf{C} = \\Big[\\mathbf{\\underline{\\Lambda}}; \\mathbf{A}, \\mathbf{B}, \\mathbf{C} \\Big]\n",
    "$$\n",
    "\n",
    "where the elements on the super-diagonal of $\\mathbf{\\underline{\\Lambda}}$ are occupied by the values $\\lambda_r$ and all other equal to zero. This is the **canonical polyadic (CP)** representation of the original tensor\n",
    "and can be visualised as shown on figure below:\n",
    "\n",
    "![tensorcpd](./images/TensorCPD.png)\n",
    "\n",
    "\n",
    "## TensorCPD class in hottbox\n",
    "\n",
    "In **`hottbox`**, this form is available through the **``TensorCPD``** class. In order to create such object, you need to pass a list of factor matrices (2d numpy arrays) and a vector of values (as 1d numpy array) for the main diagonal:\n",
    "\n",
    "```python\n",
    "tensor_cpd = TensorCPD(fmat=[A, B, C], core_values=values)\n",
    "```\n",
    "\n",
    "**Note:** all matrices should have the same number of columns and be equal to the length of ``values``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Kruskal representation of a tensor with rank=(2,).\n",
      "Factor matrices represent properties: ['mode-0', 'mode-1', 'mode-2']\n",
      "With corresponding latent components described by (3, 4, 5) features respectively.\n"
     ]
    }
   ],
   "source": [
    "I, J, K = 3, 4, 5  # define shape of the tensor in full form\n",
    "R = 2              # define Kryskal rank of a tensor in CP form \n",
    "\n",
    "A = np.arange(I * R).reshape(I, R)\n",
    "B = np.arange(J * R).reshape(J, R)\n",
    "C = np.arange(K * R).reshape(K, R)\n",
    "values = np.arange(R)\n",
    "\n",
    "tensor_cpd = TensorCPD(fmat=[A, B, C], core_values=values)\n",
    "print(tensor_cpd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The list of factor matrices **[A, B, C]** is stored in **`_fmat`** placeholder which can (should) be accessed through the correspodning property **`fmat`**. The values for the super-diagonal are stored in **`_core_values`** placeholder. But there is no direct access to them, because they are used fore creation of the core tensor:\n",
    "\n",
    "```python\n",
    "tensor_cpd.core\n",
    "```\n",
    "\n",
    "This returns an object of the **``Tensor``** class with the **``_core_values``** placed on its super-diagonal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\tFactor matrices\n",
      "Mode-0 factor matrix is of shape (3, 2)\n",
      "Mode-1 factor matrix is of shape (4, 2)\n",
      "Mode-2 factor matrix is of shape (5, 2)\n",
      "\n",
      "\tCore tensor\n",
      "This tensor is of order 3 and consists of 8 elements.\n",
      "Sizes and names of its modes are (2, 2, 2) and ['mode-0', 'mode-1', 'mode-2'] respectively.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[[0., 0.],\n",
       "        [0., 0.]],\n",
       "\n",
       "       [[0., 0.],\n",
       "        [0., 1.]]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('\\tFactor matrices')\n",
    "for mode, fmat in enumerate(tensor_cpd.fmat):\n",
    "    print('Mode-{} factor matrix is of shape {}'.format(mode, fmat.shape))\n",
    "    \n",
    "print('\\n\\tCore tensor')\n",
    "print(tensor_cpd.core)\n",
    "tensor_cpd.core.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to convert **``TensorCPD``** into the full representation, simply call: \n",
    "\n",
    "```python\n",
    "tensor_cpd.reconstruct()\n",
    "```\n",
    "\n",
    "This returns an object of the **``Tensor``** class with N-dimensional array calculated as described above and being assinged to the **``_data``** attibute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This tensor is of order 3 and consists of 60 elements.\n",
      "Sizes and names of its modes are (3, 4, 5) and ['mode-0', 'mode-1', 'mode-2'] respectively.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[[  1.,   3.,   5.,   7.,   9.],\n",
       "        [  3.,   9.,  15.,  21.,  27.],\n",
       "        [  5.,  15.,  25.,  35.,  45.],\n",
       "        [  7.,  21.,  35.,  49.,  63.]],\n",
       "\n",
       "       [[  3.,   9.,  15.,  21.,  27.],\n",
       "        [  9.,  27.,  45.,  63.,  81.],\n",
       "        [ 15.,  45.,  75., 105., 135.],\n",
       "        [ 21.,  63., 105., 147., 189.]],\n",
       "\n",
       "       [[  5.,  15.,  25.,  35.,  45.],\n",
       "        [ 15.,  45.,  75., 105., 135.],\n",
       "        [ 25.,  75., 125., 175., 225.],\n",
       "        [ 35., 105., 175., 245., 315.]]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor_full = tensor_cpd.reconstruct()\n",
    "print(tensor_full)\n",
    "tensor_full.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tucker representation, Multi-linear rank and TensorTKD class\n",
    "\n",
    "## Multi-linear rank\n",
    "\n",
    "The **multi-linear rank** of a tensor $\\mathbf{\\underline{X}} \\in \\mathbb{R}^{I_1 \\times \\cdots \\times I_N}$ is the $N$-tuple $(R_1, \\dots, R_N)$ where each $R_n$ is the rank of the subspace spanned by mode-$n$ fibers, i.e. $R_n = \\text{rank} \\big( \\mathbf{X}_{(n)} \\big)$. For a tensor of order $N$ the values $R_1, R_2, \\dots , R_N$ are not necessarily the same, whereas, for matrices (tensors of order 2) the equality $R_1 = R_2$ always holds, where $R_1$ and $R_2$ are the matrix column rank and row rank respectively.\n",
    "\n",
    "\n",
    "## Tucker representation\n",
    "![tensortkd](./images/TensorTKD.png)\n",
    "\n",
    "For a tensor $\\mathbf{\\underline{X}} \\in \\mathbb{R}^{I \\times J \\times K}$ illustrated above, the **tucker form** represents it as a dense core tensor $\\mathbf{\\underline{G}}$ with multi-linear rank ($Q, R, P$) and a set of factor matrices $\\mathbf{A} \\in \\mathbb{R}^{I \\times Q}, \\mathbf{B} \\in \\mathbb{R}^{J \\times R}$ and $\\mathbf{C} \\in \\mathbb{R}^{K \\times P}$.\n",
    "\n",
    "The tucker form of a tensor is closely related to the CP form and can be expressed through a \n",
    "sequence of mode-$n$ products in a similar way.\n",
    "\n",
    "$$\n",
    "\\mathbf{\\underline{X}} = \\mathbf{\\underline{G}} \\times_1 \\mathbf{A} \\times_2 \\mathbf{B} \\times_3 \\mathbf{C} = \\Big[\\mathbf{\\underline{G}}; \\mathbf{A}, \\mathbf{B}, \\mathbf{C} \\Big]\n",
    "$$\n",
    "\n",
    "## TensorTKD class in hottbox\n",
    "\n",
    "In **`hottbox`**, this form is available through the **``TensorTKD``** class. In order to create such object, you need to pass a list of $N$ factor matrices (2d numpy arrays) and values for the core tensor (as n-dimensional numpy array):\n",
    "\n",
    "```python\n",
    "tensor_tkd = TensorTKD(fmat=[A, B, C], core_values=values)\n",
    "```\n",
    "\n",
    "**Note:** the number of columns in each of the factor matrices should be the same as the corresponding size of the numpy array with the values for the core tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tucker representation of a tensor with multi-linear rank=(2, 3, 4).\n",
      "Factor matrices represent properties: ['mode-0', 'mode-1', 'mode-2']\n",
      "With corresponding latent components described by (5, 6, 7) features respectively.\n"
     ]
    }
   ],
   "source": [
    "I, J, K = 5, 6, 7  # define shape of the tensor in full form\n",
    "Q, R, P = 2, 3, 4  # define multi-linear rank of the tensor in Tucker form\n",
    "\n",
    "A = np.arange(I * Q).reshape(I, Q)\n",
    "B = np.arange(J * R).reshape(J, R)\n",
    "C = np.arange(K * P).reshape(K, P)\n",
    "values = np.arange(Q * R * P).reshape(Q, R, P)\n",
    "\n",
    "tensor_tkd = TensorTKD(fmat=[A, B, C], core_values=values)\n",
    "print(tensor_tkd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By analogy with the **`TensorCPD`**, the list of factor matrices **[A, B, C]** is stored in **`_fmat`** placeholder which can (should) be accessed through the correspodning property **`fmat`**. Similarly, the values of the core tensor are stored in **`_core_values`** placeholder and they cannot (should not) be accessed directly, because they are used to create a core tensors as an object of **`Tensor`** class, when the corresponding property is called:\n",
    "\n",
    "```python\n",
    "tensor_tkd.core\n",
    "```\n",
    "\n",
    "**Note:** the core values occupy all data values of a core tensor, as opposed to **`TensorCPD`** class where they are placed on the main diagonal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\tFactor matrices\n",
      "Mode-0 factor matrix is of shape (5, 2)\n",
      "Mode-1 factor matrix is of shape (6, 3)\n",
      "Mode-2 factor matrix is of shape (7, 4)\n",
      "\n",
      "\tCore tensor\n",
      "This tensor is of order 3 and consists of 24 elements.\n",
      "Sizes and names of its modes are (2, 3, 4) and ['mode-0', 'mode-1', 'mode-2'] respectively.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[[ 0,  1,  2,  3],\n",
       "        [ 4,  5,  6,  7],\n",
       "        [ 8,  9, 10, 11]],\n",
       "\n",
       "       [[12, 13, 14, 15],\n",
       "        [16, 17, 18, 19],\n",
       "        [20, 21, 22, 23]]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('\\tFactor matrices')\n",
    "for mode, fmat in enumerate(tensor_tkd.fmat):\n",
    "    print('Mode-{} factor matrix is of shape {}'.format(mode, fmat.shape))\n",
    "    \n",
    "print('\\n\\tCore tensor')\n",
    "print(tensor_tkd.core)\n",
    "tensor_tkd.core.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to convert **``TensorTKD``** into the full representation, simply call: \n",
    "\n",
    "```python\n",
    "tensor_tkd.reconstruct()\n",
    "```\n",
    "\n",
    "This return an object of the **``Tensor``** class with N-dimensional array calculated as \n",
    "described above and being assinged to the **``_data``** attibute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This tensor is of order 3 and consists of 210 elements.\n",
      "Sizes and names of its modes are (5, 6, 7) and ['mode-0', 'mode-1', 'mode-2'] respectively.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[[    378,    1346,    2314,    3282,    4250,    5218,    6186],\n",
       "        [   1368,    4856,    8344,   11832,   15320,   18808,   22296],\n",
       "        [   2358,    8366,   14374,   20382,   26390,   32398,   38406],\n",
       "        [   3348,   11876,   20404,   28932,   37460,   45988,   54516],\n",
       "        [   4338,   15386,   26434,   37482,   48530,   59578,   70626],\n",
       "        [   5328,   18896,   32464,   46032,   59600,   73168,   86736]],\n",
       "\n",
       "       [[   1458,    5146,    8834,   12522,   16210,   19898,   23586],\n",
       "        [   5112,   17944,   30776,   43608,   56440,   69272,   82104],\n",
       "        [   8766,   30742,   52718,   74694,   96670,  118646,  140622],\n",
       "        [  12420,   43540,   74660,  105780,  136900,  168020,  199140],\n",
       "        [  16074,   56338,   96602,  136866,  177130,  217394,  257658],\n",
       "        [  19728,   69136,  118544,  167952,  217360,  266768,  316176]],\n",
       "\n",
       "       [[   2538,    8946,   15354,   21762,   28170,   34578,   40986],\n",
       "        [   8856,   31032,   53208,   75384,   97560,  119736,  141912],\n",
       "        [  15174,   53118,   91062,  129006,  166950,  204894,  242838],\n",
       "        [  21492,   75204,  128916,  182628,  236340,  290052,  343764],\n",
       "        [  27810,   97290,  166770,  236250,  305730,  375210,  444690],\n",
       "        [  34128,  119376,  204624,  289872,  375120,  460368,  545616]],\n",
       "\n",
       "       [[   3618,   12746,   21874,   31002,   40130,   49258,   58386],\n",
       "        [  12600,   44120,   75640,  107160,  138680,  170200,  201720],\n",
       "        [  21582,   75494,  129406,  183318,  237230,  291142,  345054],\n",
       "        [  30564,  106868,  183172,  259476,  335780,  412084,  488388],\n",
       "        [  39546,  138242,  236938,  335634,  434330,  533026,  631722],\n",
       "        [  48528,  169616,  290704,  411792,  532880,  653968,  775056]],\n",
       "\n",
       "       [[   4698,   16546,   28394,   40242,   52090,   63938,   75786],\n",
       "        [  16344,   57208,   98072,  138936,  179800,  220664,  261528],\n",
       "        [  27990,   97870,  167750,  237630,  307510,  377390,  447270],\n",
       "        [  39636,  138532,  237428,  336324,  435220,  534116,  633012],\n",
       "        [  51282,  179194,  307106,  435018,  562930,  690842,  818754],\n",
       "        [  62928,  219856,  376784,  533712,  690640,  847568, 1004496]]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor_full = tensor_tkd.reconstruct()\n",
    "print(tensor_full)\n",
    "tensor_full.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensor Train representation, TT-rank and TensorTT class\n",
    "\n",
    "## Tensor Train representation\n",
    "\n",
    "![tensortt](./images/TensorTT.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tensor trains (TTs)** are the simplest kinds of tensor networks, i.e. a decomposition of a high-order tensor in a set of sparsely interconnected lower-order tensors and factor matrices. Mathematically, an $N$-th order tensor $\\mathbf{\\underline{X}} \\in \\mathbb{R}^{I_1 \\times I_2 \\times \\cdots \\times I_N}$ can be expressed as a TT as\n",
    "\n",
    "$$\n",
    "\\mathbf{\\underline{X}} = \\mathbf{A} \\times^1_2 \\mathbf{\\underline{G}}^{(1)}  \\times^1_3 \\mathbf{\\underline{G}}^{(2)}   \\times^1_3 \\cdots \\times^1_3 \\mathbf{\\underline{G}}^{(N-1)} \\times^1_3 \\mathbf{B} = \\Big[  \\mathbf{A}, \\mathbf{\\underline{G}}^{(1)}, \\mathbf{\\underline{G}}^{(2)}, \\cdots, \\mathbf{\\underline{G}}^{(N-1)}, \\mathbf{B}  \\Big]\n",
    "$$\n",
    "\n",
    "Each element of a TT is generally referred to as **TT-core**, and $\\mathbf{A} \\in \\mathbb{R}^{I_1 \\times R_1}$, $\\mathbf{B} \\in \\mathbb{R}^{R_{N-1}\\times I_N}$, $\\mathbf{\\underline{G}}^{(n)} \\in \\mathbb{R}^{R_n \\times I_{n+1} \\times R_{n+1}}$ and the tuple $(R_1, R_2, \\dots, R_{N-1})$ is called the **TT-rank**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TensorTT class in hottbox\n",
    "\n",
    "In **`hottbox`**, this form is available through the **``TensorTT``** class. In order to create such object, you need to pass a list of values (as numpy arrays) for \n",
    "cores:\n",
    "\n",
    "```python\n",
    "tensor_tt = TensorTT(core_values=values)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor train representation of a tensor with tt-rank=(2, 3).\n",
      "Shape of this representation in the full format is (4, 5, 6).\n",
      "Physical modes of its cores represent properties: ['mode-0', 'mode-1', 'mode-2']\n"
     ]
    }
   ],
   "source": [
    "I, J, K = 4, 5, 6  # define shape of the tensor in full form\n",
    "R1, R2 = 2, 3      # define tt rank of the tensor in Tensor train form\n",
    "\n",
    "values_1 = np.arange(I * R1).reshape(I, R1)\n",
    "values_2 = np.arange(R1 * J * R2).reshape(R1, J, R2)\n",
    "values_3 = np.arange(R2 * K).reshape(R2, K)\n",
    "\n",
    "tensor_tt = TensorTT(core_values=[values_1, values_2, values_3])\n",
    "print(tensor_tt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The list of values for these core tensors is stored in **`_core_values`** placeholder. They should not be accessed directly, because they are used\n",
    "for creation of **`Tensor`** class objects each of which represent a particular tt-core. The list of all cores can be accessed as \n",
    "\n",
    "```python\n",
    "tensor_tt.cores\n",
    "```\n",
    "\n",
    "**Note:** All components of the Tensor Train representation are conventionally considered to be a core therefore, even matrices are objects of **`Tensor`** class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\tCore tensor #0 of TT representation\n",
      "This tensor is of order 2 and consists of 8 elements.\n",
      "Sizes and names of its modes are (4, 2) and ['mode-0', 'mode-1'] respectively.\n",
      "[[0 1]\n",
      " [2 3]\n",
      " [4 5]\n",
      " [6 7]]\n",
      "\n",
      "\tCore tensor #1 of TT representation\n",
      "This tensor is of order 3 and consists of 30 elements.\n",
      "Sizes and names of its modes are (2, 5, 3) and ['mode-0', 'mode-1', 'mode-2'] respectively.\n",
      "[[[ 0  1  2]\n",
      "  [ 3  4  5]\n",
      "  [ 6  7  8]\n",
      "  [ 9 10 11]\n",
      "  [12 13 14]]\n",
      "\n",
      " [[15 16 17]\n",
      "  [18 19 20]\n",
      "  [21 22 23]\n",
      "  [24 25 26]\n",
      "  [27 28 29]]]\n",
      "\n",
      "\tCore tensor #2 of TT representation\n",
      "This tensor is of order 2 and consists of 18 elements.\n",
      "Sizes and names of its modes are (3, 6) and ['mode-0', 'mode-1'] respectively.\n",
      "[[ 0  1  2  3  4  5]\n",
      " [ 6  7  8  9 10 11]\n",
      " [12 13 14 15 16 17]]\n"
     ]
    }
   ],
   "source": [
    "for i, tt_core in enumerate(tensor_tt.cores):        \n",
    "    print('\\n\\tCore tensor #{} of TT representation'.format(i))    \n",
    "    print(tt_core)    \n",
    "    print(tt_core.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you what to access a specific tt-core of the TT representation, then it is more efficient to use a corresponding method which takes a positional number of desired core as input parameters\n",
    "\n",
    "```python\n",
    "tensor_tt.core(i=0)\n",
    "```\n",
    "\n",
    "**Note:** this parameter should not exceed the order of TT representation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\tCore tensor #0 of TT representation\n",
      "This tensor is of order 2 and consists of 8 elements.\n",
      "Sizes and names of its modes are (4, 2) and ['mode-0', 'mode-1'] respectively.\n",
      "[[0 1]\n",
      " [2 3]\n",
      " [4 5]\n",
      " [6 7]]\n",
      "\n",
      "\tCore tensor #1 of TT representation\n",
      "This tensor is of order 3 and consists of 30 elements.\n",
      "Sizes and names of its modes are (2, 5, 3) and ['mode-0', 'mode-1', 'mode-2'] respectively.\n",
      "[[[ 0  1  2]\n",
      "  [ 3  4  5]\n",
      "  [ 6  7  8]\n",
      "  [ 9 10 11]\n",
      "  [12 13 14]]\n",
      "\n",
      " [[15 16 17]\n",
      "  [18 19 20]\n",
      "  [21 22 23]\n",
      "  [24 25 26]\n",
      "  [27 28 29]]]\n",
      "\n",
      "\tCore tensor #2 of TT representation\n",
      "This tensor is of order 2 and consists of 18 elements.\n",
      "Sizes and names of its modes are (3, 6) and ['mode-0', 'mode-1'] respectively.\n",
      "[[ 0  1  2  3  4  5]\n",
      " [ 6  7  8  9 10 11]\n",
      " [12 13 14 15 16 17]]\n"
     ]
    }
   ],
   "source": [
    "for i in range(tensor_tt.order):\n",
    "    tt_core = tensor_tt.core(i)\n",
    "    print('\\n\\tCore tensor #{} of TT representation'.format(i))    \n",
    "    print(tt_core)    \n",
    "    print(tt_core.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to convert **``TensorTT``** into the full representation, simply call: \n",
    "\n",
    "```python\n",
    "tensor_tt.reconstruct()\n",
    "```\n",
    "\n",
    "This return an object of the **``Tensor``** class with N-dimensional array calculated as described above and being assinged to the **``_data``** attibute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This tensor is of order 3 and consists of 120 elements.\n",
      "Sizes and names of its modes are (4, 5, 6) and ['mode-0', 'mode-1', 'mode-2'] respectively.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[[ 300,  348,  396,  444,  492,  540],\n",
       "        [ 354,  411,  468,  525,  582,  639],\n",
       "        [ 408,  474,  540,  606,  672,  738],\n",
       "        [ 462,  537,  612,  687,  762,  837],\n",
       "        [ 516,  600,  684,  768,  852,  936]],\n",
       "\n",
       "       [[ 960, 1110, 1260, 1410, 1560, 1710],\n",
       "        [1230, 1425, 1620, 1815, 2010, 2205],\n",
       "        [1500, 1740, 1980, 2220, 2460, 2700],\n",
       "        [1770, 2055, 2340, 2625, 2910, 3195],\n",
       "        [2040, 2370, 2700, 3030, 3360, 3690]],\n",
       "\n",
       "       [[1620, 1872, 2124, 2376, 2628, 2880],\n",
       "        [2106, 2439, 2772, 3105, 3438, 3771],\n",
       "        [2592, 3006, 3420, 3834, 4248, 4662],\n",
       "        [3078, 3573, 4068, 4563, 5058, 5553],\n",
       "        [3564, 4140, 4716, 5292, 5868, 6444]],\n",
       "\n",
       "       [[2280, 2634, 2988, 3342, 3696, 4050],\n",
       "        [2982, 3453, 3924, 4395, 4866, 5337],\n",
       "        [3684, 4272, 4860, 5448, 6036, 6624],\n",
       "        [4386, 5091, 5796, 6501, 7206, 7911],\n",
       "        [5088, 5910, 6732, 7554, 8376, 9198]]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor_full = tensor_tt.reconstruct()\n",
    "print(tensor_full)\n",
    "tensor_full.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Further reading list\n",
    "- Tamara G. Kolda and Brett W. Bader, \"Tensor decompositions and applications.\" SIAM REVIEW, 51(3):455–500, 2009.\n",
    "\n",
    "- Ivan V. Oseledets,  \"Tensor-train decomposition.\" SIAM Journal on Scientific Computing 33.5 (2011): 2295-2317."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "hottbox-tutorials",
   "language": "python",
   "name": "hottbox-tutorials"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}