Chapter 2 - Basic Matrix Algebra.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "import scipy as sp\n",
    "import sympy as sy\n",
    "sy.init_printing() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.set_printoptions(precision=3)\n",
    "np.set_printoptions(suppress=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.interactiveshell import InteractiveShell\n",
    "InteractiveShell.ast_node_interactivity = \"all\" # display multiple results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "def round_expr(expr, num_digits):\n",
    "    return expr.xreplace({n : round(n, num_digits) for n in expr.atoms(sy.Number)})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Matrix Operations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Matrix operations are straightforward, the addition properties are as following:\n",
    "1. $\\pmb{A}+\\pmb B=\\pmb B+\\pmb A$\n",
    "2. $(\\pmb{A}+\\pmb{B})+\\pmb C=\\pmb{A}+(\\pmb{B}+\\pmb{C})$\n",
    "3. $c(\\pmb{A}+\\pmb{B})=c\\pmb{A}+c\\pmb{B}$\n",
    "4. $(c+d)\\pmb{A}=c\\pmb{A}+c\\pmb{D}$\n",
    "5. $c(d\\pmb{A})=(cd)\\pmb{A}$\n",
    "6. $\\pmb{A}+\\pmb{0}=\\pmb{A}$, where $\\pmb{0}$ is the zero matrix\n",
    "7. For any $\\pmb{A}$, there exists an $-\\pmb A$, such that $\\pmb A+(-\\pmb A)=\\pmb0$.\n",
    "\n",
    "They are as obvious as it shows, so no proofs are provided here.And the matrix multiplication properties are:\n",
    "1. $\\pmb A(\\pmb{BC})=(\\pmb{AB})\\pmb C$\n",
    "2. $c(\\pmb{AB})=(c\\pmb{A})\\pmb{B}=\\pmb{A}(c\\pmb{B})$\n",
    "3. $\\pmb{A}(\\pmb{B}+\\pmb C)=\\pmb{AB}+\\pmb{AC}$\n",
    "4. $(\\pmb{B}+\\pmb{C})\\pmb{A}=\\pmb{BA}+\\pmb{CA}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we need to differentiate two kinds of multiplication, <font face=\"gotham\" color=\"red\">Hadamard multiplication</font> (element-wise multiplication) and <font face=\"gotham\" color=\"red\">matrix multiplication</font>: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.array([[1, 2], [3, 4]])\n",
    "B = np.array([[5, 6], [7, 8]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 5, 12],\n",
       "       [21, 32]])"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A*B # this is Hadamard elementwise product"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[19, 22],\n",
       "       [43, 50]])"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A@B # this is matrix product"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The matrix multipliation rule is"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "19"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "43"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "22"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "50"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(A[0,:]*B[:,0]) # (1, 1)\n",
    "np.sum(A[1,:]*B[:,0]) # (2, 1)\n",
    "np.sum(A[0,:]*B[:,1]) # (1, 2)\n",
    "np.sum(A[1,:]*B[:,1]) # (2, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\"> SymPy Demonstration: Addition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's define all the letters as symbols in case we might use them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z = sy.symbols('a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z', real = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHYAAAAzCAYAAABCMvzOAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAG90lEQVR4Ae2bj3HVOBDG32NSQI7rIHTAHR2EDoASoANurgKG6wCughA6SDrgoAPogEAHue+nWEaWZXn9/O85o53Rs7ReSZ93Je3a0tvvdrtTpRbd3t7+bDEL4+g0sN/vk/Z7IKSflX5E6e3RPUEB1KWBy8h22PISw0JPNUP3QXp1xy6/x64B2axhO+F1tvOGPXb8Bd9ADZz0yWsNP5PMX5Xcn7reUNZI+VLxjuJiwVn5I5Yu9xx6hkdrgLdgnQLXVzVyrofcxUl8jHoZ8lXG/94qJeuEskvlhWUQTskTV7xbCl/Yz1CsYV1LXu2/VLpS2uUMy+g+jRsUzwVbMX+tsvAMwil5BuazNfAOxToUo9p3hu3zsecS/FEtYcrWdK3cabWk1MwVM2acwowsxDOsQWasY8D1GZaH/6ZR0/VOm3yHGgPowLpDcD7teaYDIZirDcFqbjQW3IvBUvxKxjOPYI166pypDvVrqmYwgRYD4bvSRyWWvH90XZxSOMXDv35T+lQBIoC6EkawrkYprB6M7hHXoE/od6UL4U0Gr5JlKX6OYKePVeVUQPVYdfBRr8P7KrslRlcM7uopjxNHtuWnvcxcV/XZwgmOCg/GdZgCXo17Lkxd7QpDCyuyIoJC7FPHA8q/U2IgtmxT1ekPnlKVq44aEaV4KIyA6mVYR2VAfA15S+Xpl/7D/lR+ptQaaBWvMVDDenPn1X8LK31W/Ldh/+Kh506suucMe6KMmTTNMdS1Ooq/TLFUYNwPUWPM4sWXuAxO/OsX4cdVOJIsuCGWuMWpC2vFZ8a+CUEJ+29huSvfFzzV9dQRI4HpHxsV9gslDB4rDGDunQqhJagHJx8m/otwwIO8z70rLfDbg7Wl0yGQTDNWAFjCHoVGFQ+jQXyJYtTHzhxgDARzUIb8GMrhFA4CJnwZq05ILtAQYzGcdJ7DqttdOqWqiXpnrACgjCdSjP+s6BvG2ADwFI94FOaMzUMo+YHg5Se9DsCJgR2pDgOSAchbQb3a3N2d77cPa4AF39si1cfFZSk7Y9UAxuCrzrXy4Uh/KN5jAXCvMbqHAWvDqcyyTdnPAgbGbL7WilN4wFDjVJ5n+yBs73VdhAZgBdMfMSjVJ565iPlxOfseq0Z4LWDGpoggxHUsOUb+v0rMWveepSszABDw8L9u9io/OQ3ECSZmwirvr1asKEmyYIX8Oyz599Jl5+qiOkyq51nD0kqhbWnAG/bBtmAXtFYNFMNaNbUxuWLYjRnMCrcY1qqpjckVw27MYFa4xbBWTW1Mrhh2Ywazwi2GtWpqY3LFsBszmBVuMaxVUxuTy24C8Cz6RHWmi9/ZYe+SHR3TgXHV5Tsz35Bpg4/tqb1c3RpPY3CO793WwliMqs9GjP9O/Cb3zRhEfBBPHv4WH4OMPjCuNjjnUZ/bEaDkeZ1D+VPhPLR/S72xGFWfDQF2o5gsHI9J6lN8dzRG16xhaah1EE0884FxybJ32DpnZFGGVUbtj8Zp7etQubEYVd8ZU1f0yamUll3AJnKG7fOxNDL2wPgS53inwIlS5qSDMWoJpi5bo2x/kviHnV+Sk5j7DMtG+dgD44DyG+5JEBMwp8A5AYxsE2MwDp4cJzkoGhX+PFAs5jbfdb+xeR4EB4wmNod5GGQbJ+1UnpQOwMno/1vJb2Cz6U5AmJ0FY0APxUhf0ifLKkZlctyojMu5UVumILQzeFIDrSBHDWMoFvPG2VaV6Rw/UB+8Vh5fMKt/TWGEJ8rh5Jk52uOer5JtnN/19+a8dmGM+5RcS9+xjC9Ltj948sLhVRVRSnwQmxmAUVMHxj+H9ZfKd+AkykdJMc7X4iXfDObEm8IY9ycZj7keiLFMWObZlK6yS7EEGqSlgPconHe8FBCKY9xjPzDuD+TxT0GM6Q+J85eJueMAdfeLMrr8JXSXS7q9WCgumw0rIIwElq7YqLA5wonBax8leQzNaGM5Xox6cOIuPgrnKn8S80rowejF/PWJMvWRWc/su5oMKyD35cA4+hispD4lDrmf06UGXAobM7YRpFr663vdITKj4ftyYDylOKenSuEWnR0sM0CXYR98xv0UMiz57IwVEJZSQuz7cmCcWKDhSvSMuAz4pNnIqssQQFUHfIP9f/ZcsRq+VwfGUZqeyQdN/h2WuGF2n2vVZWRYXCDf6ht/MA9l4rz6IRYqB8ZjxRxTWUYiin8ow3Z9KGrB9Ybt9bGtmoUxqwZkGP7AhvuD8K8XLjfwpxh2oMIWEMfX856Nb8VNHPRntmzwtMBDlC7aGnCGFZureQmOmymGjTWyclkzdJK/dJaleGVDztW9n7Fnfk33HWnk/PT5cj1eDcR2E1Lnm/17LB8iQuLPtY0X+fBmyR+PBmRYvsWfR4iu/weyh58je1O3zgAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}2 a & 2 b & 2 c\\\\2 d & 2 e & 2 f\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡2⋅a  2⋅b  2⋅c⎤\n",
       "⎢             ⎥\n",
       "⎣2⋅d  2⋅e  2⋅f⎦"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFUAAAAzCAYAAADmWEQdAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAC6klEQVR4Ae2bgVXcMAyGk74OwKMbXDegdAKuG8AKMAMj3AhlBboBbMDrbUA3aGGD49dd9F7OSLYTn3inPvk9k0SWo/iL7LMd0Xddd4L8Lm02m9d3whDsEej7XmT3CVq/kV+SvNqrHRcagfuEG3G8J6iUfsAz+1G+2Ynjb44AeO1xg+6WG0PN1Y2yiQQ+1+hj7ODh4C/0vyKv8Jb+1NRt1fFq+xkNXwJSJ2WU0Zh7yWU4p8GZ6ixYZnWEDVe28bzXyA/Z7g8vIaUTQPuF4zbhnGYFdP1zJ7H569l2FipwXSGvBWxPkC3RcHFKIejPEbm1XYK6BI1/AhHyVkpUbpXc2lahVnrhqQVR77ZVqIDFwNgrx/zYe626v2vbOahjiNr5F63gA+RHazsHlb1R4sOeRPNWi+Tatgp1mDoRMKmLs8xkAeDdtgp1cL9HHBfD+fjAnkrlVsmt7RJU2oU5F6h9g2w98ihBpVnk2nZpmUrl6TKVtrjOAFVc2h5KDhuubON5t8vUmg0V8spbzB15GPiO6wuAW+NonVza7kGFvOEGkCzHR2v4R3F/OB556lVpTD2Kh/X2EAHV4I0F1IBqQMDgluGpAdWAgMEtw1MDqgEBg1uGpwZUAwIGt6xZ+3dYfkUwxcQgktIulauAhgPukE1uN5w+gim0nj9sjMwOIin9ULkNaNCAVcqb2l2C6jagoRKeptbUbhUqugB/3NMMk5y/VeV0Jpd5t61CHQF7FajwJ+Qa8EL1oohflkvbOajFlkPhaAMaah6+QSfb7hxU9kbJNntSBFMIdFSomO9x15O6OMsimGIK1EHXbUCD0NYpoqZ2q546PIHrgIYpFBPd5naXlqmuAhoOuEyd3G68mAimSLwzvZwdyBHBFCnKhuthzyCCKRoYqlVLP1RqxSjQCQRUnc3skoA6G51eMaDqbGaX8DeqRbrdNlqmzr75/14xZYb2bpfvPKVaJADuADX+5z+Bkl4C6gNktKE9To9vhz1xlc3CFtQAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}0 & 0 & 0\\\\0 & 0 & 0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡0  0  0⎤\n",
       "⎢       ⎥\n",
       "⎣0  0  0⎦"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = sy.Matrix([[a, b, c], [d, e, f]])\n",
    "A + A\n",
    "A - A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAL8AAAAzCAYAAADGmNmlAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAKVElEQVR4Ae2di3HUOhSGEyYFcKGD0AGPCoAOgFsB0EEYKshAB0AFEDpIOuDRAXRwIR3k/p8iebRa27JsWfZupBnFtiydx6+joyN57RweHBzcVt5KV1dXl1uFtaAisIMIHB4ettr4LenyQ/lvkN/toI5V5IpAFwJnuhHa+BnGT3oqT3/o5dfXxfVvRWD3EZBdb9i3NDL27Yx/9zWsGlQEEhGoxp8AGLGj8rnyX+VfCU2zV12TLNmVm0hQ2Jwof4iRqcYfQ8i7zyYAU6iKfitfeLeKn65JluLKxxkS1kQ3bBYxfo3KJ8qv4jqstsZ9SXY+RbqMGEyWZYoeOdvmwkSO4Z7ym5hsixi/hGLrqXX7KSbw0vfpICvDVM8/GYOMsiwNq+M/GRNHaMhxKeMfItta65iwh7BjBQKuSZYVwJEmwtGQ6vIwx6rHNEKH/6f8VfmZDOC9joskyfRMjOl8t/BEpieS6ePMAuH5f4r/ieVzT8dz8YV/6YQsFxaLOzrHc4LJa8nDuqRYkgw8G8I2SHeVP0uGn+aqwB/xR3dCafrjl3gPsk2MB6M5aMvcU+YBwbG7r3Pi3Svl264s5ah2GO5JShu/rtoC9A9XpnMGJzJihK165CgXfQBGbx4MGt29sgafIbzUbioGviwNb9Hlgc7ZEBly1BEvsMeGcIYGe52z05LcF9BQHmUXavcO/krGXp0sbUfVYZCc94Y9djQB5hsR8T0J579Vdqlj0WS9HF73sWNsZcMYJi1CHb2eI8CSHjvd3VFldFzJ5GR5afX3eYNFqQTmXyWDP/O9UNncfdHoJ5tgAH6zBc919G21qReeHIUFwTUeFiC/BOUA7ysb3L6+lFB4ANdJfh2m6APdb3uS/FNAokBX+qQbF57RQYcdD1JUputqo/8SUiBfM+jF2xkaU/1WmgkD+DhZwtACvE+3BJmhwOqG4W3wEz7/9LGbAZM/4un6Hq8e3elBvpjxM4JDQ6OzUTg6siVQm3FjrHhJpupBcZnqmqR2GDn8wwcYdDh78INGvCE27s9DNfseNKWM5DzP9ZX9mxsDjzg6uw43xcKHfgGfqTtRht6AP1v2MaANoVFWuxC9S/hau+J00LrvFjXbkggBIjn0LCiMAqUAhp1LztBCmfCCJeRh8BHv+8nNUiX4G762b9ocEEZFOBri48ub5bzHPrLQH0nkrdoRgpnBEKPRafxew9Cj0dkGXEaaMp1QNEm5xsPbTsALRmeiTEKGvHEG7K4MAjyTDF0OiCnfzAbC5ZXFJhPLTTKevm63baOCeNMnxZLVFef0Gaa6JmTvTZ3Gb5XDyBvjFkHA5dpN/Y98Q+zllOemWXtIjkYmkWUNQArXJdelef9iWD5vNgO+CINB02xGUbpmOmZq0/k68pRz7gGJ3g9Cvazh/QnLZ742UYF0/mrto3VQ+jIc+Rct5+yofBKxEx3N3q2OAP/OljmgVTR/ojPF1/FnRkImQC618/RSvJzu7Cd/AGwdSyeMnIEXJtZQ/wojtq4HLfpCAinX4vFavMADL+v2+CFxSl+l0JpaV/x43kHGVln/RR1Sr/FbBVxM68vXVubfn+0cJUW8ia+lLCO8uZ6NsQhbPFoXa3PyDWlLDhzAVlL57AYfMl2CZyiDu+7Cxd0Pj7fCgkLXeIUkzyAjv698pozXM4kynRCGhLs/1xXW/TcZg3Wrk0W6opj0ev4s6rQQ0Qgd46lZyTeLKDsIiPd5AGcW4C2sVls0EoPV6pNDsNKYLGL8I4HCuxPfv5DhE28zA2D4YwaSmtZ00xHYGeO3Rl4N/aZbbEb9l4r5M6pQSVUExiFQjX8cbrXVHiBQjX8POrGqMA6BavzjcKut9gCBavx70IlVhXEIVOMfh1tttQcIjDJ+nqwq/1Dm402LP1218vAxqebp7x70TVQF9FVexUe0fGEl0wdl85sfZPTvlTi3vH/peKXMrwBa0yjj1547bzPxaz4UK/VT4lYFbCFPfu9Ipsu+Svt2D32V+Z0PP7NexTMQDE+y3FHmR4/8Crh5Kq/zIkmY8Bsn5OD3WJ1P/0cZP0SlpFMqGXTaKgNMliQF3ytv/bQ2C/EZiWTEAe+2BicEWvQrhs8A4KfvS9kH9tBp+LoXfY2ROl3JeBwZ3Rhvy4xBvulpMg5TnFBu8K0s6ORefU02fCvTZFxEB+fcG5KP9vyW+FjlrI71kAEBnBDhz2UGWlNJTHGIU3k37TUIj3VB7rXPo6ZFz4klRhwFwLy0AFGm2lPlRZJkwju40InvrwN8kWR58ytT9wIHP7TjR3ZLGCAerulkyfZM13R8MZnEk34Af2T5o2tetOGLCku9+4AcvfE+96PGL0UghDIPpIx5f1VlLr5sQIdY4cRHigy4kuet8itdR9/emSqjxYPp9Ln4mZhSZTgCBkPRl0nEFwcAbzO96xrDJ/F2GS/ak2fHxOL+Ufz5ahRvcSV9lUNtcicGYtQ2e8MeCy6G3/bRqo3v1+SWvo+e5MLTmNW8rYcR8HPnWZP44lEZ+Aw8fzGFg3AOYVYZAuLGw6mM1/fMueTC8MGj6A6QxQbxokZHpZnToP44igiBgQFk+HI4xKPvrgoQPJLpFB39xE4AO0Zt0yKDqvc1Sd1vvJloOG9XAnTjYSU6++vuvWZU4dN8nfznwkF88XCXymD83cmgIwOz9O4XM1A01KCOS3PgIprIgc1G7TNm/Hwiw63cjcwiDmHnAU1Z1x91Qptxu48LJX+0qoMPBlDqBXYz6KVX0rQ+Iw7Ig4fH0O+pb5KMT21ypkciZsLioURnwgVMBn3ArNP4rZFj6P70jl4MCEDu9HTcL5iMQRbkl9TBc8nlOSEW+y7s4ak7a7Owz+YSw6eLx12Cry8D5028D0bCgpmxNfXG/LbFt6AlIYlRUsQX+WiVk0f8mYEGzUKuzcRjp+GDxUTaqc1DJ/TdEgAPZlfCspLpoZiFtlKSv+OFM3TrL7cb6O5tHDuN344YjNyASSsBCjGuHdClP1qFGH5C0ZKzEGsgw9MJIUyI/1kLlPZ6jYdzstijG6B3g/LZLqU/NkGUsJZo4Df9ElP4KFLhse6v5qNVLbJ2GUBL1elFcghs52HsDAK3x8/ga13bTOfYS4HOdQtwZLiUXGy18iEpvmV02ts6783kxW5e9hvU6AuiE8LB3q3nXuMHUEtog3pHWVinxDVeuGQnY2RJi925QJAcDPyNtKBsyBLdXdkQdqYLYdDsBMZYdIY9sYYT7zOoyElJHo2PVpnYWkdCMLzfYGWTmJWpPAqHMqL1c6Ef6A9bi3j/c3+LpLtFcOn1/EniJlTW6EyODQU0UyuG70BmSuMpa/IgShB11qpjcJhVoDTihH5NbC1dsnn+UrgsYvxpGF/XFiD8Ezi8/LGOxLp8Fjx5EI3hXdu0IoDxM/Ny7H0o2dp6BYU7Y/xgJWNfYmG5gm5anwjqi10ONw2gS8X86+vNKtGNQ8B5fkIJprAmaWTvbCzdKFFPKgJCILRtFRlbP9QJe8I8pPDTxxpi+HDU811GQMbPE98ngQ4X/wPmsT0aAmVxbgAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}a + g & b + h & c + i\\\\d + j & e + k & f + l\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡a + g  b + h  c + i⎤\n",
       "⎢                   ⎥\n",
       "⎣d + j  e + k  f + l⎦"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAL8AAAAzCAYAAADGmNmlAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAJV0lEQVR4Ae2djXXUOBDHk7wUwHEdhA74qIBcB4QODjoI7yrgQQdABRA6SKjgSDqADoB0kPv/tBqfVusPedeW7UXznmJbtjUzf41GI2mtHB4cHNxT2qC7u7vbjcySURBYIAKHh4e1Nn4kXa6VfkXpzQJ1LCIXBJoQuNCN2MYvMH7oL3n6wyC9XGWXvwWB5SMgu16zb2nk7NuMf/kaFg0KAj0RKMbfAzBiR6VLpV9K33q8Ovijc5JlcOV2LFDYnCu96yqmGH8XQsF9JgHoQpX1XekquJX9dE6yZFe+myFhTeeEzXF3OeWJGgQeKu91Tf4UWXOSZQr9N3jKMTzYyKzJKJ6/BpS2LHWnp/7+pJ4fGeYkSxtmc71XjL9/zbiwh7Cj/6uDvzEnWQZXbuwCk8IeeZgTCfJKiQr/ofRZ6ZkM4K2Ok5BkeibGVL4NPJHpVDK9H1kgPP+N+J97PnSxl+IL/9yELFcei/s6ZzEHTF5KHsYl2UgysDaEbUB/Kn2UDDfuKsMf8Uf3F0rUxzfxTrJNjAejOahL3FNigeDE7uv8UulO6Z7l5TyKL0BfG0+d0ziRESOs1WOIfJUPwOjNwqDTPcir8BmCV1cZAV9kqXjrnAWdi673h7oPbyVsCGfosNc5My2j1oXxsqP4veFc5OzV8uuOeoZGctka9vjWBJivVEjoSTj/rjx6gqzkvRxe96kx9rJhmDTKMQlgoaemux2VR0+Uk0yWv73+IW+wyEVg/lkyhD3fc+WNXReVfrIJGuC/PuNMx9BWq+fik+M4I7rGwwLkpygf4ENlo9urS994vuiqT2WcCci27vKDyrsKjI6BHzMeUKdMq8e2/ktIcRPxNt3o6jdoJAzgY7LEWFE3WWaipBseHsNb4yd8/lBeI42AyU/xtLrHqxOid1KX8dOCY0OjslG4s2V7I3nUKUXiA97I4R8vYFDhzMEntfhEdnWPPVbm1+gGeZB5ntWV/zs0BkHhGw5I+FAv4JNrJmrDPgL5Gk+HxsSXhxO03jdp3HfUJKFvnQAZexYUJrbLBXAoohlaLBNeMIc89DDE2CHRzUI5+DtGvm7qHBCLO4SjMT7uvSH/tNjHkGz6lvWPXiAEu015sdH4g5djj0ZlO3BpaUpUQlaScpWH95WAF+zsiQYSMuaNM2B2JQnwgWRockB0+a77Fy4vPDYDsVwvJtDXZtvWHhBv6iQbeV1xTh9hqmtC9lZqNH6vHEZeGbcKBFyuret/EhpiK6dhbrqxh+SoZFKxjAGgeFyyyh32L4YV8mYy4JMwSOpmBxSlqaejp3aVr+ODwEAHZL1WFHpvhLXe8H6uPTn+hYsKpPNnbx+1jTIU41AXPITn2ui2VQhgYlx4fzd3qyMejlZFHuOB0btY8alIMuFR6N5NJmRkqjZpSbsqaIsTjwe6g9lk8/uSg16O6cy1Rqd883Y/dC9pnnsLGNZeCXkGN96LP3aSlTwuYMP4bw2bUBA9hxM/azX+8IW5nksRDJFGSIMoVBDoRMCM/6jzyZk8IIEfKl0o4ekdkacTwpB49mf1QPlbEGhBYDHGLx0YyVeDKN8ICMlYgMsaerXgWW4tCIHjBcmKd2cQ9VyGT7xND4Dhb4xVlF+oINCJwGKM3xt5MfTOKi0PpCKwpLAnVafyXEEgCYFi/EkwlYf2EYFi/PtYq0WnJASK8SfBVB7aRwSK8e9jrRadkhAoxp8EU3loHxHYyvhZWVW6VmLzpslXV708bCZVrf7uY2XFOqGv0iw20Qplk0zvlN74lL1OPN9vOt4p2YdOoYjufCvjZ0VViV/zoRg/JJqaWPm9L5lupxYkJ3/0VeIXnvzMehZrIBieZLmv9FGJH5BVq/I6z0LChC+53I/8dN64+r+V8aOBlDSlJgddCr5V2vhpbRak58EE7zYHJwQaGDyGTwPgp+9T2Qf20Gj4unewywqv8zgyut/K2wLanGhOTsjLQjRgn75OZfhUEc65NSTf2vP7wqdUDgULrT5kJ/yZgxOahUNUIzyRYZBa7TPJ8/vCwk2rKJSudu2rfV1nI8mEh6GLhdh/HeCzkOfNr0xtkyZ+aMeP7KYwQDxcVcmSjY+4qfhsMokn9QD+yPJT13zhxo4KU31jgRx8Z94a9vAMH4MsatMqKYXc7zj6czasemHXYx7BSgnMHhofzpXcpkmWl+MonjgAwHC664jhm/Fnw8R09bKc2/VUR8lB42vcNEv3aKw7bVq1tn+NCstG3tO40bxnihHwc+dRSXzxqAwsMfTQq9AgphhwOg8n3mxZaN6O74zBI+sMkMdGbP/vhbiYiJLq47hDOAwMIOOPwykckFtJgPDuFyWOqdS1aRXevvo+UzzwdFDV9a8uR/lrAyjm18/Fge+aIbxMI/+xcBBfQo1bJerjq8ngG2bu2S96v6RQg+egMXBRmciBvXXaZ5fxs0WGjdx1WglsHtDlNf1RJVAxY1dCzkGWa/TSq9fH4SPigDx4eDB+oIrvZXx6Z0h6osKqbV1SCh4JFzBJ2sCs0fh9q6QFhd07OjXtGcO9KcgZZEbGvSp4LLl8/eCEGOxb2MOq+yNdx3U2lhhhuXjcKfiGMnCOM3S9MBj5BhY/466PanPXM9kiJKTJN60yYaQclZ/UC9k7Ox4bDV+yWPi1I4vk12MnxIISBB6EFIRlOemxmMW2kpO/8cIZ2vjLZgPt3tqx0fh9i6ElOzB5S4BSGNcGdO5NqxAjJBSlq2+Mt8OHBzhnDOR4WlnChPifsUBur1d5OJPFH62B2ngkuj38pfTHJogSctVDlxLfqZeuh447Hniq+x9UEF4EMFm2BnR+tEQe11NSkwGMIpMa2XtAVaIR2Bw/jW+K+Wwq1wbgyHAruViLeakjU7E512B6D3Yl31hEXRCdEA627ta86E2rVMnMZb+Wkr0GoGOh/ruWq3qgEfLDQoxu9iR5iWDOGsOeOWogodm0ysXWXgG8XzXtOUeZ91Um6oH68PoR708dBfSGejHGL6DpWsNBJV0aawK3vbUuLwyBAKEfIaCLrVUPnfPqQzAdsoyumH9IXjuVJXD5J3B4+RMd6WZrN9fdiUl5uQ8Czvj1AsdFhDuxcosxfgRXA5hiYBljVq5XdbH4cHMxYU+xuILA0AiY5yeUcLGbMSixtCFRjktHILZt6eNs3aY6WaQIiX8uUEKMEJFyvlgEZPys+J5GClz9BxaXwqcHM1M1AAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}a - g & b - h & c - i\\\\d - j & e - k & f - l\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡a - g  b - h  c - i⎤\n",
       "⎢                   ⎥\n",
       "⎣d - j  e - k  f - l⎦"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B = sy.Matrix([[g, h, i], [j, k, l]])\n",
    "A + B\n",
    "A - B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\"> SymPy Demonstration: Multiplication"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The matrix multiplication rules can be clearly understood by using symbols."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFYAAAAzCAYAAAANb/8eAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAE7klEQVR4Ae2ajVEbMRCF7QwFkKQDpwMSKgA6gJQQOoBJBRnSQaACfjowHSTQQeggQAfO+w7JI3TSnXXWnQ1oZ9Yn6XS72qfVSpY0Ho1Gm+IazWazx1phKaghMB6Pg/i9U80b8YPHJzUJpSCGwKWHHVheAiy0Jw8dO3z4VFx+2xAQZs+wU/0KOwts2/flfSICaw0s8Us8FT+I/ybattLqaw0sEyhDTQjdia9XilSi8rUG1rFlS+mpk1/75NoDqxCwa1AsHpvZnapQQFjILLdXcRuLSJfXTFTvWIxx/8RX4n0Z+1PPvgmPvVUbjoyiT3pOpZs2DEbSz9oe26GP4nO14bbKRX6YbXdVaRRi3olZ9E7se6WJdzPxpi3r44l8o4c/MZUup2zenj50W5nSh1OBEY5UYaT0LzGdW8NM5d941xhj1UsYxj+LYwlhZrZE+k5lfQ9PG193rC77lP5925ienzjRlfS6I+Sryhon07ZQgPsD7oXYJQx2FbnvcqaJr7cOmCPT2ehgOPZK0oVn4rE/XEVqz3s3H0o3eqw+oGeuA4ahrLHHQso6lH3RN3+87yiDfj89ev2t2b+otiiwxjPwVj9Ao4zYMsTyh/Ur8dWlA5PpVX+D/W5boukosM4XvmdgWAW2lO+L8d4+aR7bjbF07KE7ivpQ7sgP/pVWW2z8D6qPAmsEA+AcOAljxiNvh+e26s0ND2pYrpA4PtevNBPphXSeLid24a/R89mvLRyYe+79cjc/VoYewQNqQ8t4yJne47XV2k1PVgIIpoz464cKFechox9dtHGV61cMsmtY0qeyO7giMs530AgsEgqlIWCBjYaCNHGlto9AAdZHJFO+AJsJSF9MAdZHJFO+AJsJSF9MAdZHJFO+AJsJSF9MAdZHJFO+AJsJSF9MAdZHJFO+E7D627YlvhFzkYLN4DdD2Cs+Mcy2apA6AcvGi5hdHwQPseEdbPzQhYApnR/E52J2+qJbh52AlUCOSKzQ2q4Y718pASagAi5bp1HbN/SyK73I8/6uxhpHYoTao6ooqOjo7LH6Fo9tFI6CV0RJjrSQx6q3JgLIvbABoJxHPTu97AtE6cdTvovtZjOb3hzJP/al08qVboY/oOJI98pzinEv3a13iNmdX8sLG2q8mlYZRBu3yJsyOvXE5od4Sh+NOWrTpTp0xLTRY42n0EOhCxvPzvtVJztJPyOFVQdHR+4REN4z2GrEtAP7Fg59jcBKEMsLhuGqLmzYNTIXkI/UDntJg+s9Cxup75YlRgijxe3cRpltwNYuLBgvtp7UKDzDSzyT6z1DXL5rau62XiadRkdXBQZAvNXvpSEvbGBskkF80APhsT4OjWqiwDpfcczt0pAXNqKgquOHuhSH7Vxr8nFwMamlo8Bq+LGUoZcY9hXJGGY88kNd2CDGEw7mxEgSE3uTPGguIDEhXdjLyE2K6W0xdkcCzyTcThz8nWNNxyYEZeR7I3XuqfQAJADbNSyTSOsaMmOjkicudDcCa7zWXkJz2xoqc99nS6sNq564cCSuOiVRNBQkSXlllTVCuOzH+h0iviaPzAJshV3th9BDCCK2EnqSPbYxFNTUvZ2CCliZy7NT2CvABpxFHrr0NdESCgLA5iiyHjux8cQKNSsCmy3PCAI+bqpWxWV7P5ZFsEtcrB1yrejqflFpAcsu267X6Ov/LrwOOtqjtmEAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}a & b & c\\\\d & e & f\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡a  b  c⎤\n",
       "⎢       ⎥\n",
       "⎣d  e  f⎦"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGIAAABLCAYAAACLBlLwAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAII0lEQVR4Ae2d8VEcNxTGuQx/Zxw8kwKgA5t0YDqI7QpiOsDjCjLQgZ0KYtMBdGBDB1BAZmKYNHD5fkJv0Ym929XeSnu56M0I7WolvafvPT1pJd0y29nZeaFwpdBG5/P5/HXbg5qWhsBsNrtRif22UsJ4ths8ONM1mUO6DW/q9VoInLaUPlLar6SHivgozVTgW9AaI0nYforrUS8hySnih/jhf/FeDXqmcKFwpxD36uJNkgwnCh9TGG+FImRt9wp0c3r0ZQoAmfIeq977lLpD15RSblPzMvH4fWrhZBQHqTJsRY+g0XIFr3zjN6FHpOphZ2sUoZY714SbSkZhAwqs7ZpkiYz6gGCD5LmuXwmQJ7OEzO2lR1x6efZ0/UwBuY4lS/bZoPjC750CbulGPHkdSCL86lxhX4V3UoLKMDe+sjLUoXCncGFpJWLxAwTawItp0w5dfyEUkuEUPiIM4q4PT+VDcco6X3iPUFp/8pZ3ohI/WSlVeOst48LSCsU2PvyGDBFPlJSV1GYM8KtnwkpELEMn/3XGiD9U+6Uafm9cJBC9C8I9lSRc0LVkuY6YoqASRvFdvK3NWPmfkRydt4MU4QHH0uKXFhrOnD7ZIjolXZ0BvguzJW+lyLiQvrqaYU/NGMXTvSWrluTxcZAixOjQixxbIJaZveGet4vUeMDGNcSWz0vVbUsvceUy/fmgelkobbxEXz5DFeHqDy3fA1LKFYTte8ONZIkNABfh3IVke+flC8uNeu3rxzU7t6T7tkW+pTyHKuIzNYoZlmjEmAG5Zw+XRf4u64X0FPPVB0OsNFF65yXE59zjYtP5XtXs9soVZaJRYgYAp4qZLTxX+K6AK0juliq3DgE409SYmMe/lXy807yPH459Lx68wxCYSTJOJo0TgxRBI2CsqHEHEgALaO7JU4IkBwbxhJSeHfyY6TJZ4nxt98muSYC/UPiigCU6Ik0XuKl4FvWQof7tRCBZEaqRmQGDsiOvEMaH97KIeBblc9WoC4EhrgmrZzx4IyWwrkLPQAnF3ZL4bg0lK8IDXkEf2QSGuKaRRajVgUBVxIbYQVVEVcSGILAhYtQeURWxIQhsiBij9QjerhU45NW8cZdqIzw978kPmEkO1t9uFOYKtlHWCcVoihAn3rb39J5x38l15AzwVGDNiQ2pSd9xJAdrXG4JXNe9VxpGU4SYnim8HBnj1OqwwHiDKLWOMfKDQ28lwHA0RYwh/Tp1yA3Y+tekPcK3AVlsL6RXs7ZGEWotrgkXVdw1hkjLIPZ1T0gyiOS1pogpAzNbktCRQGjdG3h4nP0vVtg0XoCwkQ8gLEyyKFlKQa5nil9R18ShKsYGdsMO1XhTim7LkfhiEM344JWAAOxZs6ft9rVJKEAYY2MQffkNdk0e9HCDHDBYHp+CnBWKMVuVZpEoAZlKz6Tgnzxh2FWhQaRe0OzJBhaYbAmDmD8t5MYHJQPCN8nm5FCMeyg2kxMO9EqUb4fNdNmPUMSPPqvF/Uou5gKIKQ4OmBQoAMsH9AMBwp46SihNyJFywO5nExBF/ONvLLZnKTECJFtBCoNleQU6FsigzGTBXNOV0l9OoIxmfEAu8b9fJrdP/8ueDx4jrAIxBARCsl+0OtaM3UCMEnw933yMTJy9OvH3JSIM0nBImrisrQgxhrkdr+GyNDVWGDG287fPo/Tct3YiPokPrmldWgbEuvX2LY9rao7x4A7UC1jvOVbMWauSv6njvC3H8nGTSeeqxlAEPaJkY8XukdRgDGGBlMZ7TXES32Ymmco82TXJyjhc5o6fK8YPYpGDBUgVeFvzJylCwDNPdkrwgND9XssSumYH24rfaO1Kck0C/FrKwPr3FeOX+aGgzVZGE+r/WFGSIgBIwDMgVRoZgSTXNDLvWl2AQFVEAMaUl1URU6If8K6KCMCY8rIqYkr0A95VEQEYU15WRUyJfsC7KiIAY8rLqogp0Q94V0UEYEx5WRUxJfoB715rTVrgY8+Bdf99BdvwYM3pb4VfFL5qDerMr86+9enk59xT1kXBvrJJliIkeThiBC4Q2PA93V4YsLS98gtmem5f52I/lnCiylkA1KXbj+CCvQk+t2DpLJf3+pKXlRkSi0cf2Zqvmg3h0aeM5MBI2REMMWCvZim2egZmqn7efQjZW7ltiB+qICXDHbA90kR8eCTUPOkIko0SZMsqh28gGPGJoAYDXbNPw955rxXrlT1ClXAsRFFj+Y3GfTqWTwaXjzSfzodKsn7bz3gSKyBDLFtjcSZXjlh82Zt5goHHgfTW7woqvZGvc7BWZWgVYpwA5Ebj3IsYC/iMm+VzifpDfutJljZqHPBcJVss76gy+Mo40rPwWT3S1WNxV5B97+/hruVvpyKCMgzCbafnAGGhsRKANKzUHTrTPdc5aZls9Fb3CaFcMvh6aV8bNuaSOg/fpSiiDXAEQOux5SMAvYQzPjx3Fqs4F7XJhhLowXaw4UMO5kGvbPtQFq6H0/J2xmqpCL0U4cEE9PhXMPEpO2ME+NZL2NfutAgrmBqvkI2po7NS5QGQ5uxTKo8e+VH2wrEe8cQ4P6vtNt1fWc3uyqePDwG27UOFKMcs7jH3wzmnIwnDccecAMBzmWzw5ZeuKIHPgXZaJZUNIdXNYTZ+TWrvEPw4Jukdiv8kwayJLwgz/cwmrOqvFCHgjYQXvlkv1xSVr7cZEKiKyADqkCqrIoaglqFMVUQGUIdUWRUxBLUMZcLpKx/yiFnUfwgYIzLwXtjywre/rDiKWLU6WKezy5BLTw9/Cv2k9L8a+PAxwFCNpQAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}g & h & i\\\\j & k & l\\\\m & n & o\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡g  h  i⎤\n",
       "⎢       ⎥\n",
       "⎢j  k  l⎥\n",
       "⎢       ⎥\n",
       "⎣m  n  o⎦"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = sy.Matrix([[a, b, c], [d, e, f]])\n",
    "B = sy.Matrix([[g, h, i], [j, k, l], [m, n, o]])\n",
    "A\n",
    "B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAAzCAYAAABSd+U7AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAUj0lEQVR4Ae2djbXUthaF4S4K4JEOSAc8qCBJB0AqCOmALCpgkQ4eqQBIB9ABkA6gg8DtgLc/jY6QNf6RLcszvldnLY1t6UjaOto6kmXPzM0bN27cVjiSb9++XR5FtohmgWaBZoFmgWttgZs3b/bOGReyykeFr0l4ca2t1RrfLNAs0CzQLDBkgTdKSOeMN0wmyC+6E7kZhd8P0e2zWaBZoFmgWaBZ4LsFNE905guluPnCJpPvmu2sWaBZoFmgWaBZYKYFrvxkwv6ewluFrwqfcu0j3Xs+X+/+YG45TW+eBWTzRf01r5Z87Vw8uXr5NTfNORaQ/Z8q/G9OnjV0c/tdei8UPil8U7i3Rt3nVsaVn0x4kYDbMhn+s8K7GR3ws3TvkH9GnqZaaIGC/iqsuT97Lp5cvf5aWuwKFmCrZfOxmtvv0vtD+NyzaJ3/s0J7z66IXUwmmsl/VnhSaD1WA29zy1CH/6nw31z9WG8lvHGR1/F8Vn+NGWil/sjFk6s3BtmlrYR7sp6roKCx+qMCDnu2rGTnnH7HnyyaSFbCONs2czLsYjJRg9hqWrzdREd4o8y5M5ljx1S3CG9a2HW7rtBfRf2RiydXb0Z/FuGeUc91Vy2y84x+xw+9WmjsIowL65yVbS+TyaxG9Si7bS6tXC570lrU+Vng3PorFw96bKs2np0fp2oimuSHJpy7AkDYakFbs729Zd/qjR2I9AbhVpLB8q/C3woPNXj+1PFIpP9QkRjaHnyj/7P0Xx4p141gRfCP8Dz11fyo41vhAE8QpTP723Yar7+B/WQiPOyxYmfkB4VXwkQ7aA/YIKfd2rNnjO4DhffS+1N63Hr/6uPRf6H4k5BZWAxrDndo3zvlgT93FOgX8P8u/Dz72lqy+CNQDreB8/hpN3z7Q9hp++YiHEU82hywKszhi3RsvGLfT3D+FFhVZw4/0LkhjIu2udZq1xAX4vITHfzJ/4Q7y2/g6HHwNHQwoKPAF1Xump7OeQbxTeG2xdlRcRD4Y3TNoCI/Tnywnr405cGpPO1Lm4pTPggHRr6c6XBGcaEtlCPBaA6bzsH6xK7nHJVvMV6PA1vRL0zUhoc3VZztdGRS0MHZnz4IttG5tZdJMfSrzsH01crb8ggO6lYI9tZ5L3cUb/jpr1ifL0q9WYJb+Rb3R4JnkD+RnuOMr5N6jfezueTLCH07t+2+7lIehT6YW/9SfeHO4ov0bBw4/YL6tuAH/J3t+6xNyrsYI2VIRn1KohP7DRuPgzxQ2fiatxfUMiV+BYAxWF3FK0POPyvuMi5D+q7hivvJ4n0+gGU/BLe8hUeIhvxkOO2oOHA6EWYMwgRoAtYvdrHxERv9LZzxndNjxfGKc/yg7z64pBevyFjJIzyQjFcTtsI/pG70OZc7gmX99Zvwx1wDMX2ytRieUf4IlOlxR+XOff+BmXbEfbFVG9bg0aY2z+WL9HCO770hH+mYcmUrG1u/5/Bja98X22CQC5GS6QSuisOX3raTX2S/FRU0doqThVSvEyUMGTs8S/5LJ+88EBenzscJIn36LkE6rL6tc1yc/3AOUul9DfpH9UCmIWF7BB2M4kTl2ABh68iJ0sPWm9JtkglGNb34WAOvL5OB8jyuS/j+wzXYdc5WF20gxBMgKmbnTn7FY4fR9pC5gszljvVXuh0AL9I2deDW6A9VYHhG+eP10AHnB/WRszV9pevRtwJr4PZllvDIjUOPX03YTHL58kXYzJewELTt3kGgNeysyib5oXoZk4xVw7spxikuAGZIxwOFR4RJmdzmUglH21OKwzjcP4VbInWuLp0z4yRs0fh4nlcs2mahLIVFt/vKx3ZJ2L7yWBgoRxhJ8+lMauzBuuu5R+UtwXtk6776fR1K6mJUPNjD9qKlK45yF9nQylhy9PV2bu8V18sdypfAR7d9YfXpGiKTeM/i5hyVr6Q/svijOsDtdHXEIS7CGrdLZZTgLuVR0bZM3I4552rzEW7FjfEFG8GNo6323HqVt8TOk/xQ+Yt9n7WhEOORTa1cO6r8Xh3F29gb9B3Sydvm0oxFRxLSlSLbLjjbdLV7n3hJqn+qlTGrAjo8FruTSbGbDpPN5CrClNc6jti6rwoerKc2Rg/snXapXOLoQ9cmX48u68pIe3q54/Uhb7odwB0p26l97a3biMPiaJQ/EW62gcEK/o+Kt7vE2hg75Y/YvaPnL4Z4hINlQnF3w1636mEEdy9fPJhnOrIlfFkV3HDhOf4l+D7fxuHSVk4ZsWmoaUIHPiOT/vDioJf1afuTpoxDdoNbYB4q4ASCqHPDHqYHi0NLnUTQr3ySYoGcvBl0REDfjj6HVhmim5gND6vcIxE2bGjSN2kwYfRhhxBs9X327YvLsfJqHnO5M+Q0WPnYRPjE86km3rTsKf6kuD/4AtyYEN6naYE1ryNeL+UREwkLRdv6xWFvKVl88TzAmb8CnK65IzyFTPEj9n1weTPJ4cKEDnj5Ando4xD4i6EEi/cVMWm4gUG8Oo0KuLZB8yCqzD1XkU7Qlx7PUJD0mcshtu4nTijGwmrrtfDaQElrd45W6Z3VfapU8RpcR3vsfqB8oV5vWyYON4iI85I6NYun/dYeJlHnmC2x1lH1MDnO4U5YwSWY4rbyYgHlbiU5/BnCbQMwPJvbCrTqKeHRA+W3hSJjna3T6rKAL24XBD77MdE7eVYGnsMPILCQg8enkEkuCBQ68DiI8LL4x1dOPo8i000FOgAHY86G+I54IzAhsGJgYODEGNCsBIjjYbsjn85xdjhkVsOmjxF5tsL74LNF5bFS4tW0+K2lrHI8dnDSTupn/37QmUqfyYb9145hFZctJXipRPnBi/x7OLjPl8LknKjSsS/PgTr2VDwrYJyt3Zq6jB4P7cEGbAmYk3PpNT9UN32fxR3pQl5e/4XYQRQf7KG0JRyoyh+Pm/4IvFKc6ws1ApuHvguNyjhRGYtxU3xst6i6gEXpQzxi8cEEwliIH3RHxdQ5FaZsvoBA+nCGcJnyhvQcURmL7ezxjvoX6TAhs0AEY5ZjTnGXYKQs5Q9jKCo7cKFHB9/CWBycF6wc375HWZOJZVp6VGUMKCacjpPLLa/UkLn1oKe6eBD1XFhnOy2rZ0u8Vmc7Dltgr/2xV9zDPXGeKXuw8zljFDYmy0cXa3avCuVn298osLpwQpxObKXjY2cfWJETVhePl5UJEwlGAXtnZUzaTKmGdyaOpn6wwF77Y6+498a7Pdj57DHeWrnXeVDHrbMTP6mwxcFbLmEbzCdnH3JutbILixT9RMdE8spHcwv6SPXRcYulFt7FgK55xr32x15x741ue7DzHjCuPZmwz/pF4bEcNXtutxWYSCb33aS3uQgXX/7jLuSujmAffXa0OcBWYbNAs0CzwE4ssOpk4ieNs5w4hvpDmBc9xxkqr8U3CzQLNAtcRwus+szkOhqwtblZoFmgWaBZ4MaNNpk0FjQLNAs0CzQLFFugTSbFJmwFNAs0CzQLNAu0yaRxoFmgWaBZoFmg2AJtMik2YSugWaBZoFmgWaBNJo0DzQLNAs0CzQLFFlg0mfBlPwV+Yvur/35GMZBzKsC3j3815HsyZyHYWeGFD2eDa65xhL+XO4q/rYDN4RQ/v7Mb2RN2Yd0Vj4S3ly9GDqU/pU12fW5HYWPMflL4pnCSvyTos4mwrM6DRZMJX/ZT4IfLcGr8yNpVE77Ff0dtvDyHhqnj+ZG2Owp8U5+ffAm/MqDzXckQd7C1Aj9GyY9Q7u27SrvAvkceDfElIj3fEzuLcRphCqfCz69quB9ZpC0h4YQntXiwaDLBDgJkDq3awKcOBZznpqJO5/f7mSxnSyXM2ICJhAnlg0I1m6vs6jLBHVZvqyxQKvXFmH3OHfsueTTGF41TfiV76S/xbuVf8CWLJpJKHK7Cg5JvwLtVpDqy5qqAOx/CnmRVzH4gUSa/unzJcU/GGMDay50xpzFQzlT0qn0xVtm5Y985j3r5MtYfmWlb8YOF99KtuFUx1uTB4jsTGQcDXQXHlsm7k6nVGkgna5AqHuLOntsKdra7mPDPUfZs2yG+nKOdO5jkvO8qgnAuvrIaD251Wj5w4Q3CrSQDhT9swjDc0j9XOBLp80u8gLYHqfxpEH+OVfrT7kd1TUUICzP7MwX7oyl+gJIfn+wMeq9nW2q/KB38JxPhAQsYGEhfdG1/VBR+S0xxpKEDWe1Wn3Tayr/lvVc7/pQeffWrj0f/heI3IbfqNmw53HFOQ3ngzx0F+g68/ADnZn/opfqCCEsWf5TBYbeMvg20vZdvplf7KByr8ag2VsoX3km++D6hXdj2Exwn79bicUz5FnjBXyAv2uZaq005PIjrkj7Pecxn4kv487dJn4HDx9HT4KNAmgJ/GMU/Hbp0nbOn/U3htsXZUXGA+BhdQw7y8w+HR+WPxSkPTuXpmM5YmvKCnfbdMz3OFXCmHSyKw1jWPvA+ses5R+UrwpzWpfKwc68NFO/aoSP9QQh6OscJkpdBF/pX5+D7mtZT45p6qUthkjvSMbwfE30mUf7xrdNfOdfKV9QXyp/FH+kZdscZXy91G/dnc8mXEfozp71jOiqvlEehD8fqKUkTxiy+SM947/SX1lliY4910rdID/7O9n3WphKMVkZ8VHmDPEBPAmc7c4Kujd+9HFA6PubthT4Gxc+8GIOVfLwy5Jw3ui7jzNJ3g1dxP1m8zweYVR6qWrlTR2HBKNQJ8eJVAQTsYJEuxnBvXOiIgPeLOzvhh28DCI5WBEpjUrR23EdJ7YxXaKzsER5QxvltxX9IrfQpfNgwmzvSpV+Q3zxnDleHT8raVLzts/gjYIb9nfK5c7WBu3Fwn/ztNN8W7BfzgGvuBHJ5VLUPhCOLL74t7x14/feQjrFf8tF1Dx7DHG7YOK0LbKJ0jxutIx5EWcHKX3sHHZ1fKg47h12RSD+c3gpn/Sc4WDr5dZLMgAn/dx2l/aVze1DsotUAyIr06bsE6fBwygaki/MfziEqva8RTGaQaUjsgRffX+D/uH/wiqwSgqGI03XYfpMuEyLS0TlEff+shPl7BYczZzvhiydD0/lMvHDQP4R4MkTH7P6ci0jYNhptW6RbcjqXO+CiT9O2wou0DR1clfoimz8CA3YGHFg/qA3Ovr4to28FVsIuGB0p4ZEblz390qlghYtcvnwRFvMlLAJte3cQQgUbZ3FD9WJ3xqbh3RJjX11jPGBhQbtYhPeNN+IJo9K5pVFHhe0E5TranlIcxuF+KGydkEcCUE4eJmXgyBdtq1CWwqJbfeUDy+ztEeXBoOzDBjvMOVfexZjTelQWA2wUi69PWbt4FU87wnajpSuOPl1kUysj5+jr6dzeK66XO5QngYed7UddQ14SwzZlTt2mo3yL+8LXm8Uf6YKd7TlsTp8twmu4vT0WY4/L8WWV8KhomybFMnQtm2X7Gt8m7AM3jrbah+pI45V3kY19vZPckN5i32dYl2K0/PFRZY3yQOlHfUB+iY3DXr+h9PFtLs1SDHxCulJ8rDgcbbq6vU+8JNXfaiV8qL37ueQWmJXY5EqiW021Kybo1J5pZTxY79OhHZ0+Up8SR5+69vk+1uW6Mpc7Xh/CptsB3JG6O7B1EWaXNsmfCDtbweClDfw6hFsFZtdUV7GERzhcJhRWrnBndfHlzvE1YHimwHbM5eqA8gqc5IaKCb6vlu3yoAatQR6M9AGZ4TUy6hcvDjqjn7Y/aUpsLTnnJQAPFXACQdS5wcgeIA4sdRJBv+JJwJHWAe40jmvflj6n1qe+RRwTdGr/tN6+SYOB2dcOSMFW0mffVvLWlBT7EHeGFiiseGzie+L5VBNvXHYuf1LsH3whblwIM6vTU8tSHrlxIr7YNjAOvKZk8cXzAMf4CjC6ZsW9peRyI/Z9cPnUMsiDaFLmLjsVsPNF7sF2k+EizWXXvnAmDTcoiFenUSjXNmAeRBW45yrSCfrS4xkKkj5zOcTW/YRgHWcJCRXYiuhbyYPG6atNnRU9CVuLtyOTwiCWSMcNqghj6uAsib6x8njddnSlYZnmHhdwJ6zgkrpov7WNFwm2XIXm8mcIuw08e1aXNG2by4gj1u9HFUc6ZmvTeaATWzgy9hk7q8sCvuAU2R3522Pvc4Cr44wKzOUGWVi4weOTStTHgzwQQBYN8DmI8nEj8Fq2nnw2dSvk6j/hray/VCCrKwYFZKMyfryMuEA+CKE4S2OFgf4XBbYptnQCqtIR7aXwMHnQ8fa+NAS0Wzanl3wMOYZEbZNLt00ivEMTHyCYHPq2gSCvrSbRM3muk19831VxDFaRjtnckS5430R57ZS3034VXp7PTZLZMq1xVH25/AF7sKXyMQ7A+ruOODlsfkop4RHt4ocWmUh48G0TZI32ZPNFOHhrjoAP4ouifVyvgdGVSX2qO8e34Gu4G+d7a5vyt6fxkzwQRjjrfkxW+fGZPyrwHHNsAgpV3dQZhGeVmpUh5Mw4ETDK5u2uMQc+WJLyc5vNu804leqi+ngA9bykvrUwqxwGMj82CRmbLLDAWn2xoOriLGthbzwa7oq1bDxcQ3nKWhhr8kBls9h4dFHeXLf9xerljQKrNCc6ZyZk5RxWbYeUWZ+X0iZUEY/Z7QvrHIOAv3SVsxizMPAMylbo3MqHO78qBrj6hS7uizMwzWLsjUfZvbfYxtk1lCsuxrg1D6a2uXJN8UyK4fmEGnFb1zwv4Q2XsW2a0fKVd/W7JatQGJnsmEjMYXMb+kh10nmLpRAzW3Jhj1VlVXmmsbhxO8tY2BcnbW0h9sajjN4rtHFGDeUqhRg35cFakwl3HzwfeSwnzT4bkwkTSbXJQOUXibDxhT/uQu7qCP4qW30zQdL52I5j296aabymHizQeBRMca1PNuVB1Wcm17obW+ObBZoFmgWugQW0GF/vmck1sFdrYrNAs0CzQLPAiAVsm4utHrZXgmgbqOjZQSionTQLNAs0CzQLXBkLpHOFGubmDtvm4q2rWF5qMln0Om9cSDtvFmgWaBZoFrhaFtBkwhcZwwtXvnXv/g/CJNfSzDb4NwAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}a g + b j + c m & a h + b k + c n & a i + b l + c o\\\\d g + e j + f m & d h + e k + f n & d i + e l + f o\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡a⋅g + b⋅j + c⋅m  a⋅h + b⋅k + c⋅n  a⋅i + b⋅l + c⋅o⎤\n",
       "⎢                                                 ⎥\n",
       "⎣d⋅g + e⋅j + f⋅m  d⋅h + e⋅k + f⋅n  d⋅i + e⋅l + f⋅o⎦"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "AB = A*B; AB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\"> Commutability"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The matrix multiplication usually do not commute, such that $\\pmb{AB} \\neq \\pmb{BA}$. For instance, consider $\\pmb A$ and $\\pmb B$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEoAAAAzCAYAAAAw/Z54AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAFT0lEQVRoBe2bi1VcNxCGd3NSAHYqMO7Ajw5IB6YExx2QkxJwB1BCcAcmFTikA9NBwB2Q77srae/VfWmzT+DOOVq9RprRr9FIV4L5bDY7IrTo4eHhR6vwGRTM5/NOPBz6d8JDFi4AavYcAzh8zbAQm68/8SP9CijzWvi0KH5+v2DQwAIEKiwiUM8PkRVH/PMYP2v2GJ7fA9874jvzIP9PKKuiwBct0XVuu3P4rut820yjwxv6vyK8RW7Lx66roz7qhI5bPolyB3tVryN/TnDdpjakBabh18j/RpDvQ739ptP0r2zBuSDcEJR5lMuxjLCyjrRxHPqtypmnQdcFUKcCXULvKb+PvKTPCA1QbBfKbiLftuOaHl06/y8d6bMCasxHncB4j8k66Dq5nI6CKVvuMvwRgnmt0/wh0Vo6jvkoAXkzMOgKQOrle1FHBRA/hLxLYu+0ro6DQNH5ac8IdZpaTcOhR15A0hL1ZZ/guYzlhxSvquMgUF0DQ4Ag1XfCxBbqBOk9QRD/TpUHklhHx95dD2vo2gnlb+wePXwuvYaD7+LbZBnyosNuOfMuOfCP6ghPkTOHb0nMhv7mGqHxvLSszFLwfKFIh35Fu8qXZSx7z66i49iulwbDYEVWC2uBpDkbEvMyEZeey3GvtK6ORT4KIZro6zpIlOmnpDuCB70ZZS/gObRjgXpp0WvpOGpRCNFS3gNA/IwRE0nw7gIwgvOlA6R3MkI7+4xZiGv+bkLHQYsKVuPp/Jp0/Tz0kjLPV5+DSjmIzqJAOpMeEQ7BytbScRAoBilILrHKPxHXKZ2hAOISYE4yMG3nlcXWrQm51acW8qIF31Cmft/iZK6r45zO3O6d9a0PCDmPjgBcIzkd9VGPbmRbUngCqhDYCagJqEIECtkmi5qAKkSgkG2yqAmoQgQK2SaL2hRQnExb1yeU1R8WCkU9braxbz1H95fAEMdvO9PS20XU/A3A+u3V+QjZ5N5MDpnH9BQ/ev3e8+qn9UirNHjzj/uPfL4VfbQPXgXTt/UGr3WNfTRoXLWaJwiOSvQ+QqJQ62p53TLkCVLpI636n0WZpF0tvlEex7I8pq66CiauBt/5AGojqKFE3lGeh3+le+u8/ap59SM0Js4+oPyRVr70aBvlUOYrcO8bAHUVUE/BmZ8IQHAPJBN5G1L3pd6P3abaZUKX0nWNtOQg9RSAEpBbLKTPzwhW9Kv6rpz+tQCe1qZVZyxx5nYi4lHYa9KaanTu9f52nkaPokdaxqBu3szm9Eso6KpLvCVACdCfccYQqPP8TryT28uk6QoJdNM61DPuhLb2+cxlmlO0JPl7aXTpAZCAJLMm7TrX3OvbbK+APVXouC/RNd7pq8ZHfwAxgRUAjWPrWpY2qWgUqMiYxYJ1jKDBWcja7CSLTk5g65E2TPYr6k7hOSN4zFH/bwSpy9EvavgdXHp05tb5EiGdh0vqot9KHe4zgb7V7oW+rUda9QpgNepoE1fGIFBjFuUptwuMyvEh+CAcuiAwYLf/1iMt5WNW7xi1wLgE7a5FY0C5zt3lcnKdH8yrDWDokHsfaVVeIAmN81YA0bZ1py97iwaXHtwXdOZRIJkreU/eUt+2vKjd0W8YrM577JFWy8odtsvOp7rRlTEIFB3cosi5YIVxu+QU9oq6hqnCo7IuU01Zaj1CLoo3/qtcQeg6XScA0PczOnpm+oNYJdS1+K+WpwdQIRsgQHUCpgfQAYwaVWPOvMH8nDMTUIWzPwE1AVWIQCFbPB743eZ2mSjf/lPFE0/kODDchEu8D/f+NIbeq1EA3Pi99yH1CQad/9j4H2aBZIH3HIYIAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}23 & 13\\\\51 & 29\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡23  13⎤\n",
       "⎢      ⎥\n",
       "⎣51  29⎦"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEoAAAAzCAYAAAAw/Z54AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAE6klEQVRoBe2ai1EbMRCG7UwKCCmBdMCjAkwHECoAOkiGEqADSAUEOjAdJNBB0kEGd+D830VS9uR7LOZ8JPg0IyStVtrVr9Vap2U8Go3eKS+k+Xw+WyCuAWE8HlfiwdJ/KM+zfCmgRuuYhcM0wwJspm/0h7QvUMYmn/4hr99fYVDCQggUWESg1g+RJ674bRu/zuxmRFUl55f2uZC/U7mQxH+eEa/F+5DRVtqUDlsScCq5jSfDyxeVxUdNNOmCTxIdYEr+Su0TZc7tgR2jNgCW5lL7Bprl66Me9Lhpk+XhEw/rTT5K9coE04mQPzC9X0P9zNCoAsqdlLOWBtA/6ewrSddPHllevjhX29HjyMxCLsYIiJmExPFFGYDE3Pdsh3j3bXvVdemBDiV9q2R6+ezYRmeOdShvUMZBARSal5GmEuuaiQ8lXzIdSYcrhwJevjRVm0UlRioCaaICZ42jtAqxkw9hp45U/6X8QRk/kUBWe2VJsjlydvMqZXn58sEuoAIAgLSrzHH8HidSH34oph0B8zk21PeofCzabaStopQMfkiw6EZ/6OWr0rHx6MUBUuBB+UL5ULRr5XsJtQ4e1i31WyuDhuP/Il4LJvSuU27hdfN7+RbGu4CyowQG1oEvugEAtaNfqtrNe/EB0o7ySpJ04JfZc+RcfHVKNgIlJbbIFYPj0eM4kgApAlYQsj8cjc6TdGNeNqtqk5I8L18aUFGp9VGaHEvAInDi/PI1AYGiTWA0LgQZSyZk7ko/7nA2sbmbgY5sPnRb+bTG5F/tZLFeuk2LOd3QxfCovHDDDXRu5+ymihGWperfsYHOkXjM6atuI1N5Qe9crodPPK6bOQizGylpl3DiWBuOsbAylVwB7tSXvvNUh+ej8rFy3wnZ5Lbk5RvVHj0kCIArLXiibJ0l5s5TROl+pPY+QBne9+LbE723D+IgO/5woDebPJUOFypT8vKlAarwLcLRwzpKC7dM61wXqBy9wzfrDMJT1j4A5URrAGoAyomAk22wqAEoJwJOtsGiBqCcCDjZBosagHIi4GRr/Nazc+gqz9MFzxnb+tyZ2T7q6t9UEQOOfGzSrg2Uqq/z1KZjFCi+9PEeaK4gbdMzCwsGHD6KeZtKTysCyz7HwOcKlNpxXdQl16UjspTYvNJ61WZ9tUFa9RXPLCrLA+uUFx9RjjqgYl+KHouXBcB/Xzdn13TJinoU72T5/OiinG9ofGFIG2/Hib8Ayn30NKApuQKlTROsuk/HjXc03MdSQdpOgNIO8ESzYRcbFINk37IsS9/1MwmcSdfZMoI7ASoXLJAmolUFSnPWPttY09JB2k6BEkAoA0gLgdI+EcllSS/8ZUxLBWk7vXDKrD2B0qjwS5RLB2k7BcquXKCVAqW2r++68UtVYTNXkLYToDhy4djlGOSB0ry/zzYgNTly7li16dk+Kpx/b6C0VpEeOgCqCYwqa0tqPduiglmzU7fGxKOAGDr6FyI8/ApXAbUtOteGRh2fDVRAxBUojei9RBmAWDpI23r0dLT4FuLnNVoH//LDTfybhBeBRZXuQOkqQPLoiFzpuXSQdgiAtuycNmEIgLZgVOruykeVJn2NjQEo564OQA1AORFwssXrAf/GZ7+w+Sltuu47p///2HIctIKEC2/IPNnaXHouFWiVz6SvkS4ceBq2WFCf/gbwa9tlU96O/QAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}36 & 44\\\\13 & 16\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡36  44⎤\n",
       "⎢      ⎥\n",
       "⎣13  16⎦"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = sy.Matrix([[3, 4], [7, 8]])\n",
    "B = sy.Matrix([[5, 3], [2, 1]])\n",
    "A*B\n",
    "B*A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How do we find commutable matrices?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAKgAAAAzCAYAAAAD6kM0AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAJ30lEQVR4Ae2cjXXUOBDHE14K4LgOQgccVADXAXAVAB3AowIe1wFQAYQOkg4upAPSwYV0kPv/tBqdrPWH7JU/kvW8p7UsjUczf42ksdfW4cHBwX2lLbq5ubneKlwLVgRGQuDw8LDWD++pvR9Kv5L0cSQ9VrErAk0InKgi9cMTHBT6UzPmYZTebIrX3xWBaRCQ71V8UK06HzQHnUaLtZUVgZ4I3EkHJZ5ROlX6pfSzJya3hl22fVL66FNtDDeWMWozC+NcviY976SDcoPHkiGjL5XOmoy/zeU4pfR/oPRV6bXSM6XJKBfjXL4mxRfroOqAZ0oAvws90sWnuwhY8LVgg3PipOdKvQfixBgP6osjGbZUYskavGwBvjesd8ctFRDTy9sGNmfMUBytrudxEox36YvFzqA9ga5jd0u878C6+ttcthTbcvXI5dvqk6wZVCPgWFe+U2K0/qv0Xem5Ov9vHSskXkbleyX4oIdK72ZwFGbQC+nzFiVE6HEqPdC9QuJ5rgJAtBsqeJ6J93OFsfBJH1xpWvws6+iJbVc659nhlfSc67FgLsbwnXmcCUnwEex4I925T2glOoXOOKhL1CnxAPXY6pUnrrtRum9lHEXwIu+RlZNX+mjnuUddg9O8zeWP+XQdAKAQf0I4HaOyYAfXiLjZ+GHXK89gxF6cuRaTEuWSn41r2p6uRfFB2MSyJGN0jNVG3BcBe5UzuE5ifeK86hiMp61LvDwe4QhiBow9nfylyq51dCReOhbHxRkvNqXul46Y+kaFNqGnpqMdVUanOPIjmhn2qS/CIbENu0fTuQ+uppcdPc6cDo07TdSuxyyM1YjxvfLYxu2CcysdtdZuZheEfEv4aDRdKj95Hp6P0em/+3NmokYwxct1ZoS/xB1YCljW6pavC8l8ETMneZYPeOIBZGCYXlzyRcluNDinPWZ8KLVvU1rml1k7F9e0RaefbIsngZSncj4zxtYXqb70+YeKojUnXQ76UtekHQiwNlvGIp3TCrituDRmSvPir3NAHIWZjiWhlzwv/7GO50lblEH/8OMdEVtsYFEMYQfPUeMVw1UU/OmDa9rsExX00m0ujL3izi9iI4Q9/gP2jROX8Tcu8RKCAFLq+YDLUlgnvBdwyBmJmGWIP2OyGdf0NodN7WPEG098fZH8QFzjtrEt1TmunyrfibG3tW4yY1IiROy0o9FBIyvdjBOd09FOsBR4roQCUKNzwrdhmfQ36OOBYmBx13gda6HzlI8RP1r8GbWdi2t0icsysNJrU56pzlPsUoybJjNugFwIpb557funVudGB/UdiSOaA7IsIphzWz6fRB1MXEXnBqJhJZbQzpESLiqTwfigt/InSt+ka/zYyMXV0i/mIyaF0ph7U1rgdwCuoVWva9bSGC4aL5ODcdNqhA38CwY99JhszpLfo+Q8PeXu9ouAeasjNxcIpVFeUKDMGmHJ/6wyHBJHtWeglNfGmOIZk15JuOnI889P0sONWGsUUKSr2cKMhH1XSpWnE8Zf+JiNa9Ju7xuk5PqSp50YqzEckckhJe4r/hL+PN7k+XojHaqG55YsfaPFXY2tt1RIecKCoTdJLZKbq9QmWHBTOMegalbM10g/VqMH0s/i6c5r2hjmwLhNn7hOurFav7gXFy4sT6xIKk4y/pHSiRIj3BFlyrDcp3f1G4aZfqUXcb7NQsSfYdUqoNJoGBfQzYnoWuJLtdNbzsgz+nspFOJl76jEn/whMXW83IUNIdOl15GQqRKqdF3cVj8yxm1NZ9ct1kGzLRjGyCxJvPlSHU+MykyKcy4qzJFOEA6KfhyLLO2Sc2toLx3UO+ISnXHLcaRr/ORhq/6uFyw5Br3r2K/2ZSCwOmgGSCvLfAisDjof9mvLGQisDpoB0soyHwKrg86H/dpyBgKrg2aAtLLMh8BeOKiedfLP0Q8lNnJY1D9FQ7oeG5Rm2bChSV/pU4uxynk/Y/AmGnvhoHqWyNv1fwhcHnhP8SpdUz/uXI5jSsgDpa9K/F8d/hFTfjZqwljlO22isTgHVQeU2LBhq6OQ6wtvxQP6LQP+L8ApcU6c9Fyptz0zYcy7Dr0nhyX+k8QsRypNvFo3xat0pfUO8vwgAxv7DKe3c3phk2K8y+SwuBk09Eb5DDPo0A4tr80wiUsfZE0YD9Y7ewbVKCD2sReR3cvLxB2Gs+oZlbwlZDy8hDHHhg28+X9M20rXXh8ckyXmg9IWiZ93TwGR90Eh3hgafeMGGurC1fOwrKMfDrCEDRv6Yuwc1+NMaIKvYE+RjRvobDqOnUR43UtZ985k2NhA5ygAz6wbNnjd0CVrownPz8CbY+OGTlwNbztKz9k3bOiLsXTGGdGbjxjLbtwggRCB7XcpFr+HyMdQLuD1sxX52TdskC6Awcu9dRtNVL6TFw+zADMnn67wCYYj2cmHYMjpHdBvJGT/tuKaSvE4UzxrmNIXY+nLhAGV37hByvDMkJFeWRrVib/Roid7rsjzLvt2iao5NmxgNsS50o/eACkeYDp19EW/dsPhCmQDoQBUx7+p2fE3E9e0FaeXsA9hVcqQnvt2zEHiapZZBmjdpy1dm2L0xZilHJmp3uhV8St0qiOW5tq9mVTeuUeReJi+G/fYkWIuLMg9Spab1XL5Y746fVVmS0zFRpXT4egeQhdkiRhkv2K5pfPIVwohUo588eMYP3N4u3gkZxKM0QOdlSp7c+mcSY/KEBKmOquOuLt5byaNLjqWlHq+irYofB+9VTNRQYu+hCMMknRpfOxVS+1jxKe8nnX3Q4ueXcIZUKmuXdcUrW/RvRZjz48zpuESM/duGzeoQ6+9dYyALVLjtnQ0Oqd4GKlTE58Qx9S00YTjkZ1Bfw8odqWAxvJ2yvfANW2HAZXalvJMdZ7q0YRxreNKSWZHF0IJ82EbN3hL+dyAvwgrJKEsN1e+kLw5qyuio5WITScb8b7jaY8R6wjjleH8fFNyEG80McvGDV6PHFw9q4sVsYHVbLSZPTTWkhmAcdNqhC1ffVPDN26QQm9wRpKE2fNN5H6wmUDHJW3YkL0hAvrLLgDEPmaEyTZuyMEVkCPqfYMUXVs6m42xGsYRT2oUuL0bN8hZCAsm3bDBAFTbhDOL27hBerEa7cWGDVFfsPotcuMGYl/SaKQOX/zGDQxUJZt9iD9tSSyBy+gYl1ASGUelBJWSo+VvijjrvfQNcbMcgaWIZ6JL2riBsGpvN2yQ7Y4W56Cm2MhHlkxu8pa8cQMOysDhuHcbNshmR3vpoH6WnmKmNpx7H6XjXm/YYIDds8x6XBFYIgI2gx77OCzoqBE86o1KaGjNrAgIgdT/VER4c2D7g4aH2xSKPstB614k2NSuvysChRGQg/LvXbhx9eLP/gOgjE4W+BhF0wAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}a e + b g & a f + b h\\\\c e + d g & c f + d h\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡a⋅e + b⋅g  a⋅f + b⋅h⎤\n",
       "⎢                    ⎥\n",
       "⎣c⋅e + d⋅g  c⋅f + d⋅h⎦"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAKgAAAAzCAYAAAAD6kM0AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAJTklEQVR4Ae2ci3XVOBCGk5wUkGU7CB0AWwHQAbAVAB3AoQIOdACpIIQOQgc8OoAONqSD7P/panxkXfn6Jfk62HOO/JAlzcyvkTSSbR0eHBycKGzRzc3N9VbkGrEiUAiBw8PDpB0eid93hd9ReFdIjrXYFYEmBC70ILbDCwwUeqwe8zAILzfR63FFYBoEZHs1GxRXZ4NmoNNIsXJZEeiJwHHP9ItILn/ogxQ1H/ztFP6498EY5h4oXInn3dsItvS4J7nPFE4VPkmP2mjcF9u1B42sQADif99ROFd4ofBIoTjRCBQei9EvhS/FGRZiIB1+KNxX8Ux6LkM2Q7CdrYFKmUcKGMjUBE+MEyP9pjC1sdAD1SpW90WoFMaU6wWOseuN7ZyHeFpgcumhSG2pUA8sPL/Qo3EuxStV7o6KTSXPEVcKYzcSeAydnEOxnW0PmgP9AWVsATugjDFZ9s1/jOxhXnrQuHEP0q1TDyrrx+F9rUCv8p/CZ4UnaiHvda6R0tIq3yiQDsLZfx22Jhc70UHy4FOaLH/r+lyy/AjZKw1DDwAC7JXumawwUak5+IorTfD/If6vPCOwu5Qc4F2Rns8GY8kS2waGiZvyFoH1fDS2P1XOI4FwkAo8U2AB9dSe6xof6UbhxOI4i0hLefcsnmuFd3bf9aw8TxRedU0fp1NegEMWGpLTTdfMzqnwJl1RYjDPpnK7xIsvRgd/Xpw4XIO4EPs5YdzHNnphK90x7MudQ7xvqfQm9IDMLo24/qU4elRHviVhuBhj2EOhxCRO/0aS6gjPz5Il7H2eNcni5SdzPDQRNwWBE/TQcLWz4mis9EY0ullg3GIbzORj20CF3tgek2sHMTzSsj9FaQAzrHge0ztBJxKeIYrhFKLHahRMaclnleMy+AOzaColNcwCwNMwcXjty6Qy3TBjz5TnL7tOnOnp6VnDxpVIViwKFyOuWLCHDMvZYCyZ+tjGKGwbh3gJwdBeGxJ1D2h01zW3wMddqIKTw2ffeJU3eIhX3i252/grD4D/bEtX6rl4M7R/CMvXPQ0XrJ2b4q9ni7Hka7KN3tiqrN1DvHohmBHiHoVhEiNM9YqhG0CyyWmH3G2y0MpjXdvy5HwOf4w0JBslQqznjHGTbQzGdqcP6pH6GiKma0BzFSljeKLAUAo1Ake6TZLyRzWca8+FkWGLJEvKnSDdA4VY1638hSMqDCUnnQMV/jLQqXoeyzElxgHvGK8m2xiMbaOBelAwRDNAWzLg/psX8h+lM9DoxmuVD8gK+E1T90wfxZPXbTWSLMh4VYvUjeLRCYMIe6o4Wel7fPoKa10zOeVdNroYzQLjPrYxFttDaU5PQyvdqhwVTqWdKdBS3BqizvRQAEUcb1wq41N6mxzZuiOuwNZaqfK1ksqi12V5ZWh+ZIQqWXT9UeUhf408L3w78NgLeayRmfpIrn8i2Fww9vK22sZQbJUPH/TpMUo3ka9M84PCZKm4wcYYFpzrWrLzYqErMYOOVyW65s2SzmOdWrGola90gxpsrZAMNz1sYxS2RxlkLVUEPd1Wb5eLGS1bgWEUwkc6d1fLOhTBOCe2O3vQfdaVWuiWy5FZHobTXwLzhHLFb689aGbdOhVXEONs2M7WQDshPC4RIGKcnJMuy7jiF507G7aLNVD1HuHseNHWlFv5nNjO2QfNjdta3i1EYDXQW1hpSxJ5NdAl1fYt1HU10FtYaUsSeTXQJdX2LdR1NdBbWGlLEnnxBspCvcKlwm+F5BdQJQ1i3/xz6iZd7il8VwBL+7ia7wcGY7zYdVCrGK3Z8brvMcDqbF9p2ePi533zz6mgdOHDofvCko+s+TXF0RgdZ92DStEpN2/go9oKVI/tlKdJ+ZfClnI9aKlX1b11nLWBSlFeRbp35SUtpQXUkqxd2XviXwpbvl6q/VCJkkN1nLuBugqc4JAEdQK+xmLf/E2OHGd60FTvOUjHTj6orP9UTDtt3ICGSs/Hxghkkw6+FOInu728/5Y8fLxgHy67D6+9v6RoRw5UL/cdxdC7ID8fctsfAy5hoQP8WzdsgPecsJUssV1gmAzjtb9pkVs0GGOMqPaHpiqFz89c8AV32riBPCKM4XuQHyV6/2Xpy8LQB2+koLzwRr/GzRv0DGNEcCZJ4QYJfCua7Q9KwyM+i0fIv3HDBo/HnLDF4DrZRaRjJ4yVZ/dfnUrglgd0oqJaN27w6Z1B6foh95CApQeiEvYxAYFn2+YNAA0997Ju7jZH5C5Nxr9xwwYE8D0nv9TsHVvJAi5NdlH7tx/ZRaZjb4yPN/kbj/ZdX5eNGyjkTMF2hnOFShm6fKjxg2Cl2efmDQzlgFr9W+Wk3YCaGqr842wn439tJXoD4NY2bOB6NthKlr52YToOwrhxiJcgW0Oz4mg9DIk1t0D3GCLx1XDqhyVa/W+u+wbKUhg0xMNTobbpRIq/0qB/be8o3eMaoEu1x1Qqb4448cC1aNuwYfbYSoekXYCRqDfGyrN7iPetGKax1fOvNsYWz9QeEC+K09N64rQuYanDDtlrLH06jDF2P/h5jaWSWJda/kw3GB9GGpJ94W+43QZsk3YxFuOjEJWG669RPOC5ihPzcOMGl0yVWs16vXD4H7EBREXmvZUMNlzScrdIcplPlARVGWi9ziVR2hdej61yMkbEmCFXuGGDYzUHbAOdu9rFKIwbDdRXMoZID+OIytIF9/ZKMNy4wfmpSlOlVzr8Jij2YTexZY9dNm9o6t0ZOc69eHcDgy8hMQ0hxIzJR7xhw2ywHWAXozBumyQxYzyT0b3S2TZugOE7H2eVyJB/rTh7Rusi/ZXC1lsFxRUnyfNS8iAnDr2tgcL3bWBwGCIGEdN7RfyrvPjZff6vj8vpcv9ciQxPNmzAH61NKJF3TthKxs52obSjMN65s4gKH0UClSGWWX3rhgQpRsrPJGnw7iKpMv+UuD8dW+nHaP30KEeFqTA+s7pQoLU4Ik4XDF3VZ1ebJ72O10pNWCwtHdu2Ib6rYbxRQpt42AI//icL/INnwsprs9iucvyJ6RaNbS4DpZfE33ymFo8fRU+Kca4GJiBG0qKxzWKg3hBXYxxpiansS8c2iw+aAnaNWxHIgYD1oKfhBIeC1XIXPTnJAe5aRncEYvtTTtzEA1tmCheKiWej10FLQ2ReaUWgLwIyUN42VhNtn//L/13IxpkL5xOLAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}a e + c f & b e + d f\\\\a g + c h & b g + d h\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡a⋅e + c⋅f  b⋅e + d⋅f⎤\n",
       "⎢                    ⎥\n",
       "⎣a⋅g + c⋅h  b⋅g + d⋅h⎦"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = sy.Matrix([[a, b], [c, d]])\n",
    "B = sy.Matrix([[e, f], [g, h]])\n",
    "A*B\n",
    "B*A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make $\\pmb{AB} = \\pmb{BA}$, we can show $\\pmb{AB} - \\pmb{BA} = 0$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "M = A*B - B*A\n",
    "M"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\\begin{align}\n",
    "b g - c f&=0 \\\\\n",
    " a f - b e + b h - d f&=0\\\\\n",
    "- a g + c e - c h + d g&=0 \\\\\n",
    "- b g + c f&=0\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we treat $a, b, c, d$ as coefficients of the system, we and extract an augmented matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A_aug = sy.Matrix([[0, -c, b, 0], [-b, a-d, 0, b], [c, 0, d -a, -c], [0, c, -b, 0]]); A_aug"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Perform Gaussian-Jordon elimination till row reduced formed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A_aug.rref()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The general solution is \n",
    "\\begin{align}\n",
    "e - \\frac{a-d}{c}g - h &=0\\\\\n",
    "f - \\frac{b}{c} & =0\\\\\n",
    "g &= free\\\\\n",
    "h & =free\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "if we set coefficients $a = 10, b = 12, c = 20, d = 8$, or $\\pmb A = \\left[\\begin{matrix}10 & 12\\\\20 & 8\\end{matrix}\\right]$ then general solution becomes\n",
    "\n",
    "\n",
    "\\begin{align}\n",
    "e - .1g - h &=0\\\\\n",
    "f - .6 & =0\\\\\n",
    "g &= free\\\\\n",
    "h & =free\n",
    "\\end{align}\n",
    "Then try a special solution when $g = h = 1$\n",
    "\\begin{align}\n",
    "e  &=1.1\\\\\n",
    "f & =.6\\\\\n",
    "g &=1 \\\\\n",
    "h & =1\n",
    "\\end{align}\n",
    "And this is a <font face=\"gotham\" color=\"red\">commutable matrix of $A$</font>, we denote $\\pmb C$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "C = sy.Matrix([[1.1, .6], [1, 1]]);C"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can see that $\\pmb{AB}=\\pmb{BA}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = sy.Matrix([[10, 12], [20, 8]])\n",
    "A*C\n",
    "C*A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Transpose of Matrices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Matrix $A_{n\\times m}$ and its transpose is \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.array([[1, 2, 3], [4, 5, 6]]); A\n",
    "A.T # transpose"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = sy.Matrix([[1, 2, 3], [4, 5, 6]]); A\n",
    "A.transpose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The properties of transpose are"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. $(A^T)^T$\n",
    "2. $(A+B)^T=A^T+B^T$\n",
    "3. $(cA)^T=cA^T$\n",
    "4. $(AB)^T=B^TA^T$\n",
    "\n",
    "We can show why this holds with SymPy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = sy.Matrix([[a, b], [c, d], [e, f]])\n",
    "B = sy.Matrix([[g, h, i], [j, k, l]])\n",
    "AB = A*B\n",
    "AB_tr = AB.transpose(); AB_tr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A_tr_B_tr = B.transpose()*A.transpose()\n",
    "A_tr_B_tr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "AB_tr - A_tr_B_tr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Identity and Inverse Matrices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\"> Identity Matrices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Identity matrix properties:\n",
    "$$\n",
    "AI=IA = A\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's generate $\\pmb I$ and $\\pmb A$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "I = np.eye(5); I"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.around(np.random.rand(5, 5)*100); A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A@I"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "I@A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\"> Elementary Matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An elementary matrix is a matrix that can be obtained from a single elementary row operation on an identity matrix. Such as:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\left[\\begin{matrix}1 & 0 & 0\\cr 0 & 1 & 0\\cr 0 & 0 & 1\\end{matrix}\\right]\\ \\matrix{R_1\\leftrightarrow R_2\\cr ~\\cr ~}\\qquad\\Longrightarrow\\qquad \\left[\\begin{matrix}0 & 1 & 0\\cr 1 & 0 & 0\\cr 0 & 0 & 1\\end{matrix}\\right]\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The elementary matrix above is created by switching row 1 and row 2, and we denote it as $\\pmb{E}$, let's left multiply $\\pmb  E$ onto a matrix $\\pmb  A$. Generate $\\pmb A$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = sy.randMatrix(3, percent = 80); A # generate a random matrix with 80% of entries being nonzero"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "E = sy.Matrix([[0, 1, 0], [1, 0, 0], [0, 0, 1]]);E"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It turns out that by multiplying $\\pmb E$ onto $\\pmb A$, $\\pmb A$ also switches the row 1 and 2. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "E*A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding a multiple of a row onto another row in the identity matrix also gives us an elementary matrix.\n",
    "\n",
    "$$\n",
    "\\left[\\begin{matrix}1 & 0 & 0\\cr 0 & 1 & 0\\cr 0 & 0 & 1\\end{matrix}\\right]\\ \\matrix{~\\cr ~\\cr R_3-7R_1}\\qquad\\longrightarrow\\left[\\begin{matrix}1 & 0 & 0\\cr 0 & 1 & 0\\cr -7 & 0 & 1\\end{matrix}\\right]\n",
    "$$\n",
    "\n",
    "Let's verify with SymPy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "A = sy.randMatrix(3, percent = 80); A\n",
    "E = sy.Matrix([[1, 0, 0], [0, 1, 0], [-7, 0, 1]]); E"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "E*A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also show this by explicit row operation on $\\pmb A$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "EA = sy.matrices.MatrixBase.copy(A)\n",
    "EA[2,:]=-7*EA[0,:]+EA[2,:]\n",
    "EA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will see an importnat conclusion of elementary matrices multiplication is that an invertible matrix is a product of a series of elementary matrices."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\"> Inverse Matrices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If $\\pmb{AB}=\\pmb{BA}=\\mathbf{I}$, $\\pmb B$ is called the inverse of matrix $\\pmb  A$, denoted as $\\pmb B= \\pmb A^{-1}$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy has convenient function ```np.linalg.inv()``` for computing inverse matrices. Generate $\\pmb A$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.round(10*np.random.randn(5,5)); A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "Ainv = np.linalg.inv(A)\n",
    "Ainv\n",
    "A@Ainv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ```-0.``` means there are more digits after point, but omitted here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font face=\"gotham\" color=\"purple\"> $[A\\,|\\,I]\\sim [I\\,|\\,A^{-1}]$ Algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A convenient way of calculating inverse is that we can construct an augmented matrix $[\\pmb A\\,|\\,\\mathbf{I}]$, then multiply a series of $\\pmb E$'s which are elementary row operations till the augmented matrix is row reduced form, i.e. $\\pmb A \\rightarrow \\mathbf{I}$. Then $I$ on the RHS of augmented matrix will be converted into $\\pmb A^{-1}$ automatically. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can show with SymPy's ```.rref()``` function on the augmented matrix $[A\\,|\\,I]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "AI = np.hstack((A, I)) # stack the matrix A and I horizontally\n",
    "AI = sy.Matrix(AI); AI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "AI_rref = AI.rref(); AI_rref"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extract the RHS block, this is the $A^{-1}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Ainv = AI_rref[0][:,5:];Ainv # extract the RHS block"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I wrote a function to round the float numbers to the $4$th digits, but this is not absolutely neccessary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "round_expr(Ainv, 4) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can verify if $AA^{-1}=\\mathbf{I}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = sy.Matrix(A)\n",
    "M = A*Ainv\n",
    "round_expr(M, 4) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We got $\\mathbf{I}$, which means the RHS block is indeed $A^{-1}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font face=\"gotham\" color=\"purple\"> An Example of Existence of Inverse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Determine the values of $\\lambda$ such that the matrix\n",
    "$$A=\\left[ \\begin{matrix}3 &\\lambda &1\\cr 2 & -1 & 6\\cr 1 & 9 & 4\\end{matrix}\\right]$$\n",
    "is not invertible."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Still,we are using SymPy to solve the problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lamb = sy.symbols('lamda') # SymPy will automatically render into LaTeX greek letters\n",
    "A = np.array([[3, lamb, 1], [2, -1, 6], [1, 9, 4]])\n",
    "I = np.eye(3)\n",
    "AI = np.hstack((A, I))\n",
    "AI = sy.Matrix(AI)\n",
    "AI_rref = AI.rref()\n",
    "AI_rref"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make the matrix $A$ invertible we notice that are one conditions to be satisfied (in every denominators):\n",
    "\\begin{align}\n",
    "-6\\lambda -465 &\\neq0\\\\\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Solve for $\\lambda$'s."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sy.solvers.solve(-6*lamb-465, lamb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's test with determinant. If $|\\pmb A|=0$, then the matrix is not invertible. Don't worry, we will come back to this. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.array([[3, -155/2, 1], [2, -1, 6], [1, 9, 4]])\n",
    "np.linalg.det(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The $|\\pmb A|$ is practically $0$. The condition is that as long as $\\lambda \\neq -\\frac{155}{2}$, the matrix $A$ is invertible."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font face=\"gotham\" color=\"purple\"> Properties of Inverse Matrices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. If $A$ and $B$ are both invertible, then $(AB)^{-1}=B^{-1}A^{-1}$.\n",
    "2. If $A$ is invertible, then $(A^T)^{-1}=(A^{-1})^T$.\n",
    "3. If $A$ and $B$ are both invertible and symmetric such that $AB=BA$, then $A^{-1}B$ is symmetric."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The <font face=\"gotham\" color=\"red\"> first property</font> is straightforward\n",
    "\\begin{align}\n",
    "ABB^{-1}A^{-1}=AIA^{-1}=I=AB(AB)^{-1}\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The <font face=\"gotham\" color=\"red\"> second property</font> is to show\n",
    "$$\n",
    "A^T(A^{-1})^T = I\n",
    "$$\n",
    "We can use the property of transpose\n",
    "$$\n",
    "A^T(A^{-1})^T=(A^{-1}A)^T = I^T = I\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The <font face=\"gotham\" color=\"red\">third property</font> is to show\n",
    "$$\n",
    "A^{-1}B = (A^{-1}B)^T\n",
    "$$\n",
    "Again use the property of tranpose\n",
    "$$\n",
    "(A^{-1}B)^{T}=B^T(A^{-1})^T=B(A^T)^{-1}=BA^{-1}\n",
    "$$\n",
    "We use the $AB = BA$ condition to continue\n",
    "\\begin{align}\n",
    "AB&=BA\\\\\n",
    "A^{-1}ABA^{-1}&=A^{-1}BAA^{-1}\\\\\n",
    "BA^{-1}&=A^{-1}B\n",
    "\\end{align}\n",
    "The plug in the previous equation, we have\n",
    "$$\n",
    "(A^{-1}B)^{T}=BA^{-1}=A^{-1}B\n",
    "$$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}