05_Lecture_I_02_Overview_of_ideas_to_come.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ This notebook is part of the addition lecture *An overview of key ideas* in the OCW MIT course 18.06 by Prof Gilbert Strang [1]\n",
    "+ Created by me, Dr Juan H Klopper\n",
    "    + Specialist Surgeon\n",
    "    + Groote Schuur Hospital\n",
    "    + Senior Lecturer in Surgery and Biostatistics & Machine Learning\n",
    "    + University Cape Town\n",
    "    + <a href=\"mailto:juan.klopper@uct.ac.za\">Email me with your thoughts, comments, suggestions and corrections</a> \n",
    "<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"><img alt=\"Creative Commons Licence\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" /></a><br /><span xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/InteractiveResource\" property=\"dct:title\" rel=\"dct:type\">Linear Algebra OCW MIT18.06</span> <span xmlns:cc=\"http://creativecommons.org/ns#\" property=\"cc:attributionName\">IPython notebook [2] study notes by Dr Juan H Klopper</span> is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\">Creative Commons Attribution-NonCommercial 4.0 International License</a>.\n",
    "\n",
    "+ [1] <a href=\"http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/index.htm\">OCW MIT 18.06</a>\n",
    "+ [2] Fernando Pérez, Brian E. Granger, IPython: A System for Interactive Scientific Computing, Computing in Science and Engineering, vol. 9, no. 3, pp. 21-29, May/June 2007, doi:10.1109/MCSE.2007.53. URL: http://ipython.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>\n",
       "\n",
       "<style>\n",
       "\n",
       "@font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "/* Formatting for header cells */\n",
       ".text_cell_render h1 {\n",
       "    font-family: 'Philosopher', sans-serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 2.2em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0, 80, 120);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       ".text_cell_render h2 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 1.9em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(245,179,64);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       "\n",
       ".text_cell_render h3 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    margin-top:12px;\n",
       "    margin-bottom: 3px;\n",
       "    font-style: italic;\n",
       "    color: rgb(94,127,192);\n",
       "}\n",
       "\n",
       ".text_cell_render h4 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {\n",
       "    font-family: 'Alegreya Sans', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 16pt;\n",
       "    color: grey;\n",
       "    font-style: italic;\n",
       "    margin-bottom: .1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h6 {\n",
       "    font-family: 'PT Mono', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 10pt;\n",
       "    color: grey;\n",
       "    margin-bottom: 1px;\n",
       "    margin-top: 1px;\n",
       "}\n",
       "\n",
       ".CodeMirror{\n",
       "        font-family: \"PT Mono\";\n",
       "        font-size: 100%;\n",
       "}\n",
       "\n",
       "</style>\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML, Image\n",
    "css_file = 'style.css'\n",
    "HTML(open(css_file, 'r').read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![SegmentLocal](KRG_elegant_logo_for_light_BG.png \"segment\")\n",
    "![SegmentLocal](UCT_logo.gif \"segment\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Imports\n",
    "from sympy import init_printing, Matrix, symbols, sqrt, Rational, acos\n",
    "from numpy import matrix, transpose, sqrt\n",
    "from numpy.linalg import pinv, inv, det, svd, norm\n",
    "from scipy.linalg import pinv2\n",
    "init_printing()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# An overview of key ideas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Moving from vectors to matrices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider two position vectors $\\mathbf{u} , \\mathbf{v} \\in \\mathbb{R}^{3}$.  They can be expressed as column vectors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\mathbf{u}=\\begin{bmatrix}1\\\\-1\\\\0\\end{bmatrix} \\\\ \\mathbf{v}=\\begin{bmatrix}0\\\\1\\\\-1\\end{bmatrix} \\tag{1} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can add constant scalar multiples of these vectors, with the scalars $x_1$ and $x_2$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {x}_{1}\\mathbf{u}+{x}_{2}\\mathbf{v}=\\mathbf{b} \\tag{2} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is simple vector addition.  It's easy to visualize that if we combine all possible combinations, that we start filling a plane through the origin.  Adding a third vector that is not in this plane will, using all possible linear combinations, extend to fill all of three-dimensional space."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\mathbf{w}=\\begin{bmatrix}0\\\\0\\\\1\\end{bmatrix} \\tag{3} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have (4) below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {x}_{1}\\mathbf{u}+{x}_{2}\\mathbf{v}+{x}_{3}\\mathbf{w}={b} \\tag{4} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can write the coefficients of the vectors as a vector of unknowns, $\\mathbf{x}$, shown in (5)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\mathbf{x} = \\begin{bmatrix} x_1 \\\\ x_2 \\\\ x_3 \\end{bmatrix}\\tag{5}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now write (4) is the form $A \\mathbf{x} = \\mathbf{b}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\begin{bmatrix} 1 & 0 & 0 \\\\ -1 & 1 & 0 \\\\ 0 & -1 & 1 \\end{bmatrix}\\begin{bmatrix} { x }_{ 1 } \\\\ { x }_{ 2 } \\\\ { x }_{ 3 } \\end{bmatrix}=\\begin{bmatrix} { x }_{ 1 } \\\\ { x }_{ 2 }-{ x }_{ 1 } \\\\ { x }_{ 3 }-{ x }_{ 2 } \\end{bmatrix} \\tag{6} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create matrix $A$ in code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHkAAABLCAYAAABUSCjvAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAE2UlEQVR4Ae2c8W3VMBDG+xB/owokBnhs0MIEtBtQNgA2ADEBemxQmADxRoAJqnaDMgASVcUC5fuKT88K95K0PuKrc5asJOdX+/P9YidxLl3s7OzsIZ8ia2l9dXV1pBWEzY8HFovFOdQsNUXgt7ifFXzEPn+cpx/5Qey79cBKUXYI2wvac8jHoB5QFW95N4Hbp65GjG6a/oHc/d2tjlE5p/+vyPto/PJWlVT4I+iW0fALzT9BXnk56Uu15SP51q6FiF388WfkC+SnyOr1AXaXCfp5T/IBUNcUmPpziu1hbdAW2u5ZeJ0jFvkI+Q3q+2JR51R1wImv0dYutF8DZrvsDzY8PuZxrWSlzQRyLScYtcunhzOlrhPYDuBozlK1kom2gAyQIMjLTDfJ/QTLayUTbbOGPHKUPqxB2FLbrCEDngCUUZvzlNFda7o20zZ3yDnUbfuPthU4sI/SNnfIMlo1XjKS+NxcI5lpmzXk9KhEgNqULLYqq4CW2mYNOQ3Pb9gu036+kZHM8lrJRFtA/rsEy1W6btqH4SwbUd3yKY65PFysbfaQAZGL+xd4ZLlezCe59PjyEruveFwrWWkzWbumE+AYnnW8jsmZx7XfMxyfQCxfY3pOHLXvoVem7Wc4fg7d1F87FWszgwyH3NngAmi/BMl3tWlq7Vtom/10rTm2NVtAbo2o0p+ArDilNVNAbo2o0p+ArDilNVNAbo2o0p+ArDilNVNAbo2o0p+ArDilNVNAbo2o0p/JIWN9eA/5HFne1yqy6pi8aoOuVcpvsT1GXt7EQ2Zr132NQhSBugy+96yNPoW+4sD/SUYyF9n5AgPZXfC9Z20AbBL4PwnkvlEeZb0eiOD6Xve0URjB9W1w1HuR7hX0wo1V4tA2FmUvpmvFKU5MApABDd0k4bqjnlACctd9d+t4VHD9qEeoNHV8R/9HnTnJT7yb9hAjdbewbdTKaN1YNnsyykcF/hPyg/S3st1Ulfb4mIFdBpRFmsgD9DkGF1vTBpbY+gL/H4tUTte/04FspSy29T1QElz/U+THNVk84XMbwfU+udipwpRtEvg/6sbLQjauL26D7z1rg+/9BNcPnQg4K90G3zvXxpveosD/uCYPnZ0NlAfkBiAOdSEgD3mogfKA3ADEoS4E5CEPNVAekBuAONSFgDzkoQbKA3IDEIe6EJCHPNRAeUBuAOJQF0whYw24KAh8SGztcvSv2ocBJW2bvaCAiOIg8NoQtfbRL76gr/JhgFXbJiMZYkyCwDUn17YxQoMvMJAn/zDAqm0TyABhEgReG2ir7VtBPoCDtMAzviZjYnmkSh4ohpyuG0PyJbpw6HdR/h88UAwZmgSgjNpcpoxu3rxEquQBC8hjpI8KAh9TUfzm5h6weISS0aq1LqN8VBC4VkGpLV1OZv1hQDFk3ubDkWShTcli6wsCL+XY+/fUhx/M+sMAq+m6JAi8F1IUlnvACrJJEHh5d6IGzQMmkDElmgSBawLDVu6B4mtyJqE4CDyry9Uu7jmqfRhg0bYZ5HSDUxQE7opsJgZ9q/ZhgEXbJtN15o/YdeiBgOwQirWkgGztUYf1BWSHUKwlBWRrjzqsL7+75j817UpcW9zddSuNY1sPgNs5alxuq5WQua7M0BYtVVtz1sSEbasHVltLUPAHZRyhkwne8qsAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1 & 0 & 0\\\\-1 & 1 & 0\\\\0 & -1 & 1\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1   0   0⎤\n",
       "⎢         ⎥\n",
       "⎢-1  1   0⎥\n",
       "⎢         ⎥\n",
       "⎣0   -1  1⎦"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = Matrix([[1, 0, 0], [-1, 1, 0], [0, -1, 1]]) # Creating a matrix and putting\n",
    "# it into a computer variable called C\n",
    "A # Displaying it to the screen"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the column-view of matrix-vector multiplication as opposed to the row view.  Matrices are seen as column, representing vectors.  Each element of the column vector $\\mathbf{x}$ is a scalar multiple of the corresponding column in the matrix $A$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ { x }_{ 1 }\\begin{bmatrix} 1 \\\\ -1 \\\\ 0 \\end{bmatrix}+{ x }_{ 2 }\\begin{bmatrix} 0 \\\\ 1 \\\\ -1 \\end{bmatrix}+{ x }_{ 3 }\\begin{bmatrix} 0 \\\\ 0 \\\\ 1 \\end{bmatrix}=\\begin{matrix} { x }_{ 1 } \\\\ -{ x }_{ 1 }+{ x }_{ 2 } \\\\ -{ x }_{ 2 }+{ x }_{ 3 } \\end{matrix} = {x}_{1}\\mathbf{u}+{x}_{2}\\mathbf{v}+{x}_{3}\\mathbf{w} \\tag{7}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we consider the solution vector, $\\mathbf{b}$, then we can write (8)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\begin{bmatrix} { x }_{ 1 } \\\\ { x }_{ 2 }-{ x }_{ 1 } \\\\ { x }_{ 3 }-{ x }_{ 2 } \\end{bmatrix} = \\begin{bmatrix}{b}_{1}\\\\{b}_{2}\\\\{b}_{3}\\end{bmatrix} \\tag{8} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By substitution this can be _converted_ to (9)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\begin{bmatrix} { x }_{ 1 } \\\\ { x }_{ 2 } \\\\ { x }_{ 3 } \\end{bmatrix}=\\begin{bmatrix} { b }_{ 1 } \\\\ { b }_{ 1 }+{ b }_{ 2 } \\\\ { b }_{ 1 }+{ b }_{ 2 }+{ b }_{ 2 } \\end{bmatrix} \\tag{9} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is, in fact, a contrived example.  Looking closely, we can write the right-hand side of (9) as a matrix times $\\mathbf{b}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\begin{bmatrix}1&0&0\\\\1&1&0\\\\1&1&1\\end{bmatrix}\\begin{bmatrix}{b}_{1}\\\\{b}_{2}\\\\{b}_{3}\\end{bmatrix} \\tag{10} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "This matrix is the inverse of $A$ such that $\\mathbf{x} = A^{-1} \\mathbf{b}$.  We will learn much more about matrix inverses and whether matrices are invertible.  The `sympy` library has a method called `.inv()` that will calculate the inverse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAAgAAAAOCAYAAAASVl2WAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAWUlEQVQYGWP4//8/AwwzMDAYAfFdIBaAibEwMjIKAAVmA/E7IDYBYiUgRgCYShANBGVADGLATWACieIDowogoUMwHBhBQQyMj9VA9aA4AcUFiL4HxOeA+DQAT1cpCApe64MAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$\\displaystyle 1$"
      ],
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.det()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAFgAAABLCAYAAAD010ABAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAEEElEQVR4Ae2dYW7TQBCFHcRvVIHEAcINUjgB7Q1abgDcoIgToHCDwAkQvQHcoGpuEA6ARBVxgfBe8FDLjOPJ2hPb8ay0snc8O7P7eVg220k7ybJshnqLqpXrzWZzqT0I2V8Ck8lkhbupxgPsJg8LDz7insrF8qPYiHuVwFyRnkN2QXkR8ALEA6hCa5cIzD6VnyOqKfoPcFlPbaMzl5SvqKcwvlaVHITwK5HyC+afoc4PFRBNfRPYBnWKAWdaxbMTVEJdoHK9pv6Jpushy31eiG36Rt2ufSLzuqb4Rp83qBjSJnuAm9oCxTXqJepbKH+p7dCiAqKHg+XLvBazHA/u2eYLdytt+DYBdpuBzTB3MUtF9QayM0BgNHuVxr6HAPgM9O4UgoxiFj73Ko199xqwMTofe9Bty3evAQOcwJNoLbKUqPZaIlrx3XfARaBV90+qHhxAXuu774AlSjVWEmHcF3uUVnz3GnC+HSM8bRkQmcunz7Z89xpwHpbfcZ3m98WLRDCfe5XGvocAmJ8gnysETyFbFiJNUWksauy794ABkIcpd9g2bQ9PiCzfQr3C7Wu2vUobvounaTvHiUnxbXLdk2i6hWyJ9g0GwqNOz8JofQ9/slS8QPsl/NK/d2nk2wwYk+ns4B2+16D4zpukZr+p794vEdqkhyQLwM5vKwAHYGcCzuYjggOwMwFn8xHBAdiZgLP5iOAA7EzA2fzeEYzzgBnqClXOY52HeG++K9/wO8/rFa4L1On9qHbfmc4iYJAwP6PylJ+HPWYH0G1UuvTNgcM/E20+4Exim5eRj4cHXeeQmQ77ZzCwM7MHhv5l/ED3Ktc/WGaP+D+0b/hj0stK/MsVMqZxfZN2+Zr3g9iY2YMOYy2jSDzp8uUed+JJl2TztbZuCPJzwUq9vXcRlZaO74HA42F/uciP9Gt3UgG4jG6/9uATT/abbrvaEqWaVYnu2qQXRvCj3IJcNYOjk2GLJUuDtgyIrGof/FSAEfDvvCFXeRbXLEtNPPkp8GINFhL69fgTT/R5H0aKZaJx0ovpLILTwb6ws8STLn1j6qNIPBls0kuswc6rTQAOwM4EnM1HBAdgZwLO5iOCA7AzAWfzEcEB2JmAs/mI4L4BxrnA6BJP+A5S52067IFxHjCPLvGkrXnPAC8STwrJNUwYKVcwMifcQHe/XymDDlESCcR/congrN0CsJVUol4ATgRn7RaAraQS9QJwIjhrtwBsJZWoF4ATwVm7BWArqUS9AJwIztrNdBZBY/hcPsrEk6bzNgPGZ/Mukz8G6zuWCOu/9US9AJwIztotAFtJJeoF4ERw1m4B2EoqUa+4i+AXvMtm4g+VlImU2mBW+YdKqErA/CIHf/m9Vqq+5KHpjlUmf35Cnf8f74yvhKD3hL8AAAAASUVORK5CYII=\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1 & 0 & 0\\\\1 & 1 & 0\\\\1 & 1 & 1\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1  0  0⎤\n",
       "⎢       ⎥\n",
       "⎢1  1  0⎥\n",
       "⎢       ⎥\n",
       "⎣1  1  1⎦"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.inv()  # Inverse of matrix A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Something else we will come across is _lower triangular form_.  Notice that all the values above the _main diagonal_ (values from the top-left to the bottom-right) as zero value.  This matrix is _lower triangular_."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's replace $\\mathbf{w}$ and create a matrix $M$.  Unlike our original $\\mathbf{w}$, this one is _special_ (for our purposes)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "x1, x2, x3, b1, b2, b3 = symbols('x1, x2, x3, b1, b2, b3') # Creating algebraic symbols\n",
    "# This reserves these symbols so as not to see them as computer variable names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAIkAAABLCAYAAABEBKR2AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAFCklEQVR4Ae1c4W3VMBDOQ/xGFUgM0BEKTEC7AWUDYAMQE6CyQWECREeACap2gzIAEhVigfJ9Tz7JCs6LeT47jn2WrCS2n3333ZezXee6GYbhCPkKOZQu7u7uTkMVVtYOApvN5gbaHIY0gv03972Kj7hnYz/98B/svlkEzgKanaDsBct9kpyDNUaKAFqtF8Hun8Y6wruwaEuSe+PK1Gd0fkT3hXyQ2lfJ30PeM5ff4nqOHHS/JWXSHAv67G0X35PsLRMEICE+I98iP0VeFcCQn2uyD3ijLnAdnD5XuJ6s2btq2UXFkwDI38inyG+A8RcCvZYEIF9D1gPIviUI5aY+uPD5nM9rTdQDOdkuKiRZK4hObu7ergM6XKLs2L2Ngep+iowkIALMzWlynOhNmFjfdeqaJJFe4mHXDIHyXZME+gsBxGv4fBDvwkV516l3ksQY/1FMo5bbqGyBVwyQeIuQCuJlfoUqS5S56fA7xvofb8bdTGghvrfIXZOEW0QYguCFjCBli/0VmvJBtid7W1fphzbdDMM3YHkYwFM8Ceu7TkaSYfgKBvCvxOPEN/javc3juq6euycJSMDDrVtMO9vDLFrfrQVe4vYVn3tPamsSAMs3kvO4vJU8++AC6hKG4GcINSd6jfeQV6adZ3h+DrlVF4BLAKBhFzWSANDVfpwE2blAfLeEEXOPqWGX7qeb3EZqoX8jSQtWzKyDkSQzwC10byRpwYqZdTCSZAa4he6NJC1YMbMORpLMALfQvZGkBStm1sFIkhngFro3krRgxcw6FCcJzhL2DhLKjAUP9qqVbZfukDtrYJna2c2MEjz4qzJ4CwBXK9suTKUO8mcPLCviSXiAxoMm5OqCt2qWTYgwdQVBigSWFSHJlJJWnoxAkcAyI0mynRbt4Bijhz7m5qcPTKxPTkaSZAiX6cCtpeYGl+9059rtrDeS7ISn6kohgHgNX1jxLlyUJycjSTKEVXegElgWtQV2rm3xIKGqzVFeOPEWoZHFy6gElpEkD9wocv1nUG4TUbh4kNA/gnVcQJvg5SUCoSlFylICyx4LvJxu/rgHuUqdXetHIGdg2U9R39YkgsQ6r0UCy4wk6yTHVmpMOUUCy6IWrho4Yv6sNnirZtkisM8eWFaMJGB9tcFbNcs2RxIuYNEma2CZTTdzVrD67v8dllEgAgHzJBEg9d7ESNI7AyL0N5JEgNR7EyNJ7wyI0N9IEgFS702MJL0zIEJ/I0kESL03MZL0zoAI/VVJgjOQrEFCEfpkbQL9qg3eyimb2tkNhMweJJSVAROdQy9+wNN1YJmKJwGQRYKEJuyYtZgHaDwARO42sEyFJLBSkSChrGywzicR0CLJMUYIfZjLY2wm1ltaKQLJJHFz9pz68vX2XDurrxCBZJJAJyGAeA1fTfEuXPxZWikCGiSJUV0lSChmIGujj4DGFli8RUg68TIqQUKhAebK3HRogWVzQO2oTyYJt4gwBIcITSlSlhIktEP8+SrKh1YWWDYP1WQLrekmZ5DQpPBWUQYBLZIUCRIqA4mNMkZAhSRw6UWChMbC23MZBJLXJJ6Y2YOEvLGK3mLN1XVgmRpJ3AIxa5BQUWZ4g0G3rgPLVKYbD0+7bRABI0mDRtVWyUiijWiD/RlJGjSqtkpGEm1EG+zP393cuD+v+2pe1Lyy9wW1+/0RgN1v8OvDqR5IEp6r8NO8UFrszCUkjJVlQ+BsV89/Ad0Nfh8wLoHRAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1 & 0 & -1\\\\-1 & 1 & 0\\\\0 & -1 & 1\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1   0   -1⎤\n",
       "⎢          ⎥\n",
       "⎢-1  1   0 ⎥\n",
       "⎢          ⎥\n",
       "⎣0   -1  1 ⎦"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "M = Matrix([[1, 0, -1], [-1, 1, 0], [0, -1, 1]]) # Creating a matrix and putting\n",
    "# it into a computer variable called M\n",
    "M # Displaying it to the screen"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And a vector of unknowns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACUAAABLCAYAAAD3YE4pAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAEEklEQVRoBe2a600cMRDHl4gC8ujgKohA+R4p0EEeFQAdECHlOyIdQAkhHUAHPDqAAiIFojRw+f/uPJbP8a5nj73opOxIc2N7xp7ZGdvr8V7TNM2WcNqC59PptBkapeuuRZ9UTZtNMQ2+qoBwCvdpZcDySWGsXbW9t3bz1GRoj/QZT8bsC2eeemaWrRMdjfJGY/SU11PpltDZZ2Nj47kEWCHAG+GecCL8JASutHS+z4tP/3VtCVJzaktc5UPhOZQ2wanwzvjLUPXvtyXIS3TINzw2OvPMywJfTcuBN3zXevp0d2f3vbU20Q+5+hDuj2o/EH8753fVXUZp0NtskB3VP2dtsSqDmBLIAMzFXuAyKh1RCk2Zha4JXuEd8QvZ8BC3ao/vsnSMWrm6T6FQeCE0Yw6C4jScR2ZQTaGHXzVKg2AM+ICBUOHMI6J4Cd4V5aFg0zHQpWTOhCgnNAfBc2wDN8IHtcVQIvNUqBoVwjILmSkLRgxqiI0N9YQvlf8n5bU0qhq+ZVyjOTdRP0LOPJyobq8hjtxVWIlRmnNsF62ba82qtQzfaFQtbMYfPWWeqNHRUzUPGX/0lHmiRkdP1TxkfPe7Ty9VTp3/bzLq8pS81JaM2klgIRkNXj0K4ZgEuhdOsRalTlpN29V7SwNyPp+h6hfCG6vnVLyY4sOjLuxM68WPaTsd3grp+TofvK0e5Gf3CCWZwN8xnup4Cx0LD2d8qOALMpTZEn7TIjA6r7X8KjScJoGYOBCuELI5Z37qvLaKk/4wOYzqhKCwVzKqpz0TxtxQCjga36stT/+LuqtGqReeAZdKRvVQzFnSd/clh2f1LZ2MyiDmEldI25nn1NQN1dWnAePK85alEoPiFwuV0dN6Vy9eXH2e8Em+HwQPsQ0cE74QQuYV9xBV8ISvOkhBgDsGXkvQCPLyQvofGVlhJUZJ+YtMT6/qSsLXy4KC8GhUwSnFptFTRbcUGkdPFZxSbBo9VXRLoXH0VMEpxaa19JT7hazjx5iMFuOaNspLfZPRifpzLid5wMN8cz7WkcaVOEg2/gGn66i6kK+pTy0Z5fvyiR2dVeab86PVS1T8eBx2zanCE+5oEEvZVfwLjrOWV6qn3wcz9mLVZVTaRaHEIGAhGaVBxhMuaB4mQpl/GEe0CNUtgVUn7JWMmib1OxRyTudugW+GLqgapVHwDNg7GZUhfCB6J9zFQFEXeMK3dDKKBYRUBjH/bkQvC6EtGjpoMioNbAGPwrhiVWaL4GplX0YVE1t4yMD3hE+yftCgTHaSznS1EX7avwmr4AlfdZCCAP/oOFK4fgYem6f7PmElRslbbAn5thDsq5PBw1dXWZdYS6Mw21YfqyPHeJXDqhgKpaf651NWSdttSLqCJDYYdL5y/gAImKN17wr4lAAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}x_{1}\\\\x_{2}\\\\x_{3}\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡x₁⎤\n",
       "⎢  ⎥\n",
       "⎢x₂⎥\n",
       "⎢  ⎥\n",
       "⎣x₃⎦"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_vect = Matrix([[x1], [x2], [x3]]) # Giving this columns vector a computer\n",
    "# variable name\n",
    "x_vect"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, $M \\mathbf{x}$ can be calculated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGQAAABLCAYAAACGGCK3AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAG40lEQVR4Ae2c620cNxSFV4EKcJwONg0EdvI/QOQO7KQCWx04EJD/gtWBnQ4sdWCnAlvqQC4gQGwhDWzONyEXXO7MDh/DWcpLAhTf5OE95CVn5mqPFovFI/lr+T53tVqtnvUVtLw0CRwdHd2q5bKvtWR9dOwUXChOZdd9chMtPokEXvX08kR5T8l3CXkthhoBPdKaMksyfuP3p11DVkfIN35hS+9XAo2Q/cp/a/RGyJZI9pvRCNmv/LdGb4RsiWS/Ge4tKwuJbgoP1MEL08lPCp/LL+V/M3kfdMO4MvFqghpx82C4kl9KYItUr/Zcm7v2ir+UvyQkT+61/K0trykEm8Wj+F5wa1wWsmCsFpOoLK0yOvQfeLhX2x3xsKdcWft1NeKeSmV9FLvuQyVPnjc2T+HW6xejKn5VvVOVPw6hxrT5S3VRj6Humfq/GagchVvjo4JZaHfyYEA1n+/oX8VxbhJCegCdCMbvQ1A0MdQkdXDBwtU4CCKIvK7nkT+xuNUdWuCT2vGaaaF5oOJYIN+SnsJNQogLRCCtoK26AngndCNQlCUr9kb53esCt/2+4iG4he3cw/ed0q5m8Irjk9lnCMKWf+dM6BQYEroL9MySEQ+xTIsU3Cwks5gsKBYUF5bJXDYhQsKOwH9mkoTyqJbOGaI+2HRFYTJuzemlPJ8suKFtvSzMmeNxTmPT9r1CQDFBdsapwLJjWDmA/qy8tfqiTiUuGbfmc6H5MedLhcy5O1OmmtckzyECBbAorwmw5at8PgmZi7Bb2T0KqT9UR/1M+xyiDr96p53AWflFHhKss6r5R5uRG05xhuRiuBfttboRPueje1lBTZP/Vn4SN8UZEg1Eq4wHLG5jTGiptH21MqkujgY23oAH3DPh/cdU5cHwsSFrvHVAjb0QogmwygYfHANw76WKcHfPTyUHbyqrpHQT+m6EJAitZJNGSEnpJvTdCEkQWskmjZCS0k3ouxGSILSSTRohJaWb0HcjJEFoJZscBCF6sj6RtxYxJeWZ3fdBECIpPTA+W2ClOzgUQkrLcbL+Z3uXJZXBKrVqg5dy98KQDknPiX3OHfJKL+cu8Jojn3T/lD9RmpeMkOXbdSmrGjcb9ll2iDlQfYHztdC+8X2o+Ea5WZVRdlsl6IvFbnCfGSxLEz7XwuO7yagLIsQMkmOgFmuQxle5E4Oe3RPkhJPvKrad2wbCUT2dRYxboDiWJHznGHJR2NUJu2k9jsGEbcH3QwP4+T8rAwPcH9RR1Dfx1PpmvM7ud1cfqjfJN3fTz+h4u7DYsjHsphxV3MlSaXYJ8h387q6yP6hDG86Qf+VxNvw/VeivVoxdwWtLFOXxvTp4JxSCNtptIHZ2x8fRzjYr/G2TxQ91I+x7Z0iHgFKwa5W/kXfPCwjC/JSvjaOuOCFCwI7A3zdDOoSXhV2EchaidoPtkY8ZtbBLNkgrjCuk+2TsIoOzg5tjlBFEcULM9l3fOpCC8jg/1mcIeTW6VOyWDLXn3zJQfeyUO6VdE6LeKc+hsnoHnjkTne7q9WLDGzK4fp9DhCGDBYlN16grvkNGEfRUMJNiEujwbLstrUxUz1yOZw5ujIRrJwwbWmJd4EWqJMRsbfsU70GuOynsWf+8cygqq24WHXSNEEcYNUQbITWw4GBohDjCqCHaCKmBBQdDI8QRRg3RRkgNLDgYGiGOMGqINkJqYMHBcBCE6FVMM5RzSK8hyrslfPXuIHZI9Sw4AGd7uSi1wQpthnKO8Puic+6Q2YzN+iaamTcb9ll2iHYHO2PDEE7pQUM5s5uSjc0yhb/RPAH70sztTiFaAbPZ4B85CyLECGg2QzlNIMnYTDirMJQT/qwfOeN7b/aPYOrDTLCRnRlv0HDNlEcZm+0aX/2xGwfH29XWLwvAjjzXRnGKoxmu/X7ctMrRIMpabfwYv/LKO63iEzPK2sjB7EAQsc1xfO6MNTbrGpb8E4Jdc/Dtr1gMvroehFn8UEfY8lGGcppUlrHZ4GwjC1Kw2yHUNulHzooTIoDsCHySoZwmFm1sZoUyQZiMXYuKf7v4Rf4J5IRiOQ6tmFFvdmOzDKx+02TsdIQKFhkYa1wrfN+jzvzxuvTsh7qAjV4AhIzr46Wtqzg4k359W+0mO9Qtnr5Q43DN/SLvHurMg0vTi7425FFGHeJzqCwGjHJaTUwi2disZzAuC/bC0FM8TZYEyhgYxLkWiqg98t/Kj7o5VNYoiJ4KWcZmfn8S1JyGcvzzT/KPnFVJiASYZWzmEzJnWti59vpX32AIVaqsYPRfYcVGSGWkuirrVoepD+9KW3DXP0T69Vt6RAKS8a2qcGnpdRDCjWDIMtu9LfR20DKjJbDzNcp/e1PHHcvVg9YAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}x_{1} - x_{3}\\\\- x_{1} + x_{2}\\\\- x_{2} + x_{3}\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡x₁ - x₃ ⎤\n",
       "⎢        ⎥\n",
       "⎢-x₁ + x₂⎥\n",
       "⎢        ⎥\n",
       "⎣-x₂ + x₃⎦"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "M * x_vect"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the solution vector, we have three equations that we can create."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ { x }_{ 1 }-{ x }_{ 3 }={ b }_{ 1 }\\\\ { x }_{ 2 }-{ x }_{ 1 }={ b }_{ 2 }\\\\ { x }_{ 3 }-{ x }_{ 2 }={ b }_{ 3 } \\tag{11}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding the left and righ-hand sides gives us $x_1 - x_3 + x_2 - x_1 + x_3 - x_2 = b_1 + b_2 + b_3$.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This leaves us with a constraint for $\\mathbf{b}$ such that $ 0={ b }_{ 1 }+{ b }_{ 2 }+{ b }_{ 3 } $."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The problem is clear to see geometrically as the new $\\mathbf{w}$ is in the same plane as $\\mathbf{u}$ and $\\mathbf{v}$.  In essence $\\mathbf{w}$ did not add anything.  All combinations of $\\mathbf{u}$, $\\mathbf{v}$, and $\\mathbf{w}$ will still be in the same plane."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first $\\mathbf{w}$ above created a matrix with three independent columns and their linear combinations could fill all of three-dimensional space (they spanned $\\mathbb{R}^{3}$).  This made the first matrix what we will call _invertible_ as opposed to the second one, $M$, which is not invertible."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below, we create the three column vectors and check that $- \\mathbf{u} + \\left( - \\mathbf{v} \\right) = \\mathbf{w}$.  A linear combination of columns (vectors) one and two, produces three."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOMAAABLCAYAAACCwTDPAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAKdklEQVR4Ae1dbW7cNhTcBPldGCnQA2xu4CYniH2DfJwgyQ0c5ARBegOnJ0iTG8Q9QWrfIDlAgTpGfxdwZxiOLO1SK60tidR6CNCU+Pk45Og9Ulp6cXl5uejjF4vFKfxBW16k7cNftviPbeXGiIcMX1vkQHP9+jtWPshVBE4lY0Ts544T5F9GjPf6zqV77HWXu3PnzjHyfEOlJ115kf4bPMlQd9/qNxNcv0u0cYi4J4n4XFG5cZoDRhybWeIErnwDbz5B/t/hn7Ijna6LtaiAE/g7/EaGI11P/GVXnTnSId9LeDRdjGYsDqdSMOIYwRU7n7bBCXnJnZd95t1dZGx1YPYeEsns16jsojXjCAloex/+a5RhhBbmXyWweRf9EcJjeJpGdtdAANiNNd+oFXuNTZeZSiLSPH1/jf5tXQSAiPznKPwQ3pOrBUVgxTX8W4wNTaFFxO4U4SHipl4WtEhZdvQU8w1jcYJ2OEZc6nGp1OpaNSMqoJlAE/V1a+mBEyD4BfxT+Feo+sPA1e9MdRgbmtxcNgQismPEDoEGnVF2HQgQM/gp5hs5dIBx27hn0UpGFKZWPIOwfTZtOrrt5IERoOlzlqjzC+I46LQw7ApBAByipUIepTbNKimTZIwMpmacTCtWEvmiDwIHyERTftVpXc90u7IQIJeW4NZRm1hJMiLzG/i+rzLa6nb8CAj01Hr3R2jaVd4AAWhHWjLUjuRW0q2REYNNjUjPBaddeQiIaNKCdQmlLW2m1lEp55qc2ouW55pUa2REDjF3kh3UNYkcMQQCPw9RiesYFgFoR224iWONBhqvNqIJxB2fExRMPXkbhX2TBQFpv1Tj0pr/pBJ3LS7O1z/Rr20sAe6e0mTM5UjIJ7RAV+VokBGZnkUJbaLmGqqOdvmQxEAyV2oCKu5WvGeMCuPXDshKSya3qPCewzceCqtmqr6h8+uM0oawKQ/HZ9mMCnfSjB6/BDglROEBorHhu+KGq8gYVT63xLmLahO1AVNxNx8h0cOEVNQSfDfs8UuAU1AUCcmNHG6UVq4iI2JkomqRWWXyRVkIgGzcXDvHYFZfdMSHKcfwRVnSWpoEAp9jHE3VytXXjPpuThmrTFNeYFLxqc+1j578/N6StvUXTEL+nMbuBwLUgm+AjczVR7h/DIwa6xCDtRmBTPNNpiot0crVyRgSMJjKWGWa8gLta906ZbOzaws40RT1F1I3HLkc8w1tnuEhQMnXzdT4dKU2uhW7cDccPxc3AkMgECwYcK/SjloziqEm4xAwuw4j0I3AXzGLuLcQGbVe9HqjG0TnMAJDIMDfo9JxrR+cyKjNEv4Ex84IGIHxEZBmXDNTtSNnM3X8QXALRoAIiGvcqwlOmlERyqB0h0bACIyAQNwNDzXHDdTFXV0wtp5hhPZdpREwAk0EpPyCZUrNqN0cJTSz+84IGIGxEOC7YrqKjOECEUoIqf5jBIzA6AhIAT5gS9SM+iHq+ehNuwEjYATqCIhz89aMWOuOdehsHazir41D9xABo1IPe5Y1GjZQ+W1quEAolnb3LlMOgEpZeYQkZeW7UZnYuLw9zjj0H2tgNYfDnu+zRzRTw0X/7uXLyd1e+CkOnc3XyR4tG4ceICELiFj6Yc86HiUoRJJR7kIXDo3AjiDAXwCdJfpSymHPDc6RjIGVCYEdZQTmjgA/NUstv0SC6lO0zB2tzFTJIZWpe4dGYLYIxHV1l/y5l2h6UASFWF8z6mnR1QGnG4E5ICCipeZ1gwSldMZmaikjYTlyIKB37DnaZpuNBwVfbcjpaaH7wcNoOszt0NnBcXCFkyCwaT5La+ZemlUykht1Mv53Q4h+iuUVrlUXP0TnQUo53C85Gk20KXwUJrJkiyoFIwIgfBRuBQrnGiY4y4T12EphxelztJXkztuhcGr0rf5qo07MTmkSGf6NcQoTWbJG/Z219avGhY/Cq5T8V6VgRCSEj8LroMPD1ZaJgvdj3HUPXxsKp6pvfHiQjA27NSG4o4zAXBEo/bBnPRQCviTjeUS6kTBX9IeWu+cW+dDNzq6+EnGCtpnVYc83NU0nnzQY9MkOOY4T7DtCHpmfa62bxHhKHJIC1CJLxglictxmcdgzyTgrMxWkmOyQY9rxmGhc5PP/IuzxHtdFuClx6Opw4ThxzEo97HkvYhvmFckoM1UJXdjfqnRMtAcgYvU/LW5V57forHHaAqyrrI2lYX03NfcL0CsRy7t6xKd/eWIVJ5Fx2m5IpACDQqzvpiphu+p2PHdcD+V+OVw8ysbpWkMkBRge9N5N7cbwGbSi//uVcepGYPscUoCVZvwa61DC9lXucAkQkdvjdh0IGKcOgNLJWjOGL4GoGfVJkBLSxRxrBIzA0AhIAQaFWCfjcuiWXJ8RMAIbEZACXNOMYunG0k40AkZgMASkAH+Qsb5ljx0xJQ7WmisyAkZgHYG4+ywFWGlG5tS60WRcx80xRmAMBMQ1nnh4wQbuxlbOYqgM8TZPgKdGqYfO5gHkBq0CSx/2DPwKxEH/E1WKcKEPxXl0HT/5yv4xNEA7hRxv8bT4hJAgUpWfIjxEXCU40+zSCETMfNhz2Ydeh/+vgRGsflMpzagIsTU9yiPHYhKVfujsyAgMUz3NHngf9lw2DvtxtKkIgwtkxMDJTFUGpU8dln7o7NR4uL3dRUBLQinCas3ILgcTENpJmXLAcIBGw6dBK42HBS7imG5nBGaNQFxGkGfV5g07FDRj7FlYo+E6y4SPAkZRWgO9JG3N4AQjMAMExLFKK1LmOhk/x04cZuqMiCYtWBdD2pKbOXZGYO4IPI8d+FDvSEVGrBvJUhJBrK3nK+VaPzkpRR7LYQSug0DgGDgnazTUcW+lpj9w/xIm435tU2cly2i30n6pBqQ1b8XvCqPJ7sOeUzNh5nHkFrpAC69BRHZrlYw87ImvF6hGtcOKy/EdyD/mobPjd2DAFogFqsv+znfALrmqKwRkeTZMVCZXZipvMAlkqpKQORzbXyYalmZsLHgT+RxlBEpHgIqOu6hrmrFBxtiLtwh5GpoYPGXnqJlTHx5QS/C4RGoMOyMwSwSiiUozNfmD9TUyYsLriIlXU/cYbc/q0Nmp8XF7s0fgTewBFd6aW10zKgMJecSNhAzaiFpwFofOCqwSQ4wdrQxuFMjS4Pe93Af4Unvglij6oDKVggO5hI7x++9PbZxqIyOZewRPJk96AGwUdNI20cedc8BxssOeSwavIBy0D9M6t9fMVAJbI0TQjiWDbdmMwEwQoGJ7D261/vIoSUZ2DoVoqnLDRHYuo+2MgBHYEgGYqLQyaaa2akVW2UpGJsK9gKd2XIY7/zECRmArBOJa8R0KvYoWZ2v5jWREYb4LoT9urcEJRsAIbEKAP/I+AZeSrzPqBTeSkRlRCTcClmC4FqD18r42AkagBQFwhu/q6fttpoFsJNxGj8r4ovI7PF91JPPGPJcIU/5jW7kx4iEDD4VNyYHm0vJPFV8KTiVjxLHYBZwixgd951bbqw2CUTlUdgaWk938LI2bOinHXaK2DwVad5BSFQ0QRxu9VFcKTiVjxLGbNU5xrch1Yu9POP8Hpi2+HpS+pNgAAAAASUVORK5CYII=\n",
      "text/latex": [
       "$\\displaystyle \\left( \\left[\\begin{matrix}1\\\\-1\\\\0\\end{matrix}\\right], \\  \\left[\\begin{matrix}0\\\\1\\\\-1\\end{matrix}\\right], \\  \\left[\\begin{matrix}-1\\\\0\\\\1\\end{matrix}\\right]\\right)$"
      ],
      "text/plain": [
       "⎛⎡1 ⎤  ⎡0 ⎤  ⎡-1⎤⎞\n",
       "⎜⎢  ⎥  ⎢  ⎥  ⎢  ⎥⎟\n",
       "⎜⎢-1⎥, ⎢1 ⎥, ⎢0 ⎥⎟\n",
       "⎜⎢  ⎥  ⎢  ⎥  ⎢  ⎥⎟\n",
       "⎝⎣0 ⎦  ⎣-1⎦  ⎣1 ⎦⎠"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "u = Matrix([[1], [-1], [0]])\n",
    "v = Matrix([[0], [1], [-1]])\n",
    "w = Matrix([[-1], [0], [1]])\n",
    "u, v, w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABLCAYAAAAGaxWkAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACuElEQVRoBe2b7UkEMRCG98TfIgoWcHZwagVqB54dqB0oViDagVqClqAViHagBQgeYgPrO0dGArf5nLnF4yYQks1kkmfffHm7a9M0zQixDcSHtm2bPiL6fw8woPu2WYWRww0yVNkPH/7FnPPXHe0fouyIy1nRYR/KlfQBwFPEqaIrTNtXOhgMRojviOslffpDX+JXVNdB3cNpgriLOCxqAJV7AcXYfaOvMcEB+hwJTbei0PvQF9F5lQ3UE0Mla4qqyOg1krXq3fbyDL+SvW+M1f7m9SXKZoG67WVH1JPQ2eaoUMAZ94VRNGuOztxeRQEW5APcaDHSWU/hFWW02F6wBuhPzGjoDRQw07M+ShMxLszQG2hkFKtMpmiVbBEnUzQiTpXJFK2SLeK0vIri/K56wBARc2pSOevdLwDRA4ZeQN0vANEDhhTo8s7RlDK1dlO0VrmQnykaUqa2nBRdc86c1rY1D78tbpRAf9wFp2z7D+knQ9gcZSW0UpWznmCkDxhSN6QGKn3AkAK1OZpSqNRuipYqlqpviqYUKrWboqWKpeqboimFSu1qRyh1jPOevwv5wuU24jWOVpXvUtRAAfkKsCuAPSIlaHoDQm8+DjVgVeYoYOgjlXWGJFDk6asHgr6la2lQAQUEPSXpekH7gvIDp66IVQv0ABSTDhJSlQLZRUEMmqnWhogSzmJQtMEQrJ7PxCrTwhIFDdAcgM2cSrE6GqCsWlc/rDbtq6IgBnXbEEF0DS+XiTd9MaiT6Qnp0OX9hBUluyhogdK7eH4P7wPRdyhvnuq+rSivAgqQO/Q6wVb1962n27aOUX5SRBSorHbWo31S7xKAPAX2cL2Pm+g6sQI44WI1UDe8F+GuZBaVoZch5HkbaJ5O+bVM0Xyt8muOUHUh/mGAzuGzwH2Jz+hAu13F/MOwy9b8AumDoxmx1r6NAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}-1\\\\1\\\\0\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡-1⎤\n",
       "⎢  ⎥\n",
       "⎢1 ⎥\n",
       "⎢  ⎥\n",
       "⎣0 ⎦"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "-u"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABLCAYAAAAGaxWkAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACu0lEQVRoBe2b/23UMBTHcxV/o6pIDHBscNAJaDegbFDYgIoJqrJB2xHKCDBB1dvgGACpFWKB8P1Wfq2l84/E7xE16rNk2Xkvtj/5OrFj567rum6F2GfiVd/33RQR7W8yDGi+717AKeEbMjw5Dr/ig/+cP0vUfwjbB7GLossplBvTBgA/IW4pKuDN6WKxEFVuUckbxDO0YtIrcdc3A7IgIG+QnALsezjeRXoD+6EF7A4r1QbAsIt2BZL1If8HCaHPeawNJqCAOEJcJ2CuYTvAhVBdVbACPQDFXYKEqjLQrwpq0IFq7akoUVgNijoEQtSLmUTlJ9P1MVwq/yplHGOzUFRUS7UranNcVQU1aBiGCJHqXrGpB301aJDpB9JlyMeJKEq/KliBXoHiXYLkLWzrSPXEKcNMJqAAuUBzdxiqHt50wrD1EfbjYSjls8zmejRD9b4CUG6BfRy/x0WkZqwyVcJrBhq69yTRhonJpOtNSCqVOGhFoNFuV3S0ZJUCrmhFoNFuV3S0ZJUCrmhOILwLrBA34aUld9qW3Wyu36o5MgSoS5i4GuDr4DJyD8pOAhpeWLj2547KFySrQXTRSX6PRmKYZF1RExmjSmaj6KCnPgwvP3GBsk6PrjWbPbJaL7EFgr4MTUkaDh+TMLxw8TZ1eC0Nsuv/hgNJxfcU0t8CMZt7dDaggx4mkV+T4oHktg8fRtn64YcIbk5c4xngN65imAwUMPdzfZGm4JxN1ztooRebXK5ok2yFQq5oQZwmlyvaJFuh0PNVFPN30wZDQcx7l8lcH1YAqg2GSUDDCkC1wVADfb73aE2ZVr8r2qpcrpwrmlOm1e6KtiqXK+eK5pRptZvM9Wxcu8FQuwAzUO0GQw3U79GaQmP9ruhYxWrnz0ZRXgg/983iDwP8JeJnEieC+leKiTpzJvlpfNL/D2J/oxleMjisAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}0\\\\-1\\\\1\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡0 ⎤\n",
       "⎢  ⎥\n",
       "⎢-1⎥\n",
       "⎢  ⎥\n",
       "⎣1 ⎦"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "-v"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABLCAYAAAAGaxWkAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACwUlEQVRoBe2b4W3UQBCFfRG/URQkCjg6OJIKuHRA6ADSQRAVREkHSUoIJUAF0V0HRwFIRIgGzHvRDlolXtszO1iyMiOtbO/u7Hx+69312b6maZoVUltIt23bNlMkxN8VGBC+bV6gUOwSO6yc24/84D/vX3S0f4y895Ivii6nUE4TA4CfkB4U3RPaqbaLxWKFtEPa18TMu17jp6qboG7gdI90iLRUNYDKk4Ci734j1gnhAH2GDS83lU3e9Sq6rHKAZmK47IaiLjJmjYwa9Wl6+Q4/zdx3gtG+zWJV7Y4CTdPL26pIlc5xjVYK+MR9NoqOukafnJ4hAwPyFm4cjFzraRvkcbDdYQzwFrPXJgMFzMNa30vTUzibrg/Qnl40FYWiJtl6nELRHnFMRaGoSbYep9ko6rqEYu2WxzK/IM4bpAssnS6PhdxAAbkB2DnAvrIHccwbEN54HHvAunQ9YPiMaF8gCYp9PnQg9BWPa80FFBC8M+r6fXSH/HVSt4rVC3QNCj5XemxUlcbyKqsGHanWQRUlnKtB0YZAiHo5k6jMgVVlHqBjAF6NqdRXxwNUVOuKI2pzXq0ygr5MLchW1WCahujT1b2SZ530XwsMQf+kA9lKmWb7DZWXHQ6iKMst9lOcPLqebfGnsPwMlra55WOgbaZ6XqbadwEFyDWi3mOq+veqJU1bH5D/UUVUqOy21qN9qvcFgHIJHOH4HU5iW4itynYDTd37WRVdUdml6xXxzFUD1CxdwTEULQhjzg5FzdIVHEPRgjDm7OerKNZ60xcOQ1K7rPXpTqnqC4dJQNMNSdUXDkOgz/caHVLGWh6KWpUr+YWiJWWs+aGoVbmSXyhaUsaa77LWMzjW+6ovHIZOwA0U633VFw5DoHGNDimkLQ9FtYoN1Z+NojyRFdIs/jDAFwGnJO4w60uCjqYGs+TNdGfFvxgHoxnBVwSXAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}-1\\\\0\\\\1\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡-1⎤\n",
       "⎢  ⎥\n",
       "⎢0 ⎥\n",
       "⎢  ⎥\n",
       "⎣1 ⎦"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "-u - v"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACoAAABLCAYAAAAGaxWkAAAACXBIWXMAAA7EAAAOxAGVKw4bAAACwUlEQVRoBe2b4W3UQBCFfRG/URQkCjg6OJIKuHRA6ADSQRAVREkHSUoIJUAF0V0HRwFIRIgGzHvRDlolXtszO1iyMiOtbO/u7Hx+69312b6maZoVUltIt23bNlMkxN8VGBC+bV6gUOwSO6yc24/84D/vX3S0f4y895Ivii6nUE4TA4CfkB4U3RPaqbaLxWKFtEPa18TMu17jp6qboG7gdI90iLRUNYDKk4Ci734j1gnhAH2GDS83lU3e9Sq6rHKAZmK47IaiLjJmjYwa9Wl6+Q4/zdx3gtG+zWJV7Y4CTdPL26pIlc5xjVYK+MR9NoqOukafnJ4hAwPyFm4cjFzraRvkcbDdYQzwFrPXJgMFzMNa30vTUzibrg/Qnl40FYWiJtl6nELRHnFMRaGoSbYep9ko6rqEYu2WxzK/IM4bpAssnS6PhdxAAbkB2DnAvrIHccwbEN54HHvAunQ9YPiMaF8gCYp9PnQg9BWPa80FFBC8M+r6fXSH/HVSt4rVC3QNCj5XemxUlcbyKqsGHanWQRUlnKtB0YZAiHo5k6jMgVVlHqBjAF6NqdRXxwNUVOuKI2pzXq0ygr5MLchW1WCahujT1b2SZ530XwsMQf+kA9lKmWb7DZWXHQ6iKMst9lOcPLqebfGnsPwMlra55WOgbaZ6XqbadwEFyDWi3mOq+veqJU1bH5D/UUVUqOy21qN9qvcFgHIJHOH4HU5iW4itynYDTd37WRVdUdml6xXxzFUD1CxdwTEULQhjzg5FzdIVHEPRgjDm7OerKNZ60xcOQ1K7rPXpTqnqC4dJQNMNSdUXDkOgz/caHVLGWh6KWpUr+YWiJWWs+aGoVbmSXyhaUsaa77LWMzjW+6ovHIZOwA0U633VFw5DoHGNDimkLQ9FtYoN1Z+NojyRFdIs/jDAFwGnJO4w60uCjqYGs+TNdGfFvxgHoxnBVwSXAAAAAElFTkSuQmCC\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}-1\\\\0\\\\1\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡-1⎤\n",
       "⎢  ⎥\n",
       "⎢0 ⎥\n",
       "⎢  ⎥\n",
       "⎣1 ⎦"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "-u - v == w"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example problems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example problem 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "+ Suppose $A$ is a matrix with the following solution\n",
    "$$ {A}{\\mathbf{x}}=\\begin{bmatrix}1\\\\4\\\\1\\\\1\\end{bmatrix} \\\\ \\mathbf{x}=\\begin{bmatrix}0\\\\1\\\\1\\end{bmatrix}+{c}\\begin{bmatrix}0\\\\2\\\\1\\end{bmatrix} $$\n",
    "+ What can you say about the columns of $A$?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "c = symbols('c')\n",
    "x_vect = Matrix([[0], [1 + 2 * c], [1 + c]])\n",
    "b = Matrix([[1], [4], [1], [1]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEcAAABLCAYAAAAicppkAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAFBUlEQVR4Ae1c4U3dMBDmVf1dvVKpAzw2oGWCwgZtNyjdoFUnQHQD6ASobAAbINgABqjUB+oCr98X+R52cnFNkpdLUp9kbJ8d++7L+eL4wpttbW3tIl0jaXS+Wq0+aA1T4M1ms1vosdB0gd6z517Dd5TZ2ac7vzLB8rGi0wF478n3wTkBWlMHI8AC+p4GDFRgTWRVwCn3660OgeQO/sakO0jHQ7hRvuX0BoY/EYChvzsCGOfkoz5Hdo38wBqgZxTIigDAIeaeCzCUA+UHZATqhHVLMgUHivNJeKMAcAXevrMipbkfljU4+1BzqahK6yGx3YzMwEm0im0zZDCxGTiYWxQXK/FxEGuiczYjS3BSlH6V0mlTfSzBEevQdBOr4r7HjMzAcY9sKq4tHeGZ7tjNwHHmcIl84cp+JpbDdjOyBucnNH+raP8GvBvPupQum2eZggPl+eK3xGO9eNGjuu4R/xHFT6xbkvm7FZSnlXwDKLK89lB/B+C0nXOvWJmD45bO1161TpzMdFklymjWLYMTgT6Dk8GJIBBpypaTwYkgEGnKlpPBiSAQacqWk8GJIBBpSrYcvvsgnbjEuNIF0m5k7ME0UU6kWyQ5J0qSLendCoMuMBqjkOuPCsBjlFKCb6bnLpqmDogfaOOJI49FqMOTiXd/hbSA8gyqVRLaeO7C4FvQBt49U5nfVR1jMzRz2HY8jPEFiTpWdCiPzfnYl/zUZUUh7xWzpMXMnWWh2DlxGTxpKXQpQSo4BOEOaD7UTG6mQI08nbCTfA5AWfua0qyFQ0Z75WDK+SSJHjDEcqb1K403qGqq5VSEhvIEhk4uOKjiEkPiR1BXAOM7E8q0LPnMBMVxUGNwoB6d9KlT3tf2AhV+Lld8UuIaeCZM/qgoaVmVNYJl8POQSwDw2W9zfFrTkc9Hv5d+vVx219Hpl2mbDLQH87hOjE7ULffyOI3qBOeFu1Ly6EAQlI86Pus0gWkhBK3Ocatj14xFUBiV4BaDS7Mvei0TEZw/riK5tFVyJ+yOrwx4tBQSN1v0LRXnzMYR0S+RNdnnAAQ64D0AEzhg8Hh3l+CLtZS/SC3mwvXasinahvqHlvNPctZBB3yJsv85Gn3Crmf2DNIxDhUQruGT6ixgjqCSBA70IDBcPoW/Kem1XkZcbgTCgSF7HHbnB5FiWaXLN1eFHMVrD2bguxWJ74KUt9hmFJzInyRwoFjFGurGRN/ysqvrunE+ZGn1NEv2ORvXRJ+A1ta7xYkoSZYjnfvOcedNj0KGbjl9349gvgxOAEdYyeCEeAS1DE4AR1jJ4IR4BLUMTgBHWMnghHgEtQxOAEdYmRQ4eG9qFLwLIXmsDXqH/ChmfQmAzNHaOninzTAYy4GS/Ocz7a1fk3vNwyvGA18wkXgy2emxyGDAgWK0AKbB0JDAGQwoIkgnPgfLgSd9crg1ygCeAOLnrSwHoEwmgOeDIuVW4GCQyQTwBBA/b7ysYDU8aF8gTSKA54Mi5cbgYIApBPAEBzVvtKxgNfLYXUce1NFHzmwEDjdeTu/JBPC0+9gIHDdQLIC31CYbG6+xz+F2nfsbJigtexzq33sADzK0Ct7V3bTG4HBAADSIAB7kaBW8qwOnzbKqG7Mpn35MfFnTMTq9rpXldCkJ7r5pAE/TZUiWo8lnysvgRODP4ETA8X0O/3Gi3PW//RFFAkFw+Esj2sePbDf9FRIKsGHiHq2W/gK3EAQfDZa9YgAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}0\\\\2 c + 1\\\\c + 1\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡   0   ⎤\n",
       "⎢       ⎥\n",
       "⎢2⋅c + 1⎥\n",
       "⎢       ⎥\n",
       "⎣ c + 1 ⎦"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_vect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABoAAABkCAYAAACYYiB/AAAACXBIWXMAAA7EAAAOxAGVKw4bAAACTUlEQVRoBe1a7U0CQRA9jL8N8YcFUALRDrQDjR1IB1qDdoCWQAnSgcEOoAB/KLGB873LzrFL7vZmTkIImU0m+zGz827e7g3LQlEUxRhStsisLMtCI5i/bPGB6WVxCqWUFzRoHJdV3OloPzfobzB2K+MS0Ujz5BYbADxAqohOBE1bDwaDMWQJGWrn0C6mrnVecPoGg2/IJWTUatyiUAEh9jXm39EHQB9RkW5TMVNn8h4ZO1BEhq3p1Nn4iqyduogMW1OVgugSqWeGiomUuY5lgbFP1B9IUfyIyRY1EJxVuS7rLaP0zZAhJ69y6vL8ZLROXYacvOrwqUOe40Fymo9jo/1PREyy5xtX+VYvIETCQ6SpmIFIGRB4cqWoixkInu/xkfGqRgiGJqBAmXoDxA+jBgLICBPXiMby5azGUgNhxqQPZYJEoLPQkVp0dY1o+M2tD2UX4qQzokDZsC9lAsTDyW/oSC06qbk2VwDkCxoXbvNRGF/hQZ5iZWh/yVjnKQgO5jCmJAUAPxiYQ686HXVSl3hPOzzjUVTFDIRIppBF8H6N9jukMyV1Urf9uKBqsj2m6Zsj0jhtsnGgJlZUY8dJHdM+tywvlHZdmFHq14E5izeQh3Vf1zfkw90MyGt+A1kt6+Gu0fHtOo9IGPBdJ0yY671Rpz7XIZn6DWSyjntbIwdKeLd0nDoLW4mtU8dfXfwAWW0K3wzJu2HpOHUWthJbpy6hw9LZG3V+gLQsS2K7tzVyoIR3a0fu65r+NbjTfwzKDWTTA1p+lGr6x2Dt8w9C3lFtYRqoBQAAAABJRU5ErkJggg==\n",
      "text/latex": [
       "$\\displaystyle \\left[\\begin{matrix}1\\\\4\\\\1\\\\1\\end{matrix}\\right]$"
      ],
      "text/plain": [
       "⎡1⎤\n",
       "⎢ ⎥\n",
       "⎢4⎥\n",
       "⎢ ⎥\n",
       "⎢1⎥\n",
       "⎢ ⎥\n",
       "⎣1⎦"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the following:\n",
    "+ $\\mathbf{x}$ is of size $m \\times n$ and that is $ 3 \\times 1 $\n",
    "+ $\\mathbf{b}$ is of size $ 4 \\times 1 $\n",
    "+ Therefor, $A$ must be of size $ 4 \\times 3 $ and each column vector in $A$ is in $\\mathbb{R}^{4}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's call these columns of $A$ $C_1, C_2, C_3$, as illustrated in (12)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\begin{bmatrix} \\vdots  & \\vdots  & \\vdots  \\\\ { C }_{ 1 } & { C }_{ 2 } & { C }_{ 3 } \\\\ \\vdots  & \\vdots  & \\vdots  \\\\ \\vdots  & \\vdots  & \\vdots  \\end{bmatrix} \\tag{12} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the particular way in which $\\mathbf{x}$ was written, we can say that we have a particular solution and a special solution, denoted as $ {A}\\left({x}_{p}+{c}\\cdot{x}_{s}\\right)= \\mathbf{b} $."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For $ c = 0 $ we have:\n",
    "$$ {A}{\\mathbf{x}}_{p}=b $$\n",
    "\n",
    "For $c = 1 $ we have:\n",
    "$$ \\begin{align} A{ x }_{ p }+A{ \\mathbf{x} }_{ s } &= b \\\\ \\because \\quad A{ \\mathbf{x} }_{ p } &= b \\\\ b+A{ \\mathbf{x} }_{ s } &= b \\\\ \\therefore \\quad A{ \\mathbf{x} }_{ s } &= 0 \\end{align} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also have that the following\n",
    "$$ { \\mathbf{x} }_{ p }=\\begin{bmatrix} 0 \\\\ 1 \\\\ 1 \\end{bmatrix},\\quad { \\mathbf{x} }_{ s }=\\begin{bmatrix} 0 \\\\ 2 \\\\ 1 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For $ \\mathbf{x}_{p}$ we have the following:\n",
    "$$ \\begin{bmatrix} \\vdots  & \\vdots  & \\vdots  \\\\ { C }_{ 1 } & { C }_{ 2 } & { C }_{ 3 } \\\\ \\vdots  & \\vdots  & \\vdots  \\\\ \\vdots  & \\vdots  & \\vdots  \\end{bmatrix}\\begin{bmatrix} 0 \\\\ 1 \\\\ 1 \\end{bmatrix} = \\mathbf{b} \\quad \\Rightarrow \\quad { C }_{ 2 }+{ C }_{ 3 }= \\mathbf{b} $$\n",
    "\n",
    "For $ \\mathbf{x}_{s}$ we have the following:\n",
    "$$ \\begin{bmatrix} \\vdots  & \\vdots  & \\vdots  \\\\ { C }_{ 1 } & { C }_{ 2 } & { C }_{ 3 } \\\\ \\vdots  & \\vdots  & \\vdots  \\\\ \\vdots  & \\vdots  & \\vdots  \\end{bmatrix}\\begin{bmatrix} 0 \\\\ 2 \\\\ 1 \\end{bmatrix}=\\mathbf { 0 } \\quad \\Rightarrow \\quad 2{ C }_{ 2 }+{ C }_{ 3 } = \\mathbf{0} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Solving for $C_2$ and $C_3$ we have the following:\n",
    "$$ \\begin{align} {C}_{3} &= -2{C}_{2} \\\\ {C}_{2}-2{C}_{2} &= b \\\\ {C}_{2} &= -b \\\\ {C}_{3} &= 2b \\end{align} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As for the first column of $A$, we need to know more about ranks and subspaces.  We see, though, that columns two and three are already constant multiples of each other.  So, as long as column one is not a constant multiple of $\\mathbf{b}$, we are safe."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "$$ A=\\begin{bmatrix} \\vdots  & 1 & 2 \\\\ { C }_{ 1 } & 4 & 8 \\\\ \\vdots  & 1 & 2 \\\\ \\vdots  & 1 & 2 \\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}