Matrix Division?

We've seen addition, subtraction, and multiplication of matrices. What about division? Is there such a thing? The short answer is no, there is not. The longer answer is in this lesson! We're going to learn about two things - matrix inverses and determinants - and then we'll come back to this idea of division.

Matrix Inverse

Let's just get right to the definition:

Definition: If $\large A$ is $\large n\times n$ and if there is another $\large n\times n$ matrix $\large B$ such that $\large AB=BA=I$, then $\large A$ is called invertible or nonsingular and $\large B$ is denoted $\large A^{-1}$ and is called the inverse of $\large A$.

As an example, consider:

$$\large A=\begin{bmatrix} 2 & -1 & 3 & 1 \\ -1 & 1 & -2 & 1 \\ 1 & 4 & -1 & 3 \\ 3 & 1 & 2 & 2 \end{bmatrix},\quad A^{-1} = \begin{bmatrix} -13 & -25 & 3 & 11 \\ 3 & 6 & -1 & -3 \\ 9 & 17 & -2 & -8 \\ -2 & -3 & 0 & 2 \end{bmatrix} $$

We can simply multiply to verify that these are inverses of each other: $\large AA^{-1}=A^{-1}A=I$.

Definition: If $\large A^{-1}$ does not exist, we say $\large A$ is called singular.

Note

Most, but not all, matrices are nonsingular (meaning, they have an inverse).

Facts About Inverses

The following are true:

$\large \left(AB\right)^{-1}=B^{-1}A^{-1}$
$\large \left(A^{-1}\right)^{-1}=A$
$\large \left(kA\right)^{-1}=\frac{1}{k}A^{-1}$
$\large \left(A^T\right)^{-1}=\left(A^{-1}\right)^T$
If $\large A$ is orthogonal, $\large A^{-1}=A^T$.

At this point, there are three natural questions to ask:

How do we know if $\large A$ is invertible?
How do we find $\large A^{-1}$?
Why do we care?

We'll answer all three questions here! The first is related to the determinant of a matrix.

Determinants

Every square matrix is associated with a number called a determinant. For larger matrices, computing a determinant can be difficult. But the computation is straightforward for $\large 2\times 2$ matrices:

$$\large \det\begin{bmatrix} a & b \\ c & d\\ \end{bmatrix} = (a)(d)-(b)(c) $$

For a $\large 3 \times 3$ matrix, we compute the determinant through a cofactor expansion. As an example, let's find:

$$\large \det\begin{bmatrix} 3 & 1 & 5 \\ -1 & 0 & 2\\ 2 & 4 & 7\\ \end{bmatrix} $$

The technique for finding the determinant of this $\large 3 \times 3$ matrix is in the video:

Note

Determinants of larger matrices can be found in the same way, but it's an awful process. For a $\large 5\times 5$ matrix, we must compute the determinants of five $\large 4\times 4$ matrices, each of which requires the determinant of four $\large 3\times 3$ matrices, and so on.

Fortunately, there is another way to compute determinants. We'll learn soon of an algorithm called Gaussian Elimination, and this provides a faster way to compute determinants of large matrices. We will not go into the details in this class, primarily because no one should be doing this by hand! We'll use a determinant command in Python for this.

Note

The actual, technical, mathematical definition of the determinant is pretty awful to work with. It is a sum over products of elements in different columns of the matrix. It looks like this, and we will never speak of it again (the $\large \sigma$ in the following is a random permutation of the integers $\large 1, 2, \dots, n$, and the sum is over all of the $\large n!$ possible permutations):

$$\large \text{det}(A) = \sum_{\sigma}\text{sgn}(\sigma)a_{1,\sigma(1)}a_{2,\sigma(2)}\dots a_{n,\sigma(n)}$$

Some Facts About Determinants

The following are true:

$\large A$ is nonsingular ($\large A^{-1}$ exists) if and only if $\large \text{det}(A)\ne 0$ (this means $\large A$ is singular if and only if $\large \text{det}(A)=0$).
If $\large A$ has a row or column of zeros, $\large \text{det}(A)=0$.
If two rows or columns of $\large A$ are multiples of each other, $\large \text{det}(A)=0$.
$\large \text{det}(AB)=\text{det}(A)\text{det}(B)$.
$\large \text{det}(cA) = c^n\text{det}(A)$ (where $\large A$ is $\large n\times n$).
$\large \text{det}\left(A^{-1}\right)=\frac{1}{\text{det}(A)}$.
$\large \text{det}\left(A^T\right)=\text{det}(A)$.
$\large \text{det}(I)=1$.

Important

The first fact listed above is the single most important takeaway from this lesson! This is how we know if $\large A$ is invertible.

The video below shows some examples of using the first few facts from this list:

Solutions of Systems of Linear Equations $\large A\vec{x}=\vec{b}$

We started this lesson with questions about matrix division. What does the matrix inverse have to do with division? What is division for? It's for solving equations like this:

$$\large 3x=9\Rightarrow x=\frac{9}{3}$$

We'd love to do that with matrices:

$$\large A\vec{x}=\vec{b}\Rightarrow \vec{x}=\frac{\vec{b}}{A}$$

There is still no such thing as matrix division, but the inverse gives us something similar:

Start with $\large A\vec{x}=\vec{b}$.
Multiply both sides of the equation by $\large A^{-1}$: we have $\large A^{-1}A\vec{x}=A^{-1}\vec{b}$, which is just $\large \vec{x}=A^{-1}\vec{b}$, the solution!

If $\large A$ is singular, $\large A^{-1}$ does not exist, and we cannot "divide" by it. This is analogous to trying to divide by zero! This is the situation when $\large A\vec{x}=\vec{b}$ has no solution.

In this way, multiplication by $\large A^{-1}$ acts like matrix division. This table elaborates on the comparison:

	$\large ax=b$	$\large A\vec{x}=\vec{b}$
Solution	Multiply both sides by $\large \frac{1}{a}$	Multiply both sides by $\large A^{-1}$
No Solution if	$a=0$, divide by 0	$\large \text{det}(A)=0$, $\large A$ is singular, $\large A^{-1}$ does not exist

Important

We see that an inverse can help us find solutions to $\large A\vec{x}=\vec{b}$. This is often very useful as a theoretical tool (to determine if solution exists and to prove things about systems). It is not the approach we take to solving systems! We will learn the actual approach in our next lesson! Finding $\large A^{-1}$ is not how we solve a system of linear equations.

A Final, Important Word on $\large A^{-1}$

We now know how to tell if a square matrix $\large A$ is invertible. But we haven't answered two of the questions from earlier. Here are the answers:

How do we compute $\large A^{-1}$? There are three answers to this question:

The first (and best) answer is: WE DON'T. We can count on one hand the number of times we've needed to explicitly compute $\large A^{-1}$ by hand. It is a very useful object from a theoretical point of view, but it's almost never important to actually compute it out completely.
The second answer is: for $\large 2\times 2$ matrices, there is a handy formula:

$$\large A = \begin{bmatrix} a & b \\ c & d \\ \end{bmatrix} \quad \Longrightarrow \quad A^{-1}=\frac{1}{ad-bc}\begin{bmatrix} d & -b \\ -c & a \\ \end{bmatrix} $$

The third answer is: The same process we will learn in our next lesson for finding solutions to $\large A\vec{x}=\vec{b}$ can be modified to find $\large A^{-1}$.

Why do we care about $\large A^{-1}$? The matrix inverse is an important concept, even if we never really compute it. It carries structural ideas of solvability. We know $\large A\vec{x}=\vec{b}$ has a solution if $\large A$ has an inverse.

A "Good" Square Matrix

We've now been introduced to a few equivalent facts about square matrices. These are characteristics of "good" square matrices, and we will find other equivalencies during this class. Let's write down what we know so far, and we'll expand on it as we go.

The following are all equivalent facts about a square $\large n\times n$ matrix $\large A$.