layout | title | date | categories | parent | math |
---|---|---|---|---|---|
default |
All solutions to Ax=b |
2021-04-13 |
math |
Linear Algebra |
katex |
It should be simple right? Well, there are 7 different possible cases, and only 3 have simple solutions. Lets summarize.
We study
where
We will also use the adjoint
A few useful properties:
- If, and only if,
$$b \in R(A)$$ , there exists a solution$$x$$ . - If
$$b \not \in R(A)$$ we are interested in finding a solution to$$b_r$$ which minimises the residual,$$|Ax - b|$$ . - If there are linearly independent columns,
$$A$$ must be tall, and admits the left inverse$$A^+ = (A^* A)^{-1}A^*$$ - If there are linearly independent rows,
$$A$$ must be fat, and admits the right inverse$$A^+ = A^* (A A^*)^{-1}$$ .
| |
Following Boyd's textbook on linear algebra, we define
- The left inverse
$$X$$ of a matrix$$A$$ is any martix that satisfies$$X A = I$$ . - The right inverse
$$Y$$ of a matrix$$A$$ is any matrix that satisfies$$A Y = I$$ .
These have the following useful properties (proofs are simple, and I refer you to Boyd's book, Ch 11):
- The left inverse exists iff and only if the columns of
$$A$$ are linearly independent. - A matrix can only have a left inverse if it is tall (or square).
- If a matrix
$$A$$ has left inverse$$X$$ , then a solution to$$Ax = b$$ is$$x = X b$$ . - A left invertible matrix
$$A$$ has left inverse $$X = (A^A)^{-1}A^$$. - Left inverses are, in general, not unique.
Similarly, we can take the transpose of the above claims:
- The right inverse exists iff and only if the rows of
$$A$$ are linearly independent. - A matrix can only have a right inverse if it is fat (or square).
- If a matrix
$$A$$ has right inverse$$Y$$ , then a solution to$$Ax = b$$ is$$x = Y b$$ . - A right invertible matrix
$$A$$ has right inverse$$Y = A (AA^*)^{-1}$$ . - Right inverses are, in general, not unique.
Then suppose a matrix is both left and right invertible. Then the left and right inverses are equal and unique!
Thus the inverse of a matrix
Notice that this only makes sense for square matrices, since for the left inverse to exist it must be tall, and for the right inverse to exist it must be fat. Thus,
The two special inverses $$X = (A^A)^{-1}A^$$ or
But what about if this is not the case? We can use the Moore-Penrose Psuedo-Inverse, which is any matrix
$$AA^+A = A$$ $$A^+ A A^+ = A^+$$ $$(AA^+)^* = AA^+$$ $$(A^+A)^* = A^+A$$
The psuedo inverse always exists, and is always unique. If
todo, proofs of each of the three simple cases above.
todo.