Deploy

content/notes/math/linear_algebra.mdx

@@ -678,6 +678,7 @@ $$
     - $\operatorname{tr}(\boldsymbol{A} + \boldsymbol{B}) = \operatorname{tr}(\boldsymbol{A}) + \operatorname{tr}(\boldsymbol{B})$
 2. **Commutativity:** $\operatorname{tr}(\boldsymbol{A}\boldsymbol{B}) = \operatorname{tr}(\boldsymbol{B}\boldsymbol{A})$
 3. $\operatorname{tr}(\boldsymbol{A}\boldsymbol{B}\boldsymbol{C}) = \operatorname{tr}(\boldsymbol{C}\boldsymbol{A}\boldsymbol{B}) = \operatorname{tr}(\boldsymbol{B}\boldsymbol{C}\boldsymbol{A})$
+4. The trace is invariant under similarity.
 
 <details>
 <summary>Proof</summary>
@@ -4835,7 +4836,7 @@ $$
 
 **(1):** This follows from the isomorphic characterization of modules of linear operators and the fact that the elementary divisors form a complete invariant for isomorphism.
 
-**(2):** If $\mathbf{A} = [\mathrm{T}]_B$, then the coordinate map $\phi_B: V\cong\mathbf{F}^n$ is also a module isomorphism $\phi_B: V_\mathrm{T} \to\mathbb{F}_\mathbf{A}^n$. Specifically, we have
+**(2):** If $\mathbf{A} = [\mathrm{T}]_B$, then the coordinate map $\phi_B: V\cong\mathbb{F}^n$ is also a module isomorphism $\phi_B: V_\mathrm{T} \to\mathbb{F}_\mathbf{A}^n$. Specifically, we have
 
 $$
   \phi_B (p(\mathrm{T})\mathbf{v}) = [p(\mathrm{T})\mathbf{v}]_B = p([\mathrm{T}]_B)[\mathbf{v}]_B = p(\mathbf{A})\phi_B (\mathbf{v})
@@ -4915,7 +4916,7 @@ $$
 <details>
 <summary>Proof</summary>
 
-If an $m\times m$-matrix $\mathbf{A}\in\mathcal{M}_m (\mathbf{F})$ has minimal polynomial $m_\mathrm{T}(x) = \prod_{i=1}^n p_i^{e_i}(x)$ of degree equal to the size $m$ of the matrix, then the previous theorem implies that the elementary divisors of $\mathbf{A}$ are precisely $\prod_{i=1}^n p_i^{e_i}(x)$. Since the matrices $C[p(x)q(x)]$ and $\operatorname{diag}(C[p(x)],C[q(x)])$ have the same size $m\times m$ and the same minimal polynomial $p(x)q(x)$ of degree $m$, it follows that they have the same multiset of elementary divisors and so are similar.
+If an $m\times m$-matrix $\mathbf{A}\in\mathcal{M}_m (\mathbb{F})$ has minimal polynomial $m_\mathrm{T}(x) = \prod_{i=1}^n p_i^{e_i}(x)$ of degree equal to the size $m$ of the matrix, then the previous theorem implies that the elementary divisors of $\mathbf{A}$ are precisely $\prod_{i=1}^n p_i^{e_i}(x)$. Since the matrices $C[p(x)q(x)]$ and $\operatorname{diag}(C[p(x)],C[q(x)])$ have the same size $m\times m$ and the same minimal polynomial $p(x)q(x)$ of degree $m$, it follows that they have the same multiset of elementary divisors and so are similar.
 </details>
 </MathBox>
 
@@ -5046,7 +5047,7 @@ $$
 ### Changing the base field
 
 <MathBox title='' boxType='proposition'>
-Let $\mathbb{F}$ and $\mathbb{K}$ be fields with $\mathbb{F}\subseteq\mathbb{K}$. Suppose that the elementary divisors of a matrix $\mathbf{A}\in\mathcal{M}_n (\mathbf{F})$ are
+Let $\mathbb{F}$ and $\mathbb{K}$ be fields with $\mathbb{F}\subseteq\mathbb{K}$. Suppose that the elementary divisors of a matrix $\mathbf{A}\in\mathcal{M}_n (\mathbb{F})$ are
 
 $$
   A = \Set{p_1^{e_{1,1}},\dots,p_1^{e_{1,k_1}},\dots,p_n^{e_{n,1}},\dots,p_n^{e_{n,k_n}}}
@@ -5095,12 +5096,36 @@ Let $\mathbf{A}\in\mathcal{M}_n (\mathbb{F})$ and let $\mathbb{E}\subseteq\mathb
 
 # Eigenvalues and eigenvectors
 
+Let $\mathbf{A} = [\mathrm{T}]_B$ be a matrix representing a linear operator $\mathrm{T}\in\mathcal{L}(V)$ on an $\mathbb{F}$-vector space $V$. A scalar $\lambda\in\mathbb{F}$ is a root of the characteristic polynomial $c_\mathrm{T}(x) = c_\mathbf{A}(x) = \det(x\mathrm{A} - \mathbf{A})$ if and only if
+
+$$
+  \det(\lambda\mathbf{I} - \mathbf{A}) = 0
+$$
+
+or equivalently, if and only if the matrix $\lambda\mathbf{I} - \mathbf{A}$ is singular. In particular, if $\dim(V) = n$, then the characteristic equation holds if and only if there exists a nonzero vector $x\in\mathbb{F}^n$ for which
+
+$$
+  (\lambda\mathbf{I} - \mathbf{A})\mathbf{x} = 0
+$$
+
+or equivalently
+
+$$
+  \mathrm{T}_\mathbf{A}x = \lambda x
+$$
+
+If $[\mathbf{v}]_B = x$, then this is equivalent to $[\mathrm{T}]_B [\mathbf{v}]_B = \lambda[\mathbf{v}]_B$, or in operator language
+
+$$
+  \mathrm{T}\mathbf{v} = \lambda\mathbf{v}
+$$
+
 <MathBox title='Eigenvalue, eigenvector and spectrum' boxType='proposition'>
 Let $V$ be a vector space over an algebraically closed field $\mathbb{F}$, which means that every non-constant polynomial has a root in $\mathbb{F}$. 
 1. A scalar $\lambda\in\mathbb{F}$ is an *eigenvalue* of an operator $\mathrm{T}\in\mathcal{L}(V)$ if there exists a nonzero vector $\boldsymbol{v}\in V$ for which $\mathrm{T}(\boldsymbol{v}) = \lambda\boldsymbol{v}$. In this case, $\boldsymbol{v}$ is an *eigenvector* of $\mathrm{T}$ associated with $\lambda$.
 2. A scalar $\lambda\in\mathbb{F}$ is an eigenvalue for a matrix $\boldsymbol{A}\in\mathcal{M}_{n}(\mathbb{F})$ if there exists a nonzero column vector $\boldsymbol{x}$ for which $\boldsymbol{A}\boldsymbol{x} = \lambda\boldsymbol{x}$. In this case, $\boldsymbol{x}$ is an eigenvector for $\boldsymbol{A}$ associated with $\lambda$.
 3. The set of all eigenvectors associated with a given eigenvalue $\lambda$, together with the zero vector, forms a subspace of $V$, called the *eigenspace* of $\lambda$, denoted by $E_\lambda$.
-4. The set of all eigenvalues of an operator or matrix is called the *spectrum* of the operator or matrix.
+4. The set of all eigenvalues of an operator or matrix is called the *spectrum* of the operator or matrix. The spectrum of $\mathrm{T}$ is denoted $\operatorname{Spec}(\mathrm{T})$.
 </MathBox>
 
 <MathBox title='Existence of eigenvectors' boxType='proposition'>
@@ -5120,13 +5145,182 @@ $$
 </details>
 </MathBox>
 
+<MathBox title='' boxType='proposition'>
+Let $\mathrm{T}\in\mathcal{L}(V)$ have minimal polynomial $m_\mathrm{T}(x)$ and characteristic polynomial $c_\mathrm{T}(x)$.
+1. The spectrum of $\mathrm{T}$ is the set of all roots of $m_\mathrm{T}(x)$ or of $c_\mathrm{T}(x)$, not counting multiplicity.
+2. The eigenvalues of a matrix are invariants under similarity.
+3. The eigenspace $E_\lambda$ of the matrix $\mathbf{A}$ is the solution space to the homogeneous system of equations
+$$
+  (\lambda\mathbf{I} - \mathbf{A})(x) = 0
+$$
+</MathBox>
+
+<MathBox title='Relation between eigenspaces and eigenvectros of distinct eigenvalues' boxType='proposition'>
+Suppose that $\lambda_1,\dots,\lambda_k$ are distinct eigenvalues of a linear operator $\mathrm{T}\in\mathcal{L}(V)$.
+1. Eigenvectors associated with distinct eigenvalues are linearly independent. That is, if $\mathbf{v}_i \in E_\lambda$, then the set $\Set{\mathbf{v}_i}_{i=1}^k$ is linearly independent.
+2. The sum $\sum_{i=1} E_{\lambda_i}^k$ is direct, i.e. $\bigoplus_{i=1}^k E_{\lambda_i}$ exists.
+
+<details>
+<summary>Proof</summary>
+
+**(1):** If $\Set{\mathbf{v}_i}_{i=1}^k$ is linearly dependent, the by renumbering if necessary, we may assume that among all nontrivial linear combinations of these vectors that equal $0$, the equation
+
+$$
+  \sum_{i=1}^j r_i \mathbf{v}_i = 0
+$$
+
+has the fewest number of terms. Applying $\mathrm{T}$ gives
+
+$$
+  \sum_{i=1}^j r_i \mathrm{T}\mathbf{v}_i = \sum_{i=1}^j r_i \lambda_i \mathbf{v}_i = 0
+$$
+
+Multiplying by $\lambda_1$ and subtracting gives
+
+$$
+  r_2 (\lambda_2 - \lambda_1)\mathbf{v}_2 +\cdots+ r_j (\lambda_j - \lambda_1)\mathbf{v}_j = 0
+$$
+
+However, this equation has fewer terms and so all of its coefficients must equal $0$. Since the $\lambda_i$ are distinct, $r_i = 0$ for $i \geq 2$ and so $r_1 = 0$. This contradiction implies that the $\mathbf{v}_i$ are linearly independent.
+</details>
+</MathBox>
+
+<MathBox title='Spectral mapping theorem' boxType='theorem'>
+Let $V$ be a vector space over an algebraically closed field $\mathbb{F}$. Let $\mathrm{T}\in\mathcal{L}(V)$ and let $p(x) \in \mathbb{F}[x]$. Then
+
+$$
+  \operatorname{Spec}(p(\mathrm{T})) = p(\operatorname{Spec}(\mathrm{T})) = \Set{p(\lambda) | \lambda\in\operatorname{Spec}(\mathrm{T})}
+$$
+
+<details>
+<summary>Proof</summary>
+
+If $\lambda$ is an eigenvalue of $\mathrm{T}$, then it can be shown that $p(\lambda)$ is an eigenvalue of $p(\mathrm{T})$. Hence, $p(\operatorname{Spec}(\mathrm{T})) \subseteq \operatorname{Spec}(p(\mathrm{T}))$. For the reverse inclusion, let $\lambda\in\operatorname{Spec}(p(\mathrm{T}))$, i.e.
+
+$$
+  (p(\mathrm{T}) - \lambda)\mathbf{v} = 0,\; \mathbf{v}\neq\mathbf{0}
+$$
+
+If
+
+$$
+  p(x) - \lambda = \prod_{i=1}^n (x-r_i)^{e_i}
+$$
+
+where $r_i \in\mathbb{F}$, then writing this as a product of (not necessarily distinct) linear factors, we have
+
+$$
+  (\mathrm{T} - r_1)\cdots(\mathrm{T} - r_1)\cdots(\mathrm{T} - r_n)\cdots(\mathrm{T} - r_n)\mathbf{v} = 0
+$$
+
+where $r_i \mathrm{I}$ is written $r_i$ for convenience. We can remove factors from the left end of this equation one by one until we arrive at an operator $\mathrm{S}$ (perhaps the identity) for which $\mathrm{S}\mathbf{v} \neq 0$ but $(\mathrm{T} - r_i\mathrm{I})\mathrm{S}\mathbf{v} = 0$. Then $\mathrm{S}\mathbf{v}$ is an eigenvector for $\mathrm{T}$ with eigenvalue $r_i$. However, since $p(r_i) - \lambda = 0$, it follows that $\lambda = p(r_i)\in p(\operatorname{Spec}(\mathrm{T}))$. Hence, $\operatorname{Spec}(p(\mathrm{T})) \subseteq p(\operatorname{Spec}(\mathrm{T}))$.
+</details>
+</MathBox>
+
+## Trace and determinant
+
+
+<MathBox title='Relation between eigenvalues and the determinant' boxType='proposition'>
+Let $\mathbf{A}\in\mathcal{M}_n (\mathbb{F})$ be an $n\times n$-matrix with characteristic polynomial $c_\mathbf{A} = x^n + \sum_{i=0}^{n-1}$ eigenvalues $\lambda_1,\dots,\lambda_n$. If $\mathbf{F}$ is algebraically closed, then $\det(\mathbf{A})$ is the constant term of $c_\mathbf{A}$ and the product of the eigenvalues of $\mathbf{A}$ 
+
+$$
+  \det(\mathbf{A}) = (-1)^n \prod_{i=1}^n
+$$
+
+<details>
+<summary>Proof</summary>
+
+Let $\mathbb{F}$ be algebraically closed and let $\mathbf{A}\in\mathcal{M}_n (\mathbb{F})$ have characteristic polynomial
+
+$$
+\begin{align*}
+  c_\mathbf{A}(x) =& x^n + \sum_{i=0}^{n-1} a_i x^i \\
+  =& \prod_{i=1}^n (x - \lambda_i)
+\end{align*}
+$$
+
+where $\lambda_i$ are the eigenvalues of $\mathbf{A}$. Then $c_\mathbf{A} (x) = \det(x\mathbf{I} - \mathbf{A})$ and setting $x = 0$ gives
+
+$$
+  \det(\mathbf{A}) = -c_0 = (-1)^{n-1} \prod_{i=1}^n \lambda_i
+$$
+
+Hence, if $\mathbf{F}$ is algebraically closed then, up to sign, $\det(\mathbf{A})$ is the constant term of $c_\mathbf{A}(x)$ and the product of the eigenvalues of $\mathbf{A}$, including multiplicity.
+</details>
+</MathBox>
+
+<MathBox title='Relation between eigenvalues and the trace' boxType='proposition'>
+Let $\mathbf{A}\in\mathcal{M}_n (\mathbb{F})$ be an $n\times n$-matrix with characteristic polynomial $c_\mathbf{A} = x^n + \sum_{i=0}^{n-1}$ eigenvalues $\lambda_1,\dots,\lambda_n$. If $\mathbf{F}$ is algebraically closed, then $operatorname{tr}(\mathbf{A})$ is the sum of eigenvalues of $\mathbf{A}$, including multiplicity, i.e.
+
+$$
+  \operatorname{tr}(\mathbf{A}) = -c_{n-1} = \sum_{i=1}^n \lambda_i 
+$$
+</MathBox>
+
+Recall that the coefficients of any polynomial
+
+$$
+\begin{align*}
+  p(x) =& x^n + \sum_{i=0}^{n-1} c_i x^i \\
+  =& \prod_{i=1}^n (x - \lambda_i)
+\end{align*}
+$$
+
+are the *elementary symmetric functions* of the roots
+
+$$
+\begin{align*}
+  c_{n-1} =& (-1)^1 \sum_i \lambda_i \\
+  c_{n-2} =& (-1)^2 \sum_{i < j} \lambda_i \lambda_j \\
+  c_{n-3} =& (-1)^3 \sum_{i < j < k} \lambda_i \lambda_j \lambda_k \\
+  \vdot& \\
+  c_0 =& (-1)^n \prod_{i=1}^n \lambda_i
+\end{align*}
+$$
+
+The most important elementary symmetric of the eigenvalues are the first and last ones.
+
+## Algebraic and geometric multiplicities
+
+<MathBox title='Algebraic and geometric multiplicity of eigenvalues' boxType='definition'>
+Let $\lambda$ be an eigenvalue of a linear operator $\mathrm{T}\in\mathcal{L}(V)$.
+1. The *algebraic multiplicity* of $\lambda$ is the multiplicity of $\lambda$ as a root of the characteristic polynomial $c_\mathrm{T}(x)$.
+2. The *geometric multiplicity* of $\lambda$ is the dimension of the eigenspace $E_\lambda$.
+</MathBox>
+
+<MathBox title='Relation between algebraic and geometric multiplicity of eigenvalues' boxType='proposition'>
+The geometric multiplicity of an eigenvalue $\lambda$ of $\mathrm{T}\in\mathcal{L}(V)$ is less than or equal to its algebraic multiplicity.
+
+<details>
+<summary>Proof</summary>
+
+We can extend any basis $B_1 = \Set{\mathbf{v}_i}_{i=1}^k$ of $E_\lambda$ to a basis $B$ for $V$. Since $E_\lambda$ is invariant under $\mathrm{T}$, the matrix of $\mathrm{T}$ with respect to $B$ has the block form
+
+$$
+  [\mathrm{T}]_B = \begin{bmatrix} \lambda \mathbf{I}_k & \mathbf{A} \\ \mathbf{0} & \mathbf{B} \end{bmatrix}
+$$
+
+where $\mathbf{A}$ and $\mathbf{B}$ are matrices of the appropriate sized and so
+
+$$
+\begin{align*}
+  c_\mathbf{T} (x) =& \det(x\mathbf{I} - [\mathrm{T}]_B) \\
+  =& \det(x\mathbf{I}_k - \lambda\mathbf{I}_k)\det(x\mathbf{I}_{n-k} - \mathbf{B}) \\
+  =& (x - \lambda)^k \det(x\mathbf{I}_{n-k} - \mathbf{B})
+\end{align*}
+$$
+
+Hence, the algebraic multiplicity of $\lambda$ is at least equal to the geometric multiplicity $k$ of $\mathrm{T}$.
+</details>
+</MathBox>
+
 # Inner product space
 
 <MathBox title='Inner product' boxType='definition'>
 Let $V$ be an $\mathbb{F}$-vector space. The map $\langle\cdot, \cdot\rangle: V\times V \to\mathbb{F}$ is called an inner product on $V$ if it satisfies for all $\boldsymbol{u},\boldsymbol{v}, \boldsymbol{w}\in V$ and $\alpha, \beta \in\mathbb{F}$
 
 1. **Positive definiteness:** $\langle \boldsymbol{v}, \boldsymbol{v} \rangle \geq 0$ and $\langle \boldsymbol{v}, \boldsymbol{v} \rangle = 0 \implies \boldsymbol{v} = 0$
-2. **Linearity in the second argument**: $\langle\boldsymbol{u}, \alpha\boldsymbol{v} + \beta\boldsymbol{w}\rangle = \alpha\langle\boldsymbol{u}, \boldsymbol{v}\rangle + \beta\langle\boldsymbol{u}, \boldsymbol{w}\rangle$
+2. **Linearity in the second argument:** $\langle\boldsymbol{u}, \alpha\boldsymbol{v} + \beta\boldsymbol{w}\rangle = \alpha\langle\boldsymbol{u}, \boldsymbol{v}\rangle + \beta\langle\boldsymbol{u}, \boldsymbol{w}\rangle$
 3. **Conjugate symmetry:** $\langle \boldsymbol{v}, \boldsymbol{w}\rangle = \overline{\langle\boldsymbol{w}, \boldsymbol{v}\rangle}$
 </MathBox>