Linear Algebra Basics 线性代数基础

[solomonxie]

It's so much better to extract the basic knowledge from Precalculus in here.

Prerequisites

• Precalculus:
  • Trigonometry: Unit circle, Inverse trig function
  • Geometry: Transformation
  • Precalculus: Conversion of complex numbers
  • Algebra: Line equation
  • Algebra: Systems of equations
  • Vector: Dot product & Cross product

Study resources

• Wiki: List of linear algebra topics
• Slant: What are the best resources to learn linear algebra?
• Pre Linear Algebra notes
• Khan academy Pre-calculus
• Essence of linear algebra: Youtube
• Mathispower4u
• MIT OCW 18.06 SC Linear Algebra

Tools

• MIT Mathlets
  • MATRIX VECTOR

Practice & Quizzes

• Symbolab (Matrices & Vectors Practice)
• Wolframalpha (Problem Generator: Linear Algebra)
• Uni of Sydney (Quizzes for Linear algebra Math1002)
• Yu Tsumura (Linear Algebra Problems and Solutions)

Study goals of Linear Algebra

• Systems of linear equations
• Row Reduction Echelon Forms (RREF)
• Matrix operations, including inverses
• Block matrices
• Linear dependence and independence
• Subspaces and bases and dimensions
• Orthogonal bases and orthogonal projections
• Gram-Schmidt process
• Linear models and least-squares problems
• Determinants and their properties
• Cramer's Rule
• Eigenvalues and eigenvectors
• Diagonalization of a matrix
• Symmetric matrices
• Positive definite matrices
• Similar matrices
• Linear transformations
• Singular Value Decomposition

[solomonxie]

❖ Vector Intro

What is a 「vector」

Definition: vector is a MAGNITUDE with a DIRECTION

Notation:

• v as a vector.
• |v| or ||v|| as its Magnitude, or Length, or Distance, or Absolute value, same idea
• Slope or angle as its Direction.
• (a, b) the two position there are called X-component & Y-component.

It's not hard to understand the basic ideas of a vector, which consists of very basic knowledges form what we learned previously in high school:

• Magnitude of vector: is the same with calculating the distance of two points
• Direction of vector: is the same with calculating the slope of a line.

「Distance」 vs. 「Displacement」

• Distance is a scalar ("3 km")
• Displacement is a vector ("3 km Southeast")
  

「Speed」 vs 「Velocity」

• Speed is how fast something moves.
• Velocity is speed with a direction.

UNDERSTAND VECTOR'S 「MOVEMENT」

Now we got a vector start from (-3,8) to the point (4, 5).
We say this vector has a magnitude ||v|| = √40 and has a direction to the bottom right.
But we could also represent the vector as only one point with two components: (7, -3).
Which has a HIDDEN INFORMATION that it start from the origin and direction is to the point (7, -3).
Are they different vectors? NO! They're the same vector, just starts at different places.

Because vector doesn't specify where it starts. A vector only means A magnitude & a direction.

That's why we could represent a vector in two ways:

• v is (-3,8) to (4, 5)
• v = (7, -3)

IT'S VERY IMPORTANT TO UNDERSTAND THIS IDEA, SO THAT WE COULD FURTHER UNDERSTAND WHY WE COULD MOVE VECTORS, AND BY SO WE COULD DO SUMS AND MULTIPLICATIONS AND SO ON

「Adding」 & 「subtracting」 vectors

"Linear algebra is built on these operations v + w and cv: Adding vectors and multiplying by scalars." - Gilbert Strang

Adding & subtracting could be seen as a movement to a vector, or say how it travels.

Refer to Maths is fun.

• Adding vectors:
  

• Subtracting vectors: is just ADDING a NEGATIVE VECTOR
  

Understand vector's 「addition & subtraction」

Refer to Intro to Linear Algebra by Gilbert Strang: 1.1.

"You can't add apples and oranges."
But you can add fruits!
Imagine you have one bag of fruits (3 apples, 4 oranges), and another bag of fruits (1 apple, 2 oranges).
So adding them together you will get one big bag of fruits (4 apples, 6 oranges),
from this big bag you could also split a smaller bag of fruits, and then you will call it subtraction.

A vector is the very idea of a bag of fruits.

「Scalar Multiplication」

It's the same with the section DILATION or Scaling in Geometry.

Just for review of dilating a graph geometrically:
When you scale a graph by a factor:

• Every line of the graph scale by the SAME factor
• Every line will be PARALLEL to its origin line.

Refer to previous note: Transformation.

[solomonxie]

❖ Vector forms

Refer to Khan academy article.

In Gilbert Strang's Intro to Linear Algebra, it's different point of view:

For row form or column form:

• Row form: v = (1, 1, -1)
• Column form:
  

"The reason for the row form (in parentheses) is to save space. But v = (1,1, -1) is not a row vector! It is in actuality a column vector, just temporarily lying down. "

For representing method:

「Unit vectors」

Simply say, a unit vector is just a vector which length is 1. Kinda like the unit circle idea.
It's also called Engineering notation, or the basis vector.

Which means, it could lie on axis or in between.

What's it for?
Basically just like the unit circle, make things easier to calculate angles or length or so.
Actually it's working together with unit circle and all other trigonometric knowledges.
so,
UNIT VECTOR IS RATHER A TRIGONOMETRIC WAY TO DEAL WITH VECTORS.
Easier to think about it, is to think about the Similar graph knowledge in the Dilation section.

「Standard Basis Vectors」 & 「Unit vector」 form

It's also called Unit Basis Vectors.
Unit vector is easy, but unit vector form needs your bit more effort to understand.

THIS FORM DOESN'T PRESENT VECTOR AS A POSITION ANYMORE, RATHER PRESENT IT AS HOW MUCH IT STRETCHES UNIT VECTORS, OR SAY PRESENT IT AS A SCALAR.

Assume that there are TWO unit vectors, one vertical, one horizontal.

How to find a vector's 「unit vector」

More intuitively to solve it just to draw it out and solve it with trigonometry.

Example: Find the unit vector of vector (-8, 5)

Convert between 「vector forms」

HIGHLY RECOMMEND TO REVIEW THE COMPLEX NUMBER CONVERSION SKILLS, HAVING THE VERY SAME IDEA.
Refer to previous note: complex number forms conversion skills.

How to find the 「direction angle」 of a vector

It's the very same problem of the topic Inverse trig function.

NOT EASY! NEED TO CONSIDER LOTS OF CONDITIONS, LIKE QUADRANTS, POSSIBLE SOLUTIONS, PRINCIPLE SOLUTION AND SO. RECOMMEND TO REVIEW INVERSE TRIG FUNCTIONS.

Refer to previous notes: Inverse Trigonometric equations.

Example: Find the direction angle of vector (-2, 5) between 0° to 360°.
Solve:

• tanθ = (-5/2)
• θ₁ = -68.2
• θ₂ = 180° + -68.2 = 111.8 because θ₁ is negative could use this trig identity.
• Vector (-2, 5) is located on Quadrant-2,
• So the answer is 111.8.

[solomonxie]

Vector span

It's extending the unit vector idea.

Refer to famous visualisation of 3Blue1Brown's video: Linear combinations, span, and basis vectors

「R²」 and 「R³」

R² means a Real numbers 2D plane.
Usually the X/Y Axes plane is this one.

R³ means Real numbers 3D plane.
Usually the X/Y/Z Axes plane.

「Linear combinations」 (Vector Addition)

DEFINITION: The sum of cv and dw is a linear combination of v and w.

Linear combinations means to add vectors together: v₁ + v₂ + v₃..... to get a new vector. Simple like that.

「Span of vectors」

It's the Set of all the linear combinations of a number vectors.

# v, w are vectors
span(v, w) = R²

span(0) = 0

One vector with a scalar, no matter how much it stretches or shrinks, it ALWAYS on the same line, because the direction or slope is not changing. So ONE VECTOR'S SPAN IS A LINE.

Two vector with scalars, we then COULD change the slope! So that we could get to any position that we want in the 2D plane, i.e., R².

Exceptions:

• span(0) = 0, it only stay at origin.
• v = w, if two vectors are the same, or collinear, then it's still ONE vector.

Linearly 「dependence」 & 「independence」

• Linear dependence: two vectors are COLLINEAR, means on the same line.
• Linear independence: two vectors are NOT COLLINEAR, means they're not on the same line.

Vectors (2, 3) and (4, 6) are the SAME VECTOR!
Because (4,6) = 2*(2,3), so it's just a scaled version of the first vector.

So we say the vectors (2, 3) and (4, 6) are DEPENDENT, because they're COLLINEAR.

Other than that, any two vectors are INDEPENDENT, if they're not NOT COLLINEAR.

List of some 「linear combinations」

Let's list some vector combinations:

• Zero Vector: span(0) = 0.
• One vector: span(v) = a line.
• Two vector: span(v₁, v₂) = R², if they're not collinear.
• Three vector or more: span(v₁, v₂, v₃...) = R². Other than two vectors, are all REDUNDANT.
  In another word:
  IF ANY TWO VECTORS ARE INDEPENDENT, THEN OTHERS ARE ALL DEPENDENT.

How to calculate a 「linear combination's independency」

Refer to Adam Panagos: Linear Algebra Example Problems - Linearly Independent Vectors #1
Refer to TheTrevTutor: [Linear Algebra] Linear Independence and Bases
Refer to Khan lecture: Span and linear independence example

It's important for knowing if a linear combination can fill out a plane or space.
For example, if two vectors aren't independent, then it's just one vector, and can only draw a line. If three vectors aren't independent, then they're just two vectors, one is redundant, so they can only fill out a 2D plane instead of a 3D space.

A linear combination is independent, iff it could satisfy this equation:

c₁v₁ + c₂v₂ + c₃v₃ .... = 0

c.. means the scalar for each vector, and you could change the scalar to any number, positive or negative.
Note that: c ≠ 0, and vectors are not all zeros.

Assume that there's a linear combination of two vectors v₁ + v₂ + v₃,
with scalars it could be c₁v₁ + c₂v₂ + c₃v₃.
To verify whether it's dependent or independent,
we assume c₁v₁ + c₂v₂ + c₃v₃= (0,0,0) and solve for c₁, c₂, c₃:

• it's independent <=> if c₁ = c₂ = c₃ = 0 all are zeros
• it's dependent <=> If c₁, c₂, c₃ at least one is NON-ZERO number

Independent Example:

Dependent Example:

[solomonxie]

❖ Vectors multiplication

Vector multiplication is FUNDAMENTAL skill to solve Matrices multiplication.

THE VERY FIRST THING TO DO WITH A VECTOR MULTIPLICATION OR MATRIX MULTIPLICATION, IS TO FORGET EVERYTHING ABOUT ARITHMETIC MULTIPLICATION !!
OTHERWISE, YOU WILL FALL INTO AN ENDLESS CONFUSION!

Just to know, multiplication of vectors or matrices, AREN'T really multiplication, but just look like that. You can see them as operations to get SOMETHING.

There're two operations are called multiplication for vectors:

• Dot product: express as V₁ · V₂, named for the dot symbol. It's meant to get the Product of two magnitudes.
• Cross product: express as V₁ × V₂, named for the cross symbol. It's meant to get a new vector.

The cross product is very very very limited in use, and NOT as fairly often in use as the dot product. So don't waste time on this unless having certain use of it.

「Boosting」

IT'S THE VERY CORE SENSE OF MAKING A MULTIPLICATION OF VECTORS OR MATRICES.

Multiplication ISN'T just Repeat counting in Arithmetic anymore.
Not 4×3 = 4+4+4 anymore!

It's rather kind of Growth, or empowerment, or boosting.
We'd say we tripled 4, or say number 4 grow with speed of 3, or to say number 4 grows with a boosting of 3.
Whatever you'd say, you get the idea.
Multiplication a process of double, triple, quadruple ....

JUST TO REMEMBER: FORGET ABOUT ARITHMETIC MULTIPLICATION, ALWAYS SEE MULTIPLICATION AS BOOSTING.

「Dot Product」

REMEMBER: A DOT PRODUCT DOESN'T GIVE YOU A VECTOR, BUT ONLY A NUMBER, A SCALAR, A PRODUCT OF TWO MAGNITUDES.

The purpose:
It is NOT to get a new vector, and NOT to Reduce dimension,
its only purpose IS to get a quantity, a magnitude, a number!

For an intuitive video refer to Khan academy physics: Dot Product.
For more explains in detail: Vector Calculus: Understanding the Dot Product
Maths is fun: dot product.
3Blue1Brown: Dot products and duality | Essence of linear algebra

Understand Dot product in 「business」

Refer to Intro to linear algebra by Gilbert Strang: 1.2.

Understand Dot product in 「physics」

It makes lots more sense to think dot product in physics way than maths algebraic way.

Just to think Two forces "a & b" are pulling a box,
so how much power did it pulled on the direction of a, or how much on the direction of b?

Vectors on 「same direction」

Let's make it easier before digging in:
assume there's no angle, Two forces "a & b" are pulling to the same way, the same direction,
so how much power would it be pulled?

Well, the force a & b working together, it's a process of Boosting the energy!
It's not ADDING together anymore, it's BOOSTING!
Let's say the force a has 3 units power, b has 6 units power.
So every 1 unit power a pulls, b will pull 2 units power.
Then it make sense:
The total power pulling the thing would be 3 · 6 = 18 units

Vectors on 「different direction」

So the Two forces AREN'T pulling the box at the same direction anymore, how much power did it pulled on the direction of a, or how much on the direction of b?

Let's think about how much power it's pulling on the direction of b.
Since a is pulling on a bit wrong way, so a's power ISN'T 100% working on b's way.
How much power left there?
It depends on the angle.
So to calculate how much left, we use |a| × cos(θ),
and we got a PROJECTION or a reflection or a shadow of a on b!
Then it become like this picture again:

How amazing it is!
And now we could Boost the power on b: |b| × |a|×cosθ

Ways of calculating dot product

There're two ways to calculate the dot product (I made up the names):

• Shadow Boost:
  
• Axes Boost:
  

Result of two ways are SAME.

Remember: Boosting is not working when two vectors are Perpendicular, which product is 0.

「Shadow Boost」

We reflect one vector on another one, then Boost the energy.

Intuition:

「Axes Boost」

We break two vectors to X-axis and Y-axis, and BOOST on each axis.

Easier to remember the formula is:

Intuition:

Examples:

Example

「Dot product」 & 「Symmetry」

Dot product has a relationship with Symmetry.

Refer to lecture of Imperial College London: Einstein summation convention and the symmetry of the dot product

[solomonxie]

Cross product [DRAFT]

Actually the cross product is very very very limited in use, and as fairly often as the dot product.

So don't waste time on this unless having certain use of it.

REMEMBER: A CROSS PRODUCT ONLY HAS USE IN 3-DIMENSIONS, AND ORDER MATTERS, AND GIVES YOU A NEW VECTOR.

List it again:

• Only work for 3-Dimensions
• Order matters
• Gives a new vector

Right-handed system:

[solomonxie]

Matrix intro

Matrices are just a rectangle array of numbers.

Prerequisites: Systems of equations

Matrices could be seen as a group of informations arranged IN A CERTAIN WAY.
IT'S SO SO SO SO SO EASIER TO THINK IT AS A SYSTEM OF EQUATIONS.

「Matrix row operations」 & 「Systems of equations」

It's very SAME with operations of systems of equations.

Refer to Khan academy article: Matrix row operations

There're different types of row operations:

• Switch any two rows
• Multiply a row by a nonzero constant
• Add one row to another

They all relate to the operations of systems of equations:

Switching any two rows:

Multiply a row by a 「nonzero constant」

Add one row to another

Solve 「system equations」 using Matrix

it's also called the Row-Echelon form and Gaussian elimination.

Khan lecture: Reduced row echelon form
Refer to Ck-12: Row Operations and Row Echelon Forms
Example of "RREF": Lec 01 - Linear Algebra | Princeton University

It's a so serious problem in all the first lesson of Linear algebra courses. It seems simple yet not easy to solve by yourself. You need to understand all the steps of how to do a REF or RREF, aka. Reduce Row Echelon Form.

The important note to apply the RREF is to know how the Pivot, or the Cursor moves.
It's more efficient to understand it with 1 or 2 practice rather than see notes here.

First we need to rewrite the system of equation to matrix form:

Then by row operations, we need to achieve this kind of result, which is also called Reduced Row Echelon Form:

It means we eliminated all other variables and only left 1 variable in one equation, which is called Identity Matrix. Then you could put back the number to the system of equations.

[solomonxie]

❖ Matrix Multiplication

IT IS A WHOLE NEW AREA ASIDE FROM MATRICES BASIC OPERATIONS.

It's very difficult to make sense of it. But mathematicians just somehow make it work, it then is a Human defined operation, it makes no sense but you just have to deal with it.
If you just need to solve the problem, you only need 5 minutes to get it around, and then you can skip all these below.
But if you'd like to understand it, then prepare yourself for a couple of hours or days on this.

It's necessary to make you confused with the operation below. Because that's most teachers start with to teach you how to multiply matrices. Don't worry, we're to skip this one and find a better perspective to solve it.

Refer to Khan academy article: Multiplying matrices
Refer to 3Blue1Brown: Dot products and duality
Refer to maths is fun: How to Multiply Matrices

This topic is very easy to use but very difficult to understand!
But I rather to understand it instead of just memorising it.
So this is the Learning path of this topic:

1. Dot product
2. Linear transformations
3. Matrix-vector product
4. Matrix-matrix product

Understand Matrix multiplication

There're so many different ways to understand it, to make sense of it, because it's so hard to understand.

The major fond of ways to understand are:

• Through Matrix transformation
• Through Real-life example
• Through Algebraic methods

Although Real-life example makes sense easily, but it aren't gonna help solving problems well.
So Matrix transformation is the ultimate way to understand Matrix multiplications.
And it proved that it is the best way for that, and for all core ideas of Linear Algebra.

Before we start, let's make things clear:

• Dot product: Vector * Vector
• Matrix-vector product: Matrix * Vector
• Matrix-Matrix product: Matrix * Matrix

Refer to Khan lecture video: Matrix vector products as linear transformations

In the GPU of a computer,
"ALL THE GRAPHIC PROCESSORS ARE JUST HARD-WIRED MATRIX MULTIPLIERS! ALL THEY DO IS JUST MULTIPLYING MATRICES!" - SAL KHAN

Linear transformation

There're so many different ways to understand Matrix Multiplications,
and Linear Transformations is the best and probably the only one makes sense and perfect intuition to you, i.e. it might be the only chance to understand it after all.

LINEAR TRANSFORMATION IS THE VERY KEY TO OPEN UP ALL GETES IN LINEAR ALGEBRA, BECAUSE IT MAKES PERFECT SENSE OF MATRIX MULTIPLICATION.

To understand matrix multiplication, Linear Transformation is the very first thing you want to learn. It's fairly important and can't get away with.

Linear transformation is a special kind of Transformation, which deals with vectors.

Need to mention that, 3Blue1Brown has done well on build intuition on this topic:
Refer to 3Blue1Brown's video: Linear transformations and matrices
Refer to the same video: How does linear transformation work on unit vectors
Refer to 3Blue1Brown's video: Three-dimensional linear transformations
Refer to 3Blue1Brown's video: Matrix multiplication as composition

"Matrices give us a language to describe these transformations, where the columns represent those coordinates.
And matrix-vector multiplication is just a way to compute what that transformation does to a given vector.
Every time you see a matrix, you can interpret it as a certain transformation of space.
Once you digest the idea, you’re in a great position to understand the linear algebra deeply.
Almost all of the topics in linear algebra will become easy to understand once you start thinking about matrices as transformations of space." - 3Blue1Brown

YOU JUST HAVE TO MEMORISE THIS EQUATION AND GET THE IDEA. THAT IS GONNA HELP YOU OUT FROM ALL THE IDEAS AND PROBLEMS IN LINEAR ALGEBRA.

Change the 「basis」

Changing basis is the very core of Linear Transformation. Every single move is based on this.

Remember a vector (a, b) could also present in unit vector form as v = ai + bj,
and unit vectors are i = (1, 0) & j = (0, 1).

If we want to transform a vector, like move, flip, rotate, scale, the thing we'll do is:
TO CHANGE THE UNIT VECTOR i AND j.

For example, there's a vector v = (-1, 2), and it can present as v = -1i + 2j, then we're to do some movement to it:

• Move: We let unit vector i = (100, 0), then the vector moves to the right becomes (-100, 2).
• Rotate: we let unit vector i = (0, 1) and j = (-1, 0), then the vector rotates 90° becomes (-2, -1).
  THAT'S THE MAGIC!!

By telling where the unit vectors are to go, we can create a pattern, a mapping rule, so that every vector uses this map, this rule, this pattern will have the same transformation!

Another example:
Assume there's a vector v=(5,7), and let the unit vector i=(3,-2) and j=(2,1), and present this TRANSFORM PATTERN as below:

And we present this Applying a transformation to a vector in the form below:

SO WHENEVER YOU ENCOUNTER MATRIX MULTIPLICATION AGAIN, NEVER READ IT AS TWO VECTORS OR TWO MATRICES MULTIPLYING TOGETHER!

How to interpret a 「Matrix Multiplication」

There're only TWO part of this matrix multiplication:

• The Graph: the 1st on right item.
• The Rules: All the rest Matrices on the left of The Graph.

The Graph could be one point (vector) or many points (vectors), e.g.:

• A point: (2,3)
• A triangle: [ (3,0) (0,4) (3,4) ]
• A rectangle: [ (3,0) (3,4) (0,4) (0,0)]
• Any shape in any dimension.....

SO ALL YOU NEED TO DO, IS JUST TO APPLY THOSE RULES ONE BY ONE, LEFT BY RIGHT, AND GET A NEW GRAPH!!!

For example, we apply two transform rules to a vector (x, y):

It's exactly same with the function principles: Shear( Rotate(x, y) ).

Break up the Matrices with its 「Geometric」 meaning

In the transform rule as below:

WE HAVE TO BREAK THE MATRICES INTO SINGLE PARTS BEFORE WE DO THE CALCULATION.

And since we made the rule for i & j, so let's apply the unit vector to the Graph:

Note that:

• The original graph is v = 5i + 7j, so after applying the new rule of i & j, we get:
  v = 5(3,-2) + 7(2,1)
• And now we could do the Vector multiply a scalar method, to get this:
  v = (15,-10) + (14,7)
• Then we could do Add two vectors:
  v = (19, -3)
• So after applying the transformation rule, we successfully transformed the vector to a new position:
  (19, -3)

Example

Solve:

• It seems a Matrix Multiplication problem
• First we need to form the original matrix by identifying all vertices:
  
• Transform the matrix by multiplying the Coordinate Matrix:
  
• Draw it out by the result matrix

[solomonxie]

❖ Matrix Transformation

It's a subset of Linear transformation, just with higher dimension rules & multiple points graph multiplying together.

YOU BREAK THE RULE IN TO DIFFERENT UNIT VECTORS i, j, k... AND BREAK THE GRAPH INTO DIFFERENT POINTS, AND APPLY EACH i, j, k RULES TO EACH POINT.

「Size」 of Matrix multiplication

Unlike Vector multiplication gives you only a number, Matrix multiplication gives you another Matrix, but a SHRINK-SIZED Matrix.

In General:
To multiply an m×n matrix by an n×p matrix, the ns must be the same,
and the RESULT is an m×p matrix.

Example: 「3x2 Matrix」 with 「2x2 Matrix」

Solve:

• First we know it's a 3x2 Matrix multiply a 2x2 Matrix, it's valid, and the new Matrix's size would be 3x2.
• And then we multiply each by each according to their dimension:
  
• We get the answer:
  

Example: 2x2 Matrix with 2x3 Matrix

Solve:

• It's will get a 2x3 new Matrix (just for intuition), then we get the answer:
  

More examples

Refer to Symbolab the Online math solver, which offers answers of any matrices operation step by step.

Common Matrix Transformations

One single matrix to present the movement? Yes!

Refer to Math planet: Transformation using matrices

「Rotation」 of Shapes

▶ Use the tool: Desmos Matrix Calculator

Example: Transform Graphs

Solve:

• Just to list all points of this graph:
  (-4,8), (-8,-4), (-4,-4), (0,4)
• Arrange points to a Matrix:
  
• Apply the Matrix R to the Graph's matrix. Do the math.

「Reflection」 of Shapes

[solomonxie]

Composition of Matrix Multiplication

Composition of Matrix Multiplication means More than one linear transformations applies to a graph one by one.

Although you can see two matrices multiplying together as a transform applying to a graph.
But actually sometime you can see it in slightly different perspective.

Refer to 3Blue1Brown: Matrix multiplication as composition

So extend from the equation above, we know that the transformation rules could Unite together as one and then apply it to the graph.
And here it comes, MATRICES MULTIPLICATION COULD ALSO REPRESENT TRANSFORM RULES UNITE TOGETHER AS A RESULT RULE, RATHER THAN ONE RULE WORK ON A GRAPH.

Order matters!

[solomonxie]

❖ Determinant of Transformation

It's quite easy to calculate, and not too hard to understand what's behind it.

The Determinant of a transformation is How much the AREA of the new Graph scaled.

JUST TO REMEMBER: THE DETERMINANT IS ABOUT AREA OF THE GRAPH!

Refer to 3Blue1Brown: The determinant

「Unit vector」 graph

We all know the unit vector i & j made an area of 1.
But when we do a Linear transformation to the unit vector graph, the area is not 1 anymore, might be bigger or smaller.
So how much it re-sized we call it the determinant.

Note that:

• Since a/1 = a, so calculating the Area of the new unit vector graph, is equal to the scalar itself.
• Calculating how much the unit vector graph scaled, is exactly equal to how much the whole graph scaled.

Irregular shape

If it's not a grid square can be approximately very well by many many small piece of grid squares.

「Determinant formula」 for 2x2 Matrix

Refer to Khan video.

It will be so much easier if you just to memorise the formula, than to understand where it comes from, which is also not necessary to do.

「Determinant formula」 for 3x3 Matrix

I hope you're not gonna have chance to apply this formula.

"This (determinant) is both tricky to show and derive, and is kind of pointless. Knowing how to do the operations (of determinant) isn't a useful skill anymore because we just type det(A) into a computer. Thus I'll just type det(A) and my computer gives me the answer, done. From a learning perspective, it doesn't add much. So I'm not going to teach you how to do determinants. If you want to know, then look up a QR decomposition online, or better yet, look in a linear algebra textbook." - David Dye, Imperial College London

「Zero determinant」

If the determinant of a transformation det(M) = 0, then it means the Transformation squishes the graph to a line or a point!

「Negative determinant」

A negative determinant means the graph has been flipped over by the transformation.
Then the j unit vector flip over to the LEFT side of i unit vector.

Refer to 3Blue1Brown for visualisation

[solomonxie]

❖ Inverse Matrices

There is actually no concept of division of matrix. But similarly we can let it multiply an inverse to achieve the same goal.

Refer to 3Blue1Brown: Inverse matrices, column space and null space
Refer to maths is fun: Inverse of a Matrix.

SPOILER ALERT: EVEN 3x3 MATRIX INVERSE IS ALREADY TOO HEAVY TO CALCULATE, SO BETTER JUST TO MEMORISE THE 2x2 AND LET COMPUTER DO ALL THE HIGHER DIMENSIONS.

Understand the 「Inverse Matrix」

3Blue1Brown's video perfect explained the intuition of it, pretty much everything you need to know.
Link: Inverse matrices, column space and null space

It makes lots more sense in geometric meanings, that an Inverse Matrix just to RECOVER the transformation of a graph back to before.

Why is the 「Inverse Matrix」 at the Left of Vector

Because Matrix A is always as a "Coefficient" to the vector, or as a transformation rule to the vector, so it's always on the left of the vector ( or the graph).

「Identity Matrix」

It's a simple yet important notation for doing dividing a matrix.

The Identity Matrix is the matrix equals to the number of 1:

It's very much more intuitive to think a identity matrix as one unit vector.

• 1-Dimension: x = 1
• 2-Dimensions: v = (1, 1)
• 3-Dimensions: v = (1, 1, 1)

The features of 「Identity Matrix」

• It is square (m×m Matrix)
• It can be large or small (2×2, 3×3, 100×100, ... whatever)
• It has 1s on the diagonal and 0s everywhere else
• Its symbol is the capital letter 𝗜

More importantly, IT CAN SWITCH SIDE WHEN MULTIPLYING ANOTHER MATIRX!
It's very special, and is the ONLY matrix can IGNORE the order when multiplying another matrix.

「Not Invertible」 Matirces

Two conditions make a matrix NOT invertible:

• The matrix is not a Square Matrix (m×m matrix).
• The Determinant is ZERO. Such matrix is also called a Singular matrix

「Adjugate Matrix」

It's also called the Adjoint of a matrix, or Classical Adjoint.

Refer to maths is fun: Inverse of a Matrix using Minors, Cofactors and Adjugate.

「Adjugate」 of 2x2 Matrix

Calculate the 「Inverse」 of a 「Matrix」

"Calculating it for a 2x2 is fairly straightforward, 3x3 becomes a little bit hairy, 4x4 will take you all day, 5x5 you're almost definitely gonna do a careless mistake if you do an inverse of matrix." - Sal Khan

2x2 Matrix inverse

With a 2x2 matrix, you really don't need to think much and waste time on the full steps, just simply follow this formula 1/Determinant × Adjugate

Refer to Khan lecture.

3x3 Matrix inverse

We can calculate the Inverse of a Matrix by:

• Step 1: calculating the Matrix of Minors,
• Step 2: then turn that into the Matrix of Cofactors,
• Step 3: then the Adjugate, and
• Step 4: multiply that by 1/Determinant.

I tend not to note the full content here, because it's so useless in normal math life. Because it's way to hairy to calculate even with a 3x3 matrix. So just get the idea and let computer do the rest.

Example

Solve:

• The formula for Inverse Matrix is Adjugate(A) / Determinant(A).
• The Determinant of A is -3*-5 - 2*6 = 3
• The Adjugate of A is [(-5, -2), (-6, -3)]
• So the answer is :
  

Solving 「Systems of equations」 with 「Inverse Matrices」

Khan lecture: Solving linear systems with matrix equations

[solomonxie]

Rank of Transformation

Means the Dimension of output of a transformation.

Refer to 3Blue1Brown: Inverse matrices, column space and null space

After doing a transformation to a graph, it is:

• Rank 1: if the output of the graph is 1-Dimensional, which is a line.
• Rank 2: if the output of the graph is 2-Dimensional, which is a plane.
• Rank 3: .....

In the case of 2x2 Matrices, the Rank 2 is the best it can be.
In the case of 3x3 Matrices, the Rank 2 means it collapsed, or dimensional reduced.

When it's the highest rank the Matrix can be, we call it THE FULL RANK.

[solomonxie]

Column space of Matrix

Means The SET of all possible outputs a matrix.
In another word, it's THE SPAN OF THE COLUMNS OF THE MATRIX.

The columns of the matrix tell where the basis vectors land, like i & j.
And the span of all basis vectors gives you all possible outputs, which is the Column space.

[solomonxie]