Matrices, rank, eigenvalues and eigenvectors notes — Unit 1

In school, matrices felt like a tedious way to write numbers in brackets. In engineering, a matrix is essentially a machine: it takes a vector as an input, transforms it (rotates, stretches, or shears it), and outputs a new vector. Every topic in this unit—Rank, Eigenvalues, Cayley-Hamilton—is just a different way of asking 'how does this specific machine behave?'

There are exactly three legal moves you can make on a matrix that preserve its 'Rank' (its amount of useful information). These are called elementary row (or column) operations.

• 1. Interchange: Swapping two rows (R_i ↔ R_j).
• 2. Scaling: Multiplying a row by a non-zero scalar (R_i → k * R_i).
• 3. Addition: Adding a scalar multiple of one row to another (R_i → R_i + k * R_j).

[ 1  2  3 ]           [ 1       2       3   ]
[ 2  5  7 ]   ====>   [ 2-2(1)  5-2(2)  7-2(3)]
[ 3  1  2 ]           [ 3       1       2   ]

Result:
[ 1  2  3 ]
[ 0  1  1 ]
[ 3  1  2 ]

Imagine you have 3 equations, but one of them is just the first equation multiplied by 2. That third equation contains zero new information. The 'Rank' of a matrix is the number of rows that actually contain unique, independent information.

Finding the rank by calculating determinants (minors) is a nightmare for a 4x4 matrix, because you'd have to check sixteen 3x3 determinants. Instead, we use systematic reduction methods: Echelon Form and Normal Form.

A matrix is in row-echelon form if it looks like a staircase of zeros from the bottom left to the top right.

Find the rank of the following 3x4 matrix using Echelon form:

A = [  1   2   3   4 ]
    [  2   4   6   8 ]
    [  3   6   9  12 ]

1. 1.Step 1: The pivot a11 is already 1. We need to make the '2' and '3' below it into zeros.
2. 2.Operation 1: R2 → R2 - 2R1
3. 3.Operation 2: R3 → R3 - 3R1

R2: [ 2-2(1)  4-2(2)  6-2(3)  8-2(4) ]  => [ 0  0  0  0 ]
R3: [ 3-3(1)  6-3(2)  9-3(3) 12-3(4) ]  => [ 0  0  0  0 ]

Matrix becomes:
    [  1   2   3   4 ]
    [  0   0   0   0 ]
    [  0   0   0   0 ]

The reduction finished surprisingly fast. Let's analyze the result:

    [  1   2   3   4 ]  <-- Non-zero row
    [  0   0   0   0 ]  <-- Zero row
    [  0   0   0   0 ]  <-- Zero row

The matrix is now in strict row-echelon form. All zero rows are at the bottom. We simply count the number of non-zero rows.

Normal form is the ultimate reduction. Instead of stopping at a staircase, we destroy EVERYTHING except a tiny identity matrix in the top-left corner. To do this, we are allowed to use BOTH row and column operations.

Reduce to normal form and find rank:

A = [  1   2  -1   4 ]
    [  2   4   3   5 ]
    [ -1  -2   6  -7 ]

1. 1.Pivot is a11 = 1. First, zero out the column below it using Row Ops.
2. 2.R2 → R2 - 2R1
3. 3.R3 → R3 + R1

After Row Ops:
    [  1   2  -1   4 ]
    [  0   0   5  -3 ]
    [  0   0   5  -3 ]

Now, zero out the row to the right of a11 using Column Ops.

1. 1.C2 → C2 - 2C1
2. 2.C3 → C3 + C1
3. 3.C4 → C4 - 4C1

Notice how C1 remains unchanged, and the first row becomes zeros:
    [  1   0   0   0 ]
    [  0   0   5  -3 ]
    [  0   0   5  -3 ]

Next step: Move to a22. But wait, a22 is 0! We must swap columns to bring a non-zero number there. Let's swap C2 and C3 (C2 ↔ C3).

After C2 ↔ C3:
    [  1   0   0   0 ]
    [  0   5   0  -3 ]
    [  0   5   0  -3 ]

Now a22 is 5. We need to zero out the element below it (R3) using a row op.

1. 1.R3 → R3 - R2
2. 2.This wipes out the entire third row!

    [  1   0   0   0 ]
    [  0   5   0  -3 ]
    [  0   0   0   0 ]

Finally, zero out the -3 to the right of the 5 using a column op, and divide C2 by 5.

1. 1.C4 → C4 + (3/5)C2
2. 2.C2 → C2 / 5

    [  1   0 | 0   0 ]
    [  0   1 | 0   0 ]
    ------------------
    [  0   0 | 0   0 ]

Why did we learn rank? To solve massive systems of equations. A system of equations (AX = B) is 'consistent' if it has at least one valid solution. We test this using Rouche's Theorem.

⇒Rouche's Theorem

Create the Augmented Matrix [A|B] by pasting the constants column B onto the end of A. Reduce [A|B] to echelon form, and compare the rank of A to the rank of the whole [A|B].

If all constants in B are zero, the system is Homogeneous. A homogeneous system is ALWAYS consistent! Why? Because x=0, y=0, z=0 will ALWAYS satisfy equations equal to 0. This is called the 'Trivial Solution'.

⇒When do non-trivial solutions exist?

We only care when there are OTHER solutions besides all zeros. This happens if the rank is strictly less than the number of variables (ρ(A) < n).

Q: Find 'k' such that the system has non-trivial solutions.
Method: Instantly write |A| = 0. Plug in the matrix, solve the cubic/quadratic for k. Done in 2 minutes.

If a matrix A represents a transformation machine, an 'Eigenvector' (X) is a special arrow that doesn't get knocked off its course when it goes through the machine. It only gets stretched or shrunk. The amount it stretches is the 'Eigenvalue' (λ).

⇒The Core Equation

AX = λX
=> AX - λX = 0
=> (A - λI)X = 0

This is exactly a Homogeneous System (AX = 0). For X to be a 'special arrow', it cannot be the zero vector (trivial solution). Based on the Golden Rule from the previous page, for a non-trivial X to exist, the determinant of the coefficient matrix must be zero!

Expanding a 3x3 determinant containing (a11 - λ) is an algebra nightmare. You will drop a minus sign and fail the 15-mark question. Use the trace-determinant shortcut instead.

S2 = Minor(a11) + Minor(a22) + Minor(a33)
Minor(a11) = (a22*a33 - a23*a32)
This takes 30 seconds and avoids all expanding errors.

Find the eigenvalues for the matrix A:

A = [  8  -6   2 ]
    [ -6   7  -4 ]
    [  2  -4   3 ]

1. 1.1. S1 (Trace) = 8 + 7 + 3 = 18
2. 2.2. S2 (Minors) = |7 -4; -4 3| + |8 2; 2 3| + |8 -6; -6 7|
3. 3. S2 = (21 - 16) + (24 - 4) + (56 - 36) = 5 + 20 + 20 = 45
4. 4.3. S3 (Det A) = 8(21-16) - (-6)(-18-8) + 2(24-14) ... wait.
5. 5. Let's do it carefully: 8(5) + 6(-18+8) + 2(24-14) = 40 + 6(-10) + 2(10) = 40 - 60 + 20 = 0.

The characteristic equation is λ³ - 18λ² + 45λ - 0 = 0.

λ(λ² - 18λ + 45) = 0
λ(λ - 3)(λ - 15) = 0
Eigenvalues: λ = 0, 3, 15

To find the eigenvector X for a specific λ, we plug λ back into (A - λI)X = 0. Because |A - λI| = 0, the three equations will be dependent (one is redundant). We pick any TWO distinct equations and use Cramer's rule (cross-multiplication).

Continuing the previous example, let's find the eigenvector for λ = 0.

(A - 0I)X = 0
[  8  -6   2 ] [x]   [0]
[ -6   7  -4 ] [y] = [0]
[  2  -4   3 ] [z]   [0]

1. 1.Pick the first two equations:
2. 2.1) 8x - 6y + 2z = 0
3. 3.2) -6x + 7y - 4z = 0
4. 4.Apply cross-multiplication:
5. 5.x / ((-6)(-4) - (2)(7)) = -y / ((8)(-4) - (2)(-6)) = z / ((8)(7) - (-6)(-6))
6. 6.x / (24 - 14) = -y / (-32 + 12) = z / (56 - 36)
7. 7.x / 10 = -y / -20 = z / 20
8. 8.x / 1 = y / 2 = z / 2 (Simplifying the ratios)

The Cayley-Hamilton theorem states a beautiful truth: 'Every square matrix satisfies its own characteristic equation.' If you replace the scalar λ with the matrix A, the polynomial equals the zero matrix.

This theorem is incredibly powerful for two tasks in exams: finding the inverse without using cofactors, and evaluating massive powers of A (like A^8) without doing 8 matrix multiplications.

Assume we proved A³ - 6A² + 11A - 6I = 0. Find A⁻¹.

1. 1.Step 1: Multiply the entire equation by A⁻¹.
2. 2.A⁻¹(A³ - 6A² + 11A - 6I) = A⁻¹(0)
3. 3.A² - 6A + 11I - 6A⁻¹ = 0 (Notice A * A⁻¹ becomes I)
4. 4.Step 2: Isolate the term containing A⁻¹.
5. 5.6A⁻¹ = A² - 6A + 11I
6. 6.A⁻¹ = 1/6 * [A² - 6A + 11I]

Instead of calculating 9 cofactors, transposing, and dividing by |A|,
we only have to calculate A² (one multiplication), do basic addition,
and divide by 6. It cuts calculation time in half.

If a matrix A has distinct eigenvectors, we can transform it into a Diagonal Matrix (D). A diagonal matrix is perfectly clean: its eigenvalues sit on the main diagonal, and everything else is zero. Raising D to the power of 100 just means raising those three diagonal numbers to the 100th power!

⇒The Process (P⁻¹AP = D)

1. 1.1. Find eigenvalues (λ1, λ2, λ3).
2. 2.2. Find eigenvectors (X1, X2, X3).
3. 3.3. Construct the Modal Matrix P by pasting the eigenvectors side-by-side as columns: P = [ X1 | X2 | X3 ].
4. 4.4. Calculate P⁻¹.
5. 5.5. The matrix product P⁻¹ A P will magically collapse into a diagonal matrix D, where the diagonal elements are exactly λ1, λ2, and λ3.