Linear Algebra and Differential Equations, Explained Simply

Part 1 used vectors and matrices as tools for moving shapes around. This post asks what they are in general, how to solve many equations at once, and what a matrix does to the directions it acts on. The second half uses calculus to describe things that change at a rate tied to their size: cooling coffee, decaying atoms, growing populations.

Linear algebra needs only the algebra from part 1. The differential equations section also uses derivatives. The proof habits from part 2 help, but you can read this post without them.

How to use this post

The layout is unchanged: the idea, how to do it, worked examples, the usual mistake, practice. Every number below was checked by running the calculation, but you will learn more by redoing each one on paper before reading the steps.

1. Vectors, combinations and independence

A vector is a list of numbers, such as (1, 3) in the plane or (1, 2, 3) in space. You can add two vectors and scale one by a number, and that is all linear algebra assumes.

Recall adding two vectors tip to tail:

A linear combination of u and v is any vector a·u + b·v. The span of some vectors is the set of all their linear combinations. Vectors are independent if none of them is a combination of the others, which means a·u + b·v = 0 only when a = 0 and b = 0.

How to do it

1. To check whether a vector w is a combination of u and v, write w = a·u + b·v and solve for a and b.
2. To check independence, ask whether one vector is a multiple of another (two vectors) or whether the equation above has a solution besides all zeros.

Example 1: 2·(1, 3) + 3·(0, 1)

• Scale: (2, 6) and (0, 3).
• Add: (2, 9).

Example 2: is (5, 7) a combination of (1, 1) and (1, −1)?

• Write a(1, 1) + b(1, −1) = (5, 7). That gives a + b = 5 and a − b = 7.
• Adding the two equations: 2a = 12, so a = 6, and then b = −1.
• Check: 6(1, 1) − (1, −1) = (5, 7). Yes.

Example 3: independence

• (1, 2) and (2, 4): the second is 2 times the first. They are dependent, and their span is a single line.
• (1, 0) and (0, 1): neither is a multiple of the other. They are independent and span the whole plane.

The usual mistake: assuming any two vectors in the plane span it. They do not when one is a multiple of the other. Two dependent vectors only reach a line.

2. Solving systems of equations

A system of linear equations asks for numbers that satisfy several equations at once. Elimination solves it by combining equations to remove unknowns one at a time.

How to do it

1. Use the first equation to remove the first unknown from all the equations below it.
2. Use the next equation to remove the next unknown from the ones below that.
3. When you reach a single unknown, solve it, then work back up.
4. The legal moves are: swap two equations, multiply an equation by a non-zero number, and add a multiple of one equation to another. Apply each move to the whole equation, both sides.

Example 1: x + 2y = 5 and 3x + 4y = 11

• Subtract 3 times the first equation from the second: (3x + 4y) − 3(x + 2y) = 11 − 15, which gives −2y = −4.
• So y = 2, and then x = 5 − 4 = 1.
• Check: 1 + 4 = 5 and 3 + 8 = 11.

Example 2: three unknowns

 x +  y + z = 6
2x −  y + z = 3
 x + 2y − z = 2

1. Subtract 2 times row 1 from row 2: −3y − z = −9.
2. Subtract row 1 from row 3: y − 2z = −4.
3. Add one third of the new row 2 to the new row 3. The y terms cancel and the z terms combine: −2z − z ÷ 3 = −(7 ÷ 3)z, and the right sides give −4 − 3 = −7. So −(7 ÷ 3)z = −7.
4. So z = 3. Back up: −3y − 3 = −9 gives y = 2. Then x + 2 + 3 = 6 gives x = 1.
5. Check all three: 1 + 2 + 3 = 6, 2 − 2 + 3 = 3, 1 + 4 − 3 = 2.

Example 3: three possible outcomes

• x + y = 1 and x + y = 2: no solution. The lines are parallel.
• x + y = 1 and 2x + 2y = 2: infinitely many. The second equation is the first in disguise, so both describe the same line.
• Anything else, like Example 1: exactly one solution.

The usual mistake: changing only the left side when you scale or combine equations. Whatever you do to one side, do to the other.

3. Matrices: products, determinants and inverses

A matrix is a grid of numbers. Multiplying a matrix by a vector gives a new vector, so a matrix acts as a machine on vectors. Multiplying two matrices gives the machine that does one, then the other.

How to do it

1. Matrix times matrix: each entry of the product is a row of the left matrix times a column of the right one (multiply pairs, then add).
2. Determinant of a 2 × 2 matrix with rows (a, b) and (c, d): a·d − b·c. Its size is the factor by which the matrix scales areas.
3. Inverse of a 2 × 2 matrix: if the determinant is not 0, swap a and d, flip the signs of b and c, and divide everything by the determinant. If the determinant is 0, there is no inverse.

Example 1: order matters. Take [[1, 2], [3, 4]] and [[0, 1], [1, 0]] (as rows).

• First times second: [[2, 1], [4, 3]]. The second matrix swapped the columns.
• Second times first: [[3, 4], [1, 2]]. Now it swapped the rows.
• The two answers differ, so for matrices A × B is not the same as B × A.

Example 2: determinants.

• [[3, 1], [2, 4]]: 3 × 4 − 1 × 2 = 10. Areas grow by a factor of 10.
• [[1, 2], [2, 4]]: 1 × 4 − 2 × 2 = 0. Everything is squashed onto a line, and that cannot be undone, so there is no inverse.

Example 3: an inverse, used to solve a system. Take A = [[2, 1], [5, 3]].

1. Determinant: 2 × 3 − 1 × 5 = 1.
2. Swap 2 and 3, flip the signs of 1 and 5, divide by 1: the inverse is [[3, −1], [−5, 2]].
3. Check: A times its inverse gives [[1, 0], [0, 1]], the identity matrix.
4. Now solve 2x + y = 4 and 5x + 3y = 11. Multiply the inverse by (4, 11): x = 3 × 4 − 11 = 1 and y = −5 × 4 + 2 × 11 = 2.
5. Check: 2 + 2 = 4 and 5 + 6 = 11.

Try it. Set the four numbers to 2, 0, 0, 2 and read the determinant. Then set them to 1, 2, 2, 4 and watch the letter collapse onto a line.

The usual mistake: treating matrices like numbers. You cannot rely on A × B = B × A, and you cannot "divide" by a matrix, only multiply by its inverse, and only when it has one.

4. Eigenvalues and eigenvectors

When a matrix A acts on most vectors it turns them. A few special vectors only get stretched or shrunk along their own line. Such a vector v is an eigenvector and the stretch factor λ is its eigenvalue:

A·v = λ·v, with v not the zero vector.

How to do it (2 × 2 matrices)

1. Find the eigenvalues from λ² − (trace)λ + (determinant) = 0. The trace is a + d, the sum of the diagonal.
2. For each eigenvalue, subtract it from the diagonal of A and solve (A − λI)v = 0. Any non-zero solution is an eigenvector.
3. Check each answer by multiplying A by v.

Example 1: A = [[4, 1], [2, 3]]

1. Trace 4 + 3 = 7 and determinant 12 − 2 = 10, so λ² − 7λ + 10 = 0. That factors as (λ − 5)(λ − 2) = 0, giving λ = 5 and λ = 2.
2. For λ = 5, A − 5I has rows (−1, 1) and (2, −2). The first row says −x + y = 0, so y = x. An eigenvector is (1, 1).
3. Check: A times (1, 1) is (5, 5), which is 5 times (1, 1).
4. For λ = 2, A − 2I has rows (2, 1) and (2, 1). The row says 2x + y = 0, so y = −2x. An eigenvector is (1, −2).
5. Check: A times (1, −2) is (2, −4), which is 2 times (1, −2).

Why this is useful: repeated multiplication. Applying A to an eigenvector n times just multiplies it by λⁿ. To compute A³ applied to (3, 0):

1. Write (3, 0) using the eigenvectors: (3, 0) = 2(1, 1) + 1(1, −2).
2. The first part gets multiplied by 5³ = 125 and the second by 2³ = 8.
3. Result: 2 × 125 × (1, 1) + 8 × (1, −2) = (250 + 8, 250 − 16) = (258, 234).

Multiplying the matrix by itself three times and applying it gives the same (258, 234), but this route scales to A¹⁰⁰ without doing 100 multiplications.

Where it is used. Principal component analysis in data science finds the main directions of variation in a data set as eigenvectors of a matrix built from the data.

The usual mistake: thinking an eigenvector is a single vector. Any non-zero multiple of an eigenvector is also one, so (2, 2) works as well as (1, 1). Another: expecting real eigenvalues every time. The matrix with rows (0, −1) and (1, 0) turns every vector by 90 degrees, so no direction survives. Its trace is 0 and its determinant is 1, giving λ² + 1 = 0, which has no real solution.

5. Differential equations

A differential equation links a function to its own derivative. Its solution is a function, not a number. The simplest and most useful one says that something changes at a rate proportional to its current size:

dy/dx = k·y

Whenever the answer is "how much is there after a while", this is the equation. Money with compound interest, bacteria, decaying atoms and cooling drinks all follow it, possibly with a shift.

How to do it (separation of variables)

1. Move the y terms to one side and the x terms to the other: dy ÷ y = k dx.
2. Integrate both sides: ln y = kx + c, where c is the constant of integration.
3. Raise e to both sides: y = C·e^(kx), where C is a new constant.
4. Use the starting value to find C. At x = 0 the formula gives y = C, so C is the starting value.
5. Check by differentiating. The derivative of C·e^(kx) is k·C·e^(kx), which is k·y.

Example 1: dy/dx = −2y with y = 5 at x = 0

• The solution is y = 5e^(−2x).
• At x = 1: y = 5e^(−2), about 0.677. The quantity shrinks fast.

Example 2: radioactive decay. Take the half-life of carbon-14 as about 5,730 years. The amount N follows dN/dt = −k·N.

1. After one half-life the amount halves: e^(−k × 5730) = 1/2.
2. Take logarithms: k = ln 2 ÷ 5730, about 0.000121 per year.
3. Check: starting from 100 g, one half-life later you have 100 × e^(−ln 2) = 50 g.

Example 3: Newton's law of cooling. A coffee starts at 90 °C in a 20 °C room. Cooling is faster when the gap is larger: dT/dt = −k(T − 20). Suppose k = 0.1 per minute (an assumed value).

1. Let u = T − 20. Then du/dt = −0.1u, the equation from before, and u starts at 70.
2. So u = 70e^(−0.1t) and T = 20 + 70e^(−0.1t).
3. After 10 minutes: T = 20 + 70e^(−1), about 45.8 °C.
4. When does it reach 40 °C? 70e^(−0.1t) = 20 gives e^(−0.1t) = 2/7, so t = 10 × ln 3.5, about 12.5 minutes.

Example 4: dy/dx = 2xy with y = 3 at x = 0

1. Separate: dy ÷ y = 2x dx.
2. Integrate: ln y = x² + c, so y = C·e^(x²), and C = 3.
3. Check: the derivative of 3e^(x²) is 3e^(x²) × 2x, which is 2x·y. It works.

When there is no formula: Euler's method. Many equations cannot be solved exactly. A computer follows the slope in small steps instead. For dy/dx = y with y = 1 at x = 0, going to x = 1:

With two steps, each new y is the old y plus 0.5 times the old y: 1, then 1.5, then 2.25. Smaller steps follow the curve more closely.

The usual mistake: forgetting the starting value, which leaves an unknown constant, or skipping the check. Differentiating your answer takes seconds and catches most errors.

The mistakes in one table

Where to go next

The next post in the series is real analysis, which goes back to calculus and proves why it works. For more on this one, the MIT OpenCourseWare course 18.06 Linear Algebra, taught by Gilbert Strang, offers video lectures, problem sets with solutions, and exams with solutions, free under a Creative Commons license. Khan Academy also has courses on linear algebra and differential equations.

Sources and further reading

• MIT OpenCourseWare, 18.06 Linear Algebra (Gilbert Strang).
• Gilbert Strang, Introduction to Linear Algebra, the course text.
• Sheldon Axler, Linear Algebra Done Right, a proof-based alternative for a second pass.
• Khan Academy, courses on linear algebra and differential equations.