Linear Algebra for Programmers with Python in 21 Days

Build it, then check it against somebody else's.

Every time you call a solver, fit a curve, rotate a model, reduce a dataset or rank a graph, something is factoring a matrix below you. This book is about what that something does, why it does it that way, and how to tell when it has quietly failed.

The method is the same every day: build the routine yourself in plain Python, then check it against LAPACK. Twenty-one sittings leave you with a working library and, more usefully, with a reliable sense of which numerical results deserve your trust.

What you build

One library, matkit, grown across all twenty-one days. By Day 21 it solves a linear
system, factors a matrix four ways, fits a curve to noisy data, finds eigenvalues, computes a
singular value decomposition, compresses an image, and solves a large sparse system
iteratively.

The twenty-one days

• Day 1: What Linear Algebra Actually Computes


• Day 2: Vectors in Code


• Day 3: Matrices as Transformations


• Day 4: Matrix Multiplication and What It Costs


• Day 5: Solving Ax = b by Elimination


• Day 6: Pivoting, Failure, and the First Warning Signs


• Day 7: LU Factorization


• Day 8: Inverses, Determinants, and Why You Rarely Want Either


• Day 9: Vector Spaces, Span, and Independence


• Day 10: The Four Fundamental Subspaces


• Day 11: Norms, Conditioning, and What Floating Point Does


• Day 12: Orthogonality and Projection


• Day 13: The QR Factorization


• Day 14: Least Squares


• Day 15: Eigenvalues and Eigenvectors


• Day 16: Diagonalization, Powers, and Markov Chains


• Day 17: Symmetric Matrices, the Spectral Theorem, and Cholesky


• Day 18: How a Real Eigenvalue Solver Works


• Day 19: The Singular Value Decomposition


• Day 20: Low-Rank Approximation


• Day 21: Sparse Matrices, Iterative Solvers, and the Capstone

How it is written

• Every output block is the exact text a program printed. A tool runs each program, captures
  what it printed, and a second tool proves the block in the book is a byte-for-byte copy.


• Nothing is asserted that could have been measured. Where the text says one method beats
  another, a program below it runs both and prints how much.


• matkit never calls its own oracle: no routine in it calls numpy.linalg or scipy.linalg.
  Those exist to disagree with it, and four of them do.


• Failures are printed. Four routines disagreed with LAPACK and all four disagreements are in
  the book with their causes beside them.

Who it is for

You can write Python and you have used NumPy at least once. You either never took a linear
algebra course, or you took one and it left you able to invert a 3 x 3 matrix by hand and unable
to say what that was for. You do not need to remember any of it.

396 pages. 128 programs. 233 figures.