Matrix Decompositions
Matrix decompositions factor matrices into simpler, structured pieces. They enable efficient and stable solutions to linear systems, least squares, eigenproblems, and compression.
Key Points
- LU decomposition solves linear systems by triangular substitution.
- QR decomposition is numerically stable for least squares.
- SVD reveals rank, principal components, and best low-rank approximations.
Formulas
LU
$$A = LU$$
QR
$$A = QR$$
SVD
$$A = U\Sigma V^\top$$
Eckart-Young-Mirsky
$$\min_{\operatorname{rank}(B) \le k} \|A-B\|_2 = \sigma_{k+1}(A)$$
Code Example
import numpy as np
A = np.random.randn(5, 3)
U, s, Vt = np.linalg.svd(A)
# Best rank-2 approximation
A2 = U[:, :2] @ np.diag(s[:2]) @ Vt[:2, :]
print(np.linalg.norm(A - A2, 2)) # ~ sigma_3