Numerical Linear Algebra for HPC
Numerical linear algebra provides the computational kernels for scientific computing, optimization, and machine learning. Efficiency, stability, and parallelism are critical at scale.
Key Points
- BLAS and LAPACK provide standardized high-performance kernels.
- Iterative methods (CG, GMRES) are essential for large sparse systems.
- Matrix-free and low-rank approximations reduce memory and time complexity.
Formulas
Conjugate gradient
$$\text{For } A \succ 0, \text{ solve } Ax = b \text{ in at most } n \text{ steps.}$$
Krylov subspace
$$\mathcal{K}_k(A, b) = \operatorname{span}\{b, Ab, A^2b, \dots, A^{k-1}b\}$$
Conditioning
$$\kappa(A) = \sigma_{\max}(A)/\sigma_{\min}(A)$$
Code Example
from scipy.sparse import diags
from scipy.sparse.linalg import cg
n = 1000
A = diags([-1, 2, -1], [-1, 0, 1], shape=(n, n))
b = np.ones(n)
x, info = cg(A, b)
print(info)