Linear Algebra Difficulty: Advanced

Tensors and Multilinear Algebra

Tensors generalize scalars, vectors, and matrices to multiway arrays. Tensor decompositions and contractions are used in deep learning, physics, and data analysis.

Key Points

  • Tensors are multilinear maps from vector spaces to the scalar field.
  • Tensor contractions generalize matrix multiplication and traces.
  • CP and Tucker decompositions compress high-dimensional data.

Formulas

Tensor product
$$(a \otimes b)_{ij} = a_i b_j$$
Contraction
$$C_{ij} = \sum_{k} A_{ikj} B_{k}$$
CP decomposition
$$\mathcal{X} \approx \sum_{r=1}^R \lambda_r \mathbf{a}_r \circ \mathbf{b}_r \circ \mathbf{c}_r$$

Code Example

import numpy as np

A = np.random.rand(3, 4, 5)
B = np.random.rand(5)
# Contract over the last mode
C = np.tensordot(A, B, axes=([2], [0]))
print(C.shape)  # (3, 4)

Tags

  • multiway
  • tensor-decomposition

References

  • Tensor Decompositions and Applications
    Tamara G. Kolda and Brett W. Bader · SIAM Review · source
  • Multilinear Algebra
    Werner H. Greub · Springer · source

Knowledge Graph