Krylov subspace

In linear algebra, the order-r Krylov subspace generated by an n-by-n matrix A and a vector b of dimension n is the linear subspace spanned by the images of b under the first r powers of A (starting from $A^{0}=I$ ), that is,^[1]

{\mathcal {K}}_{r}(A,b)=\operatorname {span} \,\{b,Ab,A^{2}b,\ldots ,A^{r-1}b\}.

Background[]

The concept is named after Russian applied mathematician and naval engineer Alexei Krylov, who published a paper about it in 1931.^[2]

Properties[]

${\mathcal {K}}_{r}(A,b),A{\mathcal {K}}_{r}(A,b)\subset {\mathcal {K}}_{r+1}(A,b)$ .
Vectors $\{b,Ab,A^{2}b,\ldots ,A^{r-1}b\}$ are linearly independent until $r<r_{0}$ , and ${\mathcal {K}}_{r}(A,b)\subset {\mathcal {K}}_{r_{0}}(A,b)$ . Thus, $r_{0}$ denotes the maximal dimension of a Krylov subspace.
The maximal dimension satisfies $r_{0}\leq 1+\operatorname {rank} A$ and $r_{0}\leq n+1$ .
More exactly, $r_{0}\leq \deg[p(A)]$ , where $p(A)$ is the minimal polynomial of $A$ . Furthermore, there exists a $b$ such that $r_{0}=\deg[p(A)]$ .
${\mathcal {K}}_{r}(A,b)$ is a cyclic submodule generated by $b$ of the torsion $k[x]$ -module $(k^{n})^{A}$ , where $k^{n}$ is the linear space on $k$ .
$k^{n}$ can be decomposed as the direct sum of Krylov subspaces.

Use[]

Krylov subspaces are used in algorithms for finding approximate solutions to high-dimensional linear algebra problems.^[1] Many linear dynamical system tests in control theory, especially those related to controllability and observability, involve checking the rank of the Krylov subspace. These tests are equivalent to finding the span of the Grammians associated with the system/output maps so the uncontrollable and unobservable subspaces are simply the orthogonal complement to the Krylov subspace.^[3]

Modern iterative methods such as Arnoldi iteration can be used for finding one (or a few) eigenvalues of large sparse matrices or solving large systems of linear equations. They try to avoid matrix-matrix operations, but rather multiply vectors by the matrix and work with the resulting vectors. Starting with a vector $b$ , one computes $Ab$ , then one multiplies that vector by $A$ to find $A^{2}b$ and so on. All algorithms that work this way are referred to as Krylov subspace methods; they are among the most successful methods currently available in numerical linear algebra.

Issues[]

Because the vectors usually soon become almost linearly dependent due to the properties of power iteration, methods relying on Krylov subspace frequently involve some orthogonalization scheme, such as Lanczos iteration for Hermitian matrices or Arnoldi iteration for more general matrices.

Existing methods[]

The best known Krylov subspace methods are the Arnoldi, Lanczos, Conjugate gradient, (Induced dimension reduction), GMRES (generalized minimum residual), BiCGSTAB (biconjugate gradient stabilized), (quasi minimal residual), (transpose-free QMR), and minimal residue methods.

References[]

^ ^a ^b Simoncini, Valeria (2015), "Krylov Subspaces", in Nicholas J. Higham; et al. (eds.), The Princeton Companion to Applied Mathematics, Princeton University Press, pp. 113–114
^ Krylov, A. N. (1931). "О численном решении уравнения, которым в технических вопросах определяются частоты малых колебаний материальных систем" [On the Numerical Solution of Equation by Which are Determined in Technical Problems the Frequencies of Small Vibrations of Material Systems]. Izvestiia Akademii nauk SSSR (in Russian). 7 (4): 491–539.
^ Hespanha, Joao (2017), Linear Systems Theory, Princeton University Press

v t Numerical linear algebra
Key concepts	Floating point Numerical stability
Problems	System of linear equations Matrix decompositions Matrix multiplication (algorithms) Matrix splitting Sparse problems
Hardware	CPU cache TLB Cache-oblivious algorithm SIMD Multiprocessing
Software	MATLAB Basic Linear Algebra Subprograms (BLAS) LAPACK Specialized libraries General purpose software