Lecture 17 Summary

Re-interpret Arnoldi and Lanczos as partial Hessenberg factorization. How that they correspond to AQ_n=Q_nH_n+h_n+1,nq_n+1e_n* for Arnoldi and AQ_n=Q_nT_n+h_n+1,nq_n+1e_n* for Lanczos, where H_n and T_n are upper Hessenberg and tridiagonal matrices respectively and e_n is the unit vector in the n-th direction.

Discussed how rounding problems cause a loss of orthogonality in Lanczos, leading to "ghost" eigenvalues where extremal eigenvalues re-appear. In Arnoldi, we explicitly store and orthogonalize against all q_j vectors, but then another problem arises: this becomes more and more expensive as n increases. The solution to this is restarting schemes, where we go for n steps and then restart with the k "best" eigenvectors. For k=1 this is easy, but explained why it is nontrivial for k>1: we need to restart in such a way that maintains the Lanczos (or Arnoldi) property that the residual AQ_n - Q_nH_n is nonzero only in the n-th column (and that column is orthogonal to Q_n).

Discussed the implicitly restarted Lanczos method, which does n-k steps of shifted QR to reduce the problem from n to k dimensions. The key thing is that, because the Q matrices in QR on tridiagonal matrices are upper Hessenberg, their product can be shown to preserve the Lanczos property of the residual for the first k columns.