Diagonalization and symmetric matrices

Let's suppose we have a matrix  that has  eigenvectors. We put these vectors into a matrix X that is invertible and multiply the two matrices. This gives us the following:

We know from  that when dealing with matrices, this becomes , where  and each x_i has a unique λ_i. Therefore, .

Let's move on to symmetric matrices. These are special matrices that, when transposed, are the same as the original, implying that  and for all , . This may seem rather trivial, but its implications are rather strong.

The spectral theorem states that if a matrix  is a symmetric matrix, then there exists an orthonormal basis for , which contains the eigenvectors of A.

This theorem is important to us because it allows us to factorize symmetric matrices. We call this spectral decomposition (also sometimes referred to as Eigendecomposition).

Suppose we have an orthogonal matrix Q, with the orthonormal basis of eigenvectors  and  being the matrix with corresponding eigenvalues.

From earlier, we know that  for all ; therefore, we have the following:

Note: Λ comes after Q because it is a diagonal matrix, and the s need to multiply the individual columns of Q.

By multiplying both sides by Q^T, we get the following result: