Covariance
Table of Contents
1. Gram Matrix
\(X^TX\) is called Gram Matrix or Gramian. It is the matrix of all possible inner products between all parsi of column vectors of the matrix. It is useful to determine linear independence and also used in kernel methods like SVM.
2. Scatter Matrix
When \(X\) is centered and gramian is taken we get scatter matrix. i.e.
\begin{align*} (X - \mu)^T (X - \mu) \end{align*}is called scatter matrix
3. Covariance Matrix
If the scatter matrix is normalized then it is covriance matrix:
\begin{align*} \frac 1 {n - 1} (X - \mu)^T(X - \mu) \end{align*}The eigenvectors of Covriance matrix give the principle components (direction of maximum variance) of the data. This is the basis of PCA.