Pseudo Inverse

For the optimization problem \(\textrm{argmin}_x ||Ax - b||^2\), \(x = pinv(A)b\) is the solution. \(pinv\) being the psuedoinverse of the matrix \(A\). (from https://youtu.be/WGccIFf6MF8?t=357)

Left pseudo inverse (such that \(A^{\dagger} A = I\)) is given by the formula:

\begin{align*} A^{\dagger} = (A^TA)^{-1} A^T \end{align*}

Right pseudo inverse (such that \(A A^{\dagger} = I\)) is given by the formula:

\begin{align*} A^{\dagger} = A^T (AA^T)^{-1} \end{align*}

The above formulas are valid when \(A\) has full column/row rank and thus \(A^T A\) is invertible. Otherwise, the pseudo-inverse can be expressed interms of Singular Value Decomposition (SVD) as follows:

\(A^{\dagger} = V \Sigma^{\dagger} U^T\)