Quasi-Newton Methods for Image Restoration

J. Nagy and K. Palmer

Many iterative methods that are used to solve $A\bfx=\bfb$ can be derived as quasi-Newton methods for minimizing the quadratic function $\frac{1}{2}\bfx^TA^TA\bfx-\bfx^TA^T\bfb$. In this paper, several such methods are considered, including conjugate gradient least squares (CGLS), Barzilai-Borwein (BB), residual norm steepest descent (RNSD) and Landweber (LW). Regularization properties of these methods are studied by analyzing the so-called "filter factors". The algorithm proposed by Barzilai and Borwein is shown to have very favorable regularization and convergence properties. Secondly, we find that preconditioning can result in much better convergence properties for these iterative methods.