hessQuik: Fast Hessian computation of composite functions

Abstract

hessQuik is a lightweight software library for fast computation of second-order derivatives (Hessians) of composite functions (that is, functions formed via compositions) with respect to their inputs. The core of hessQuik is the efficient computation of analytical Hessians with GPU acceleration. hessQuik is a PyTorch package that is user-friendly and easily extendable. The repository includes a variety of popular functions and layers, including residual layers and input convex layers, from which users can build complex models through composition. hessQuik layers are designed for ease of composition - users only need to select the layers and the package provides a convenient wrapper to compose the functions properly. Each layer provides two modes for derivative computation and the mode is automatically selected to maximize computational efficiency. hessQuik includes easy-access, illustrative tutorials on Google Colaboratory, reproducible experiments, and unit tests to verify implementations. hessQuik enables users to obtain valuable second-order information for their models simply and efficiently.

Publication
The Journal of Open Source Software