Times

This course was offered in Spring 2018 and Spring 2020. Given sufficient demand, I will offer it again in the Spring of 2022.

Description

This course provides students with the mathematical background needed to analyze and further develop numerical methods at the heart of deep learning. The course is simultaneously geared towards computer science students, who may be interested/experienced with deep learning and would like to strengthen their theoretical foundation, and mathematics students, who may have a background in numerical analysis and who are curious about the emerging technology of deep learning. The course primarily targets graduate students. However, exceptional advanced undergraduate students may be admitted on a case-by-case basis.

The ultimate goal is for students to implement a simple, mathematically sound deep learning code from scratch. This look under the hood will provide valuable insight into successful and well-tested software packages for deep learning tasks and understand their success (and failure). Students may use MATLAB, Julia, or Python.

Class time will be split between hands-on coding exercises, lectures providing the necessary theoretical background, and discussions about selected articles published in this emerging field. Classes will provide an overview of state-of-the-art methods for deep learning with a focus on deep residual networks, and also contain crash courses on some of the most important aspects of numerical analysis that are involved in deep learning, such as convex and nonlinear optimization, differential equations, and optimal control.

Prerequisites

In order to succeed in this class, students need to have a solid background in multivariate calculus and linear algebra and some programming experience in MATLAB, Julia, or Python. In addition, students are also expected to have experience or skills in either numerical analysis (optimization, partial differential equations) or machine learning (e.g., CS534, CS584, or similar). To get a stronger background on optimization, students are encouraged to enroll in MATH 347.