MATH Seminar

Title: Regression with Tropical Rational Functions and Application to Neural Network Initialization
Seminar: CODES@emory
Speaker: Alex Dunbar of Emory University
Contact: Lars Ruthotto, lruthotto@emory.edu
Date: 2022-11-03 at 10:00AM
Venue: MSC W301
Download Flyer
Abstract: The tropical semiring and its associated polynomial and rational functions provide an algebraic framework for understanding continuous piecewise linear functions. We propose an alternating minimization heuristic for regression over the space of tropical rational functions. The method alternates between fitting the numerator and denominator via tropical polynomial regression, which is known to admit a closed-form solution. Our work is motivated by applications to ReLU neural network training. ReLU neural networks are a popular class of network architectures in the machine learning community which have recently been connected to tropical rational functions. We present experiments demonstrating the behavior of the alternating minimization method. Additionally, we present preliminary experiments which leverage tropical rational regression to initialize weights in ReLU neural networks and discuss geometric aspects of the network initialization problem.