All Seminars

Title: Galerkin Transformer
Seminar: Numerical Analysis and Scientific Computing
Speaker: Shuhao Cao of Washington University in St. Louis
Contact: Yuanzhe Xi, yuanzhe.xi@emory.edu
Date: 2021-10-15 at 12:30PM
Venue: https://emory.zoom.us/j/94914933211
Abstract:
Transformer in "Attention Is All You Need" is now THE ubiquitous architecture in every state-of-the-art model in Natural Language Processing (NLP), such as BERT. At its heart and soul is the "attention mechanism". We apply the attention mechanism the first time to a data-driven operator learning problem related to partial differential equations. Inspired by Fourier Neural Operator which showed a state-of-the-art performance in parametric PDE evaluation, an effort is put together to explain the heuristics of, and to improve the efficacy of the attention mechanism. It is demonstrated that the widely-accepted "indispensable" softmax normalization in the scaled dot-product attention is sufficient but not necessary. Without the softmax normalization, the approximation capacity of a linearized Transformer variant can be proved rigorously for the first time to be on par with a Petrov-Galerkin projection layer-wise. Some simple changes mimicking projections in Hilbert spaces are applied to the attention mechanism, and it helps the final model achieve remarkable accuracy in operator learning tasks with unnormalized data. The newly proposed simple attention-based operator learner, Galerkin Transformer, surpasses the evaluation accuracy of the classical Transformer applied directly by 100 times, and betters all other models in concurrent research. In all experiments including the viscid Burgers' equation, an interface Darcy flow, an inverse interface coefficient identification problem, and Navier-Stokes flow in the turbulent regime, Galerkin Transformer shows significant improvements in both speed and evaluation accuracy over its softmax-normalized counterparts and other linearizing variants such as Random Feature Attention (Deepmind) or FAVOR+ in Performer (Google Brain). In traditional NLP benchmark problems such as IWSLT 14 De-En, the Galerkin projection-inspired tweaks in the attention-based encoder layers help the classic Transformer reach the baseline BLEU score much faster.
Title: PDE Models of Infectious Disease: Validation Against Data, Time-Delay Formulations, Data-Driven Methods, and Future Directions
Seminar: Numerical Analysis and Scientific Computing
Speaker: Alex Viguerie of Gran Sasso Science Institute
Contact: Alessandro Veneziani, avenez2@emory.edu
Date: 2021-10-08 at 12:30PM
Venue: MSC W201
Abstract:
In the wake of the COVID-19 epidemic, there has been surge in the interest of mathematical modeling of infectious disease. Most of these models are based on the classical SIR framework and follow a compartmental-type structure. While the majority of such models are based on systems of ordinary differential equations (ODEs), there have been several recent works using partial differential equation (PDE) formulations, in order to describe epidemic spread across both space and time. This talk will focus on the application of such PDE models, and discuss different PDE formulations, the advantages and disadvantages, and assess their performance against measured data. Emphasis is placed on the incorporation of time-delay formulations and the application of modern data-driven techniques to further inform and enhance the performance of such models.
Title: Turán density of cliques of order five in 3-uniform hypergraphs with quasirandom links, Part 2
Seminar: Combinatorics
Speaker: Mathias Schacht of The University of Hamburg and Yale University
Contact: Dwight Duffus, dwightduffus@emory.edu
Date: 2021-10-08 at 3:00PM
Venue: MSC E408
Abstract:
We continue with the proof that 3-uniform hypergraphs with the property that all vertices have a quasirandom link graph with density bigger than 1/3 contain a clique on five vertices. This time we focus on the structure of holes in reduced hypergraphs, which leads to a restricted problem that is easier to solve.
Title: Geometric equations for matroid varieties
Seminar: Algebra and Number Theory
Speaker: Ashley Wheeler of Georgia Institute of Technology
Contact: David Zureick-Brown, dzureic@emory.edu
Date: 2021-10-05 at 4:00PM
Venue: MSC W301
Abstract:
Each point $x$ in $Gr(r, n)$ corresponds to an $r \times n$ matrix $A_x$ which gives rise to a matroid $M_x$ on its columns. Gel'fand, Goresky, MacPherson, and Serganova showed that the sets $\{y \in Gr(r, n)\,|\,M_y = M_x\}$ form a stratification of $Gr(r, n)$ with many beautiful properties. However, results of Mn\"ev and Sturmfels show that these strata can be quite complicated, and in particular may have arbitrary singularities. We study the ideals $I_x$ of matroid varieties, the Zariski closures of these strata. We construct several classes of examples based on theorems from projective geometry and describe how the Grassmann--Cayley algebra may be used to derive non-trivial elements of $I_x$ geometrically when the combinatorics of the matroid is sufficiently rich.
Title: Inference, Computation, and Games
Seminar: Numerical Analysis and Scientific Computing
Speaker: Florian Schaefer of Georgia Institute of Technology
Contact: Yuanzhe Xi, yxi26@emory.edu
Date: 2021-09-24 at 12:30PM
Venue: MSC W201
Abstract:

In this talk, we develop algorithms for numerical computation, based on ideas from competitive games and statistical inference.

In the first part, we propose competitive gradient descent (CGD) as a natural generalization of gradient descent to saddle point problems and general sum games. Whereas gradient descent minimizes a local linear approximation at each step, CGD uses the Nash equilibrium of a local bilinear approximation. Explicitly accounting for agent-interaction significantly improves the convergence properties, as demonstrated in applications to GANs, reinforcement learning, and computer graphics.

In the second part, we show that the conditional near-independence properties of smooth Gaussian processes imply the near-sparsity of Cholesky factors of their dense covariance matrices. We use this insight to derive simple, fast solvers with state-of-the-art complexity vs. accuracy guarantees for general elliptic differential- and integral equations. Our methods come with rigorous error estimates, are easy to parallelize, and show good performance in practice.

Title: Clusters and semistable models of hyperelliptic curves
Seminar: Algebra and Number Theory
Speaker: Jeffrey Yelton of Emory University
Contact: David Zureick-Brown, dzureic@emory.edu
Date: 2021-09-21 at 4:00PM
Venue: MSC W301
Abstract:
For every hyperelliptic curve $C$ given by an equation of the form $y^2 = f(x)$ over a discretely-valued field of mixed characteristic $(0, p)$, there exists (after possibly extending the ground field) a model of $C$ which is \emph{semistable} -- that is, a model whose special fiber (i.e. the reduction over the residue field) consists of reduced components and has at worst very mild singularities. When $p$ is not $2$, the structure of such a special fiber is determined entirely by the distances (under the discrete valuation) between the roots of $f$, which we call the \emph{cluster data} associated to $f$. When $p = 2$, however, the cluster data no longer tell the whole story about the components of the special fiber of a semistable model of $C$, and constructing a semistable model becomes much more complicated. I will give an overview of how to constructnice" semistable models for hyperelliptic curves over residue characteristic not $2$ and then describe recent results (from joint work with Leonardo Fiore) on semistable models in the residue characteristic $2$ situation.
Title: Steklov-Poincaré analysis of the basic three-domain stent problem
Seminar: Numerical Analysis and Scientific Computing
Speaker: Irving Martinez of Emory University
Contact: Yuanzhe Xi, yxi26@emory.edu
Date: 2021-09-17 at 12:30PM
Venue: MSC W201
Abstract:
The Steklov-Poincaré problem was previously considered in the artery lumen and wall setting with a single interface. Here the analysis is expanded to incorporate solute behavior in the presence of a fixed-volume, solid, simple stent. In this geometry, a third domain is added to the two-domain structure of artery wall and lumen. Through this intersecting domain volume setting there are three interfaces: lumen-wall, stent-lumen, and wall-stent. Steady-state incompressible Navier-Stokes equations are used to explain the behavior of blood through the lumen, while advection-diffusion dynamics are considered for the solute mechanics across the lumen, wall, and stent. Having a fixed blood velocity value, Steklov-Poincaré decomposition of the advection-diffusion equations is applied locally to each of the interfaces. To unify these instances on a global scale, their overall intersection is explored in a smaller manifold, reducing the problem to one previously solved by Quarteroni, Veneziani, and Zunino. Through finite element analysis (FEM), the solution is discretized and found to be convergent. Finally, computational simulations with one, three, and five stent rings, placed between the volumes of inner and outer cylindrical meshes, were performed using NGSolve, confirming the convergence of the solution and its relation to the coarseness of the mesh.
Title: Turán density of cliques of order five in 3-uniform hypergraphs with quasirandom links
Seminar: Combinatorics
Speaker: Mathias Schacht of The University of Hamburg and Yale University
Contact: Dwight Duffus, dwightduffus@emory.edu
Date: 2021-09-17 at 3:00PM
Venue: MSC E408
Abstract:
We show that 3-uniform hypergraphs with the property that all vertices have a quasirandom link graph with density bigger than 1/3 contain a clique on five vertices. This result is asymptotically best possible. This is joint work with S. Berger, S. Piga, Chr. Reiher and V. R$\ddot{\mathrm{o}}$dl.
Title: The Interplay of Curvature and Control in Dynamical Systems
Seminar: Numerical Analysis and Scientific Computing
Speaker: Romeil Sandhu of Strony Brook University
Contact: Lars Ruthotto, lruthotto@emory.edu
Date: 2021-09-10 at 12:30PM
Venue: N301
Abstract:
This talk will focus on recent advances in geometry and control with a specific emphasis on how curvature (a measure of “flatness” in Riemannian geometry) is intimately tied to rate functions with applications in areas of inverse problems in autonomy, systems biology, to seemingly disparate areas in economics. In the first part of this talk, we will motivate our discussion with a broader thematic result due to Lott, Villani, and Sturm whereby one form of curvature, namely Ricci curvature, is intimately connected to Boltzmann entropy. In turn, we reexamine the open problem of developing Ricci curvature over discrete metric spaces and how such advances that leverage coarse geometry can be employed to exploit (network) functionality. For example, by placing a probability structure on a graph as opposed to dealing directly with the discrete space, the graph can be treated as a Riemannian manifold for which there exists a richness of tools and advantages that will be discussed. Other (combinatorial) discretizations will be introduced and their use in the context of control towards biological systems. From this and through the lens of Riemannian geometry, we will then pivot towards inverse problems in imaging for the second half of the talk. Here, we will show that stability of classical 3D shape inversion from a 2D scene is intimately tied to a form of curvature. Time permitting, we will close with a few problems in economics. Such disparate applications are presented with the intent to highlight the richness of interplay between Riemannian geometry and control and as such, this talk is designed to be accessible to a general audience with an interest in any of the above domains with a general interest in dynamical systems.

Romeil Sandhu is currently an Assistant Professor at Stony Brook University with appointments in Biomedical Informatics and Applied Mathematics & Statistics Departments. He is the recipient of the AFOSR YIP Award for work on 2D3D feedback control and machine learning for autonomous systems and NSF CAREER Award for work on geometric optimization of time-varying networks. Romeil first received his B.S. and M.S, and Ph.D. degrees from the Georgia Institute of Technology. His research interest focuses on the broad area of applied geometry, topology, & control towards the understanding of faltering autonomous agents in an unknown environment where ambiguity often arises.
Title: Angle ranks of Abelian varieties
Seminar: Number Theory
Speaker: David Zureick-Brown of Emory University
Contact: David Zureick-Brown, dzureic@emory.edu
Date: 2021-09-07 at 4:00PM
Venue: MSC W301
Abstract:
The speaker will discuss an elementary notion -- the angle rank of a polynomial -- and how this relates to the Tate conjecture for Abelian varieties over finite fields.