Abstract
GPT, Stable Diffusion, AlphaFold 2, etc., all these state-of-the-art deep learning models use a neural architecture called "Transformer". Since the emergence of "Attention Is All You Need", Transformer is now the ubiquitous architecture in deep learning. At Transformer's heart and soul is the "attention mechanism". In this talk, we shall give a specific example the following research program: whether and how one can benefit from the theoretical structure of a mathematical problem to develop task-oriented and structure-conforming deep neural networks? An attention-based deep direct sampling method is proposed for solving Electrical Impedance Tomography (EIT), a class of boundary value inverse problems. Progresses within different communities will be briefed to answer some open problems on the mathematical properties of the attention mechanism in Transformers.
Brief Biography
Prof. Shuhao Cao is an Assistant Professor at the Division of Computing, Analytics, and Mathematics, University of Missouri-Kansas City. He earned his doctoral degree from Purdue University in 2014 under the supervision of Prof. Zhiqiang Cai. His research interests encompass scientific computing and data-driven methods for partial differential equations, including Finite Element Methods, Multilevel Methods, Transformer Neural Networks, and the development of open-source software packages.