Enhance Neural Network Training for Complex PDEs: Algorithms, Architectures, and Integration with Traditional Methods

Overview

Neural networks have emerged as powerful tools for solving partial differential equations (PDEs), with applications spanning physics, chemistry, biology, and beyond. Based on our research, neural networks demonstrate superior approximation capabilities compared to traditional methods like finite element approaches and can overcome the curse of dimensionality. Moreover, they can approximate operators, enabling efficient PDE solutions without retraining when equation parameters change. However, training neural networks to effectively approximate such functions and operators presents significant challenges due to the inherently non-convex nature of the problem, especially for complex PDEs in biology and engineering. Two critical factors influencing performance are the design of training algorithms and the architecture of neural networks. In this talk, I will present several recent works to tackle these challenges by improving training algorithms, redesigning neural network architectures, and integrating traditional numerical methods. First, we introduce homotopy training algorithms, which help escape local minima and achieve faster convergence. Next, we explore strategies to optimize neural network architectures and refine loss functions by leveraging insights into training dynamics, thereby enhancing overall performance. Finally, we propose combining traditional numerical methods with operator learning to solve complex PDEs with multiple solutions more effectively.