Deep Neural Networks for Large-scale Complex Spatial and Spatio-temporal Processes

Abstract

Environmental statistics play a critical role in various interconnected domains, encompassing weather and climate forecasting, air quality monitoring, and sustainable urban planning. However, because of their high inherent unpredictability and nonstationarity, modeling complex spatio-temporal dynamics of environmental processes is challenging. This dissertation develops a set of DNN based methods for large-scale spatial and spatio-temporal processes.

This dissertation firstly focuses on large-scale bivariate spatial interpolation. The proposed method is called Biv.DeepKriging, which is a spatially dependent deep neural network (DNN) with an embedding layer constructed by spatial radial basis functions for bivariate spatial data prediction. It is shown that the commonly used  Linear Model of Coregionalization (LMC) for bivariate spatial processes can be viewed as a special case of the proposed Biv.DeepKriging. Additionally, a distribution-free uncertainty quantification method based on bootstrap and ensemble DNN has been developed to provide prediction uncertainties. The proposed approach outperforms the traditional cokriging predictor with commonly used covariance functions, such as the linear model of co-regionalization and parsimonious bivariate Matérn covariance with 20 times faster computational scalability. 

The second focus of this dissertation is the development of a distribution-free spatio-temporal architecture for forecasting non-Gaussian and nonstationary processes. The proposed two stage framework performs interpolation in the first step with the utilization of a dependent DNN structure via basis functions. The second stage forecasts at a future time point for a particular location using a Long Short-Term Memory (LSTM) network. Quantile-based probabilistic forecasting is utilized for uncertainty quantification.

The third contribution of this dissertation is a deep learning-based scalable approach for the nonstationary Matérn covariance function estimation. In this work, the capabilities of the Convolutional Neural Network (CNN) has been exploited to obtain most effective partitions of the nonstationary field to construct the subregion based spatially varying parameters for the covariance function. Additionally, a parallel high-performance implementation of the nonstationary modeling and predictions on most recent hardware architectures, including shared memory, GPUs, and distributed memory systems is developed. This combined framework shows huge improvement in the spatially varying parameter estimation for the nonstationary covariance function over traditional fixed-splitting approaches. 

Finally, this dissertation establishes an innovative deep learning-based approach for modeling nonstationary spatial processes via domain deformation to emulate stationary isotropic covariance structures. This study tackles previous challenges associated with domain deformation techniques, including structural rigidity and the limited applicability of deformation functions to more than two dimensions. It achieves this through the utilization of neural autoregressive flows (NAFs) as spatial warping functions. The concept of warping functions is expanded beyond the constraints of one and two-dimensional spatial processes, highlighting the potential for more versatile and generic invertible functions. Encouraging initial outcomes are showcased with the employment of NAFs as spatial warping functions.

Brief Biography

Pratik Nag, currently pursuing his Ph.D. in the Statistics department within CEMSE, specializes in the exploration of intricate spatio-temporal phenomena. His research endeavors involve remedying the limitations of conventional statistical methods through the application of deep learning techniques. Prior to his tenure at KAUST, Pratik completed his master's degree at the Indian Statistical Institute.