Deep neural networks that “get” spatial dependence

About

Deep neural networks (DNNs) can improve on interpolation for estimates made in climate science and related fields. The key, KAUST researchers have found, lies in tweaking them so that they can better apprehend spatial dependence.

Until now, the most commonly used method for prediction in geospatial statistics has been “kriging,” a technique that uses the weighted average of values of a variable taken at a cluster of locations to estimate its value at a point nearby. Weights are assigned by a covariance function, describing the relationship between the likely value of a variable — known at a given point — and the distance from that point.

Kriging’s power is limited by its simplifying assumptions: that the estimated variable is distributed normally and that its variance across space is constant.

To take fine particle pollution (PM2.5) as an example, concentrations are generally right-skewed and variance will differ between variables such as land and sea. Kriging using a high number of data points also requires huge computational power.

DNNs make no such assumptions and use fewer calculations to pick out patterns from data, but they also have failings. Most importantly, DNNs lack a common-sense form of reasoning, sometimes called the “First Law of Geography,” that states that things close together are more alike than things far apart. In short, DNNs do not “get” spatial dependence.

“A DNN will automatically learn from data, but it will not explicitly express spatial dependence as Kriging does. We will not be sure how it captures spatial dependence, so we can’t see if its conclusion is reasonable and we can’t interpret the results,” says researcher Chen Wanfang.

DeepKriging, as the KAUST team have called their new method, in effect predisposes a DNN toward thinking in terms of spatial dependence by using the Karhunen–Loève theorem of stochastic processes to select basis functions that are integrated into its input layer.

“The basis functions transform inputted coordinates from Euclidean to a higher dimensional space in which it is easier for the DNN to grasp their spatial relationship,” says researcher Li.

To test DeepKriging, the researchers used a pollution map of the U.S. of PM2.5 data collected at 841 monitoring stations. When it came to predicting values at stations for which readings were removed, they found their new method performed significantly better than either Kriging or DNNs: up to ten times as fast as Kriging, computation-wise, with a mean standard error that was, in some cases, less than half. “DeepKriging is highly scalable and suitable for large-scale spatial prediction,” adds Ying Sun, who led the research. 

For more exciting KAUST research stories visit KAUST Discovery. 

References

1. Chen, W., Li, Y., Reich, B.J. & Sun, Y. DeepKriging: Spatially dependent deep neural networks for spatial prediction.Statistica Sinica Preprint No: SS-2021-0277 (2022).| article