Checking your assumptions

About

A three-step model checking workflow has the potential to revolutionize how researchers evaluate the suitability of their statistical models for specific datasets. Developed by KAUST, the workflow is a computationally efficient alternative to existing model diagnostics that will give researchers more confidence in model selection[1].

“When we build statistical models to analyze complex datasets, such as climate or biomedical data, we inevitably make assumptions about the underlying statistical distributions, dependencies, and how things change over space and time,” says Rafael Cabral from the research team. “But two crucial questions are often overlooked: ‘Are my model assumptions reasonable?’ and ‘If I change these assumptions, how much would that affect the final results such as predictions or decisions?’ We have developed a principles-based way to answer those questions.”

Modern statistical models often contain multiple layers and include hidden components that cannot be directly observed. Existing model-checking tools focus only on the top observable layer, and whether the model has a reasonable fit to the observed data. However, fit alone does not reveal whether the underlying structural assumptions – such as a ‘bell-curve’ Gaussian versus a skewed non-Gaussian distribution – are correct for the given dataset.

“When a model is flawed, several problems can arise,” says Cabral. “For instance, if we incorrectly assume a Gaussian distribution for spatial models when in fact the data is non-Gaussian, we might smooth away important local patterns, which could significantly affect our conclusions and produce poor predictions.”

Read the full story on KAUST Discovery.