Discriminant analysis: Fundamental limits and Optimal designs

Overview

Abstract

Despite the recent advances in big data processing, enabled by the emergence of large-scale machine learning techniques, several statistical questions regarding the behavior in the regime of high dimensions of well-established and fundamental methods have remained unresolved. Of these methods is the class of discriminant analysis techniques which includes linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA) classifiers as popular instances. These methods are optimal in the sense that they allow for achieving the lowest misclassification error rate, if the assumptions under which they have been derived are satisfied, that is Gaussian data with common covariances for linear discriminant analysis, and Gaussian data with different covariances for quadratic discriminant analysis. In practice, however, the covariances and means associated with each class are not known, and are often estimated using a set of observations with known labels, referred to as in the machine learning terminology as training data. The resulting classifiers obtained by substituting the unknown means and covariances by their estimates are known to lead to poor performances, even under the assumption of Gaussian data. As far as theoretical analysis is concerned, it is a curious observation to discover that very little has been thus far done to understand the behavior of LDA and QDA in practical scenarios. This talk aims to provide an overview of the most important contributions related to the understanding of the major problems of using discriminant analysis classifiers in the regime of high-dimensions. On the operational level, we propose improved LDA and QDA based classifier that yield impressive performances under some settings.

Brief Biography

Abla Kammoun (Member, IEEE) was born in Sfax, Tunisia. She received the engineering degree in signal and systems from Tunisia Polytechnic School, La Marsa, and the Master’s and Ph.D. degrees in digital communications from Telecom Paris Tech (formerly, École Nationale Supérieure des Télécommunications). From 2010 to 2012, she was a Postdoctoral Researcher with the TSI Department, Telecom Paris Tech. She was then with Supelec, Alcatel-Lucent Chair on Flexible Radio until 2013. She is currently a ResearchScientist with KAUST. Her research interests include performance analysis of wireless communication systems, random matrix theory, and statistical signal processing.