Distributed AI: Optimal algorithms for distributed stochastic non-convex optimization

Overview

Abstract

In many emerging applications, it is of paramount interest to learn hidden parameters from data. For example, self-driving cars may use onboard cameras to identify pedestrians, highway lanes, or traffic signs in various light and weather conditions. Problems such as these can be framed as classification, regression, or risk minimization in general, at the heart of which lies machine learning and stochastic optimization. In many practical scenarios, distributed and decentralized learning methods are preferable as they benefit from a divide-and-conquer approach towards data at the expense of local (short-range) communication. In this talk, I will present our recent work that develops a novel algorithmic framework to address various aspects of distributed stochastic first-order optimization methods for non-convex problems. A major focus will be to characterize regimes where the distributed solutions outperform their centralized counterparts and lead to optimal convergence guarantees. Moreover, I will characterize certain desirable attributes of distributed methods in the context of linear speedup and network-independent convergence rates. Throughout the talk, I will demonstrate such key aspects of the proposed methods with the help of provable theoretical results and numerical experiments on real data.

Brief Biography

Usman A. Khan is an Associate Professor of Electrical and Computer Engineering (ECE) at Tufts University, USA, since September 2017. His research interests include signal processing, optimization and control, and machine learning. He has published extensively in these topics with more than 100 articles in journals and conference proceedings~and holds multiple patents. Recognition of his work includes the prestigious National Science Foundation (NSF) Career award, several NSF REU awards, an IEEE journal cover, three best student paper awards in IEEE conferences, and several news articles including two in IEEE spectrum. 

Dr. Khan joined Tufts as an Assistant Professor in 2011 and held a Visiting Professor position at KTH, Sweden, in Spring 2015. Prior to joining Tufts, he was a postdoc in the GRASP lab at the University of Pennsylvania. He received his B.S. degree in 2002 from University of Engineering and Technology, Pakistan, M.S. degree in 2004 from University of Wisconsin-Madison, USA, and Ph.D. degree in 2009 from Carnegie Mellon University, USA, all in ECE. Dr. Khan is an \textit{IEEE Senior Member} and is an elected full member of the \textit{Sensor Array and Multichannel Technical Committee} with the \textit{IEEE Signal Processing Society} since 2019, where he was an Associate member from 2010 to 2019. He was an elected full member of the \textit{IEEE Big Data Special Interest Group} from 2017 to 2019, and has served on the \textit{IEEE Young Professionals Committee} and on \textit{the IEEE Technical Activities Board}. He was an Editor of the \textit{IEEE Transactions on Smart Grid} from 2014 to 2017 and is currently an Associate Editor of the \textit{IEEE Control System Letters}, \textit{IEEE Transactions on Signal and Information Processing over Networks}, and \textit{IEEE Open Journal of Signal Processing}. He is the Chief Editor of the \textit{Proceedings of the IEEE} special issue on \textit{Optimization for Data-driven Learning and Control} and a Guest Associate Editor for \textit{IEEE Control System Letters} special issue on \textit{Learning and Control} both to appear in 2020. He is the Technical Area Chair for the Networks track in \textit{IEEE 2020 Asilomar Conference on Signals Systems and Computers}, has served on the TPCs of several IEEE conferences, and has organized/chaired several IEEE workshops and sessions.