Graduate Seminars

back Back to all Graduate Seminars

CS Graduate Seminar| A new look at stochastic variance reduced gradient methods

Start Date: November 27, 2017
End Date: November 27, 2017

By Professor Robert M. Gower (Telecom ParisTech)
 
I will present a new perspective on stochastic variance reduced (SVR) methods as methods that maintain an estimate of the Jacobian of an auxiliary vector valued function. This auxiliary vector valued function is formed by stacking the individual data functions from the empirical risk minimization problem. Through this observation we extend the class of SVR methods by updating the Jacobian estimate using randomized sparse sketches of the true Jacobian. By choosing different randomized sketches we recover several known and new methods, all of which converge linearly, as dictated by a single convergence theorem. When specialized to known methods, our convergence theorem recovers the best known convergence results for SAGA, and furthermore, we obtain new results for mini-batch and non-uniform sampling variants of SAGA. Thus our work unites all SAGA variants under one framework.
 
Biography: Robert M. Gower joined Telecom ParisTech as an Assistant Professor in 2017. He is interested in designing and analyzing new stochastic algorithms for solving big data problems in Machine Learning and scientific computing. A mathematician by training, his academic studies started with a Bachelors and a Masters degree in applied mathematics at the state University of Campinas (Brazil), where he designed the current state-of-art algorithms for automatically calculating high order derivatives using back-propagation. His PhD in stochastic numerics at the University of Edinburgh earned him the 2nd place of the 2017 Leslie Fox prize in numerical analysis. After which in 2016 he was granted the Fondation Sciences Mathématiques de Paris postdoctral Laureate fund to continue his work as a postdoc in ENS.
 

More Information:

For more info contact: Professor Peter Richtarik : email: Peter.Richtarik@KAUST.EDU.SA
 
Date: Monday 27th Nov 2017
Time:12:00 PM - 01:00 PM
Location: Building 9, Lecture Hall 1, Room 2322​
Refreshments: Light Lunch will be available at 11:45 am