About Filip Hanzely Filip Hanzely Ph.D., Applied Mathematics and Computational Science optimization machine learning Filip Hanzely is a recent PhD graduate from the group of Peter Richtarik. His research focuses mostly on various aspects of optimization for machine learning, designing provably efficient algorithms for solving big data problems. Filip is joining Toyota Technological Institute in Chicago (TTIC) as a research assistant professor in December 2020. Education and Early Career Filip Hanzely graduated with honor for his bachelor degree in Economics and Financial Mathematics from Comenius University in Bratislava, Slovakia. Later on in 2017, he received his master in science degree in Mathematics and Events Presented Events Nov 15 - Nov 21, 2020 Federated Learning of a Mixture of Global and Local Models Filip Hanzely, Ph.D., Applied Mathematics and Computational Science Nov 16, 12:00 - 13:00 KAUST We propose a new optimization formulation for training federated learning models. The standard formulation has the form of an empirical risk minimization problem constructed to find a single global model trained from the private data stored across all participating devices. In contrast, our formulation seeks an explicit trade-off between this traditional global model and the local models, which can be learned by each device from its own private data without any communication. Further, we develop several efficient variants of SGD (with and without partial participation and with and without variance reduction) for solving the new formulation and prove communication complexity guarantees. Jul 26 - Aug 1, 2020 Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters Filip Hanzely, Ph.D., Applied Mathematics and Computational Science Jul 30, 16:00 - 18:00 KAUST Many key problems in machine learning and data science are routinely modeled as optimization problems and solved via optimization algorithms. With the increase of the volume of data and the size and complexity of the statistical models used to formulate these often ill-conditioned optimization tasks, there is a need for new efficient algorithms able to cope with these challenges. In this thesis, we deal with each of these sources of difficulty in a different way. To efficiently address the big data issue, we develop new methods which in each iteration examine a small random subset of the training data only. To handle the big model issue, we develop methods which in each iteration update a random subset of the model parameters only. Finally, to deal with ill-conditioned problems, we devise methods that incorporate either higher-order information or Nesterov's acceleration/momentum. In all cases, randomness is viewed as a powerful algorithmic tool that we tune, both in theory and in experiments, to achieve the best results.
Federated Learning of a Mixture of Global and Local Models Filip Hanzely, Ph.D., Applied Mathematics and Computational Science Nov 16, 12:00 - 13:00 KAUST We propose a new optimization formulation for training federated learning models. The standard formulation has the form of an empirical risk minimization problem constructed to find a single global model trained from the private data stored across all participating devices. In contrast, our formulation seeks an explicit trade-off between this traditional global model and the local models, which can be learned by each device from its own private data without any communication. Further, we develop several efficient variants of SGD (with and without partial participation and with and without variance reduction) for solving the new formulation and prove communication complexity guarantees.
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters Filip Hanzely, Ph.D., Applied Mathematics and Computational Science Jul 30, 16:00 - 18:00 KAUST Many key problems in machine learning and data science are routinely modeled as optimization problems and solved via optimization algorithms. With the increase of the volume of data and the size and complexity of the statistical models used to formulate these often ill-conditioned optimization tasks, there is a need for new efficient algorithms able to cope with these challenges. In this thesis, we deal with each of these sources of difficulty in a different way. To efficiently address the big data issue, we develop new methods which in each iteration examine a small random subset of the training data only. To handle the big model issue, we develop methods which in each iteration update a random subset of the model parameters only. Finally, to deal with ill-conditioned problems, we devise methods that incorporate either higher-order information or Nesterov's acceleration/momentum. In all cases, randomness is viewed as a powerful algorithmic tool that we tune, both in theory and in experiments, to achieve the best results.
Related Sites Optimization and Machine Learning (OML) Applied Mathematics and Computational Science (AMCS) Related Content Events 2 Related Links Also view list of Publications on KAUST Repository
