KAUST professor Peter Richtárik receives Charles Broyden Prize for second consecutive year
The prize honors work advancing communication-efficient optimization for large-scale distributed learning systems.
About
KAUST Computer Science Professor Peter Richtárik has received the 2023 Charles Broyden Prize, awarded annually by the journal Optimization Methods and Software. The recognition marks Richtárik’s second consecutive year receiving the prize, following his 2022 award.
The prize recognizes the paper “Stochastic Distributed Learning with Gradient Quantization and Double-Variance Reduction,” authored by Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko, Peter Richtárik and Sebastian Stich, with the work conducted at KAUST as part of doctoral research and an international research visit.
Named after British mathematician Charles George Broyden (1933–2011), a pioneer of numerical optimization and a co-originator of the BFGS algorithm, the Charles Broyden Prize is an international distinction established by the editorial board of Optimization Methods and Software. The award recognizes outstanding theoretical contributions to optimization.
The 2023 award-winning paper builds on a line of research on communication-efficient distributed optimization that includes DIANA, a method previously introduced by Richtárik and collaborators. This research addresses a central challenge in large-scale machine learning: reducing the amount of information exchanged between computing nodes while preserving exisiting convergence guarantees. In other words, the DIANA algorithm can provably train a machine learning model with a substantially smaller communication footprint, which leads to substantially faster training in situations where the connection speed among the compute nodes is slow.
In the new work, the authors develop and analyze advanced variance-reduction techniques that strengthen theoretical guarantees and further improve communication and computation efficiency. Further, this new work enables a myriad of new communication compression methods to be used, which makes the method much more versatile and practical. These advances address a key bottleneck in large-scale artificial intelligence systems, where communication overhead often limits performance.
The project also reflects KAUST’s emphasis on research excellence and advanced graduate education, where foundational theory is closely integrated with doctoral training and international collaboration. At the time of the research, Horváth, Kovalev and Mishchenko were KAUST Ph.D. students, while Stich collaborated as a visiting researcher. Since completing the work, Horváth has joined MBZUAI in Abu Dhabi as an assistant professor, Kovalev and Mishchenko have moved into research scientist roles at Yandex and Meta, respectively, and Stich is now an assistant professor at CISPA in Saarbrücken, Germany.
“This research was developed at KAUST with doctoral students working in my lab, and a visiting international collaborator with whom I have collaborated multiple times before,” Richtárik said. “It reflects an environment that supports rigorous research contributing to scalable and reliable artificial intelligence systems.”
Richtárik’s consecutive recognition with the Charles Broyden Prize places KAUST among a small group of institutions making sustained contributions to the theoretical foundations and practical efficiency of optimization, machine learning and large-scale computation.
