
Optimization Methods and Software for Federated Learning
This dissertation identifies five key challenges in Federated Learning (FL), including data and device heterogeneity, communication issues, privacy concerns, and software implementations. More broadly, our work serves as a guide for researchers navigating the complexities of translating theoretical methods into efficient real-world implementations, while also offering insights into the reverse process of adapting practical implementation aspects back into theoretical algorithm design.
Overview
Federated Learning (FL) is a novel, multidisciplinary Machine Learning paradigm where multiple clients, such as mobile devices, collaborate to solve machine learning problems. Initially introduced in Konečný et al. (2016a,b); McMahan et al. (2017), FL has gained further attention through its inclusion in the National AI Research and Development Strategic Plan (2023 Update) of the United States (Science and on Artificial Intelligence, 2023). The FL training process is inherently decentralized and often takes place in less controlled settings compared to data centers, posing unique challenges distinct from those in fully controlled environments. In this thesis, we identify five key challenges in Federated Learning and propose novel approaches to address them. These challenges arise from the heterogeneity of data and devices, communication issues, and privacy concerns for clients in FL training. Moreover, even well-established theoretical advances in FL require diverse forms of practical implementation to enhance their real-world applicability. Our contributions advance FL algorithms and systems, bridging theoretical advancements and practical implementations. More broadly, our work serves as a guide for researchers navigating the complexities of translating theoretical methods into efficient real-world implementations and software. Additionally, it offers insights into the reverse process of adapting practical implementation aspects back into theoretical algorithm design. This reverse process is particularly intriguing, as the practical perspective compels us to examine the underlying mechanics and flexibilities of algorithms more deeply, often uncovering new dimensions of the algorithms under study.
Presenters
Brief Biography
Konstantin Burlachenko is a Ph.D. candidate at the KAUST Optimization and Machine Learning Lab under the supervision of Professor Peter Richtarik. Before joining KAUST, Konstantin worked in several prominent Moscow companies, such as Huawei, NVIDIA, and Yandex. He holds a master’s degree in computer science from Bauman Moscow State Technical University, Russia.
After his graduation, he worked as a Senior Engineer for Acronis ,Yandex ,NVIDIA, and as a Principal Engineer for HUAWEI. Konstantin attended in Non-Degree Opportunity program at Stanford between 2015 and 2019 and obtained:
One of his sports achievements is the title of candidate Master of Sport in Chess.