distributed computing

EF-BV: distributed optimization with compressed communication

-

B9 L2 R2322 H1

In the big data era, it is necessary to rely on distributed computing. For distributed optimization and learning tasks, in particular in the modern paradigm of federated learning, specific challenges arise, such as decentralized data storage. Communication between the parallel machines and the orchestrating distant server is necessary but slow. To address this main bottleneck, a natural strategy is to compress the communicated vectors. I will present EF-BV, a new algorithm which converges linearly to an exact solution, with a large class of deterministic or random, biased or unbiased compressors.

Mojtaba Alshams

Education and Early Career Earned a BS in Electrical Engineering with 2nd honor from King Fahd University of Petroleum & Minerals where the degree specialization was more oriented around communication, electronics and embedded systems. Had about three years of high professional experience and empowered leadership in industrial electrical and instrumentation engineering works in district cooling plant construction project. Currently pursuing MS degree in Electrical & Computer Engineering at KAUST with a thesis in distributing ML algorithms across hardware accelerators. Simultaneously, engaging

Ali Charara

Ali Charara obtained both, his MS Degree and PhD Degree from Computer Science at KAUST in 2011 and 2018 respectively. His master thesis supervisor was Alyn Rockwood and his PhD thesis supervisor was David Keyes. Ali's research interests were in High Performance Computing with a focus on parallel computing and algorithms using multicore and manycore Computer Graphics Units. Education Profile MS Degree, Computer Science, KAUST (2011) PhD Degree, Computer Science, KAUST (2018)