mathematical optimization

Rising Stars in AI Symposium 2022 at KAUST

-

B4 B5 A0215

Rising Stars in AI Symposium 2022 at KAUST The "Rising Stars in AI Symposium" is geared towards young researchers (including PhD students, PostDocs, young faculty) who have recently published promising work at leading venues. There will be dozens of brief in-person presentations about papers recently accepted at major AI conferences such as NeurIPS, CVPR, EMNLP, ACL, ICML, ICLR, etc. All speakers will be encouraged to start with an intro for non-AI experts. To view the complete program and for more event details, please visit the symposium website. Attend the Symposium The symposium will be

On the resolution of a theoretical question related to the nature of local training in federated learning

-

B1 L3 R3119

In this talk, I will explain the problem, its solution, and some subsequent work generalizing, extending and improving the ProxSkip method in various ways. We study distributed optimization methods based on the local training (LT) paradigm - achieving improved communication efficiency by performing richer local gradient-based training on the clients before parameter averaging - which is of key importance in federated learning. Looking back at the progress of the field in the last decade, we identify 5 generations of LT methods: 1) heuristic, 2) homogeneous, 3) sublinear, 4) linear, and 5) accelerated. The 5th generation, initiated by the ProxSkip method of Mishchenko et al (2022) and its analysis, is characterized by the first theoretical confirmation that LT is a communication acceleration mechanism.

Optimization for Deep Learning

-

B5 L5 R5209

Abstract The field of optimization for machine learning has undergone significant changes in recent years with deep learning models increasing in scale and fine-tuning taking a more prominent role. In this presentation, I will share a perspective on the direction of changes in the field and highlight interesting research directions. I will provide real-world examples of what practitioners want from optimization methods to train deep networks at scale. I will then present my recent work on adaptive methods, such as Adam and Adagrad, and explain how we can estimate the learning rate for these

Jongho Park

I am a research scientist in Computer, Electrical and Mathematical Science and Engineering Division (CEMSE) at King Abdullah University of Science and Technology (KAUST). I received a B.S. degree in Mathematical Sciences/Computer Science in 2013, and M.S. and Ph.D. degrees in Mathematical Sciences in 2015 and 2019, respectively (Advisor: Chang-Ock Lee), all from KAIST. From 2019 to 2020, I was a postdoctoral researcher in Department of Mathematical Sciences at KAIST (Mentor: Chang-Ock Lee). From 2020 to 2023, I was a research assistant professor in Natural Science Research Institute at KAIST

Ayed M. Alrashdi

Ph.D. Student (former), Electrical and Computer Engineering
Ph.D. degree in electrical and computer engineering from the King Abdullah University of Science and Technology (KAUST) working with Professor Tareq Al-Naffouri in the Information System Lab (ISL),Assistant Professor with the Electrical Engineering Department at University of Ha'il