By David Murphy
Deep learning — a machine learning technique that teaches computers to process data similarly to how humans do—might be the most successful form of AI; however, current training algorithms are far from fully automated. Instead, they require constant human supervision to ensure that algorithms converge correctly.
Therein lies the paradox: we use machine learning algorithms to automatize tasks, but their training still requires a large amount of human supervision. The work of one of KAUST’s newest faculty members focuses on designing “parameter-free” algorithms that he believes are “better and truly automatic.”
Francesco Orabona, associate professor of computer science, joins KAUST from Boston University’s Department of Electrical & Computer Engineering (BU ECE).
Orabona—who received his B.Sc. and M.S. ('03) in electrical engineering from the University of Naples Federico II and his Ph.D. in electrical engineering ('07) from the University of Genoa, Italy—describes himself as an individual who endeavors “to create through mathematics.”
Parameter-free machine learning is the primary focus of the former head of BU ECE’s Optimization and Machine Learning Lab. In particular, he is interested in designing better algorithms to train deep learning models.
“My research uses classic optimization methods, but it also uses ideas from information theory and even gambling. Indeed, in my research, I have shown that one can obtain better optimization algorithms using strategies to bet on coin flips that, in turn, have a tight connection with information theory.”
While at KAUST, Orabona’s goal is to develop and train large machine learning models: “For example, large language models like ChatGPT are state-of-the-art machine learning models able to produce text, answer questions and reason on problems,” he noted. “Like many similar models, ChatGPT is a closed-source model; that is, we don't have access to the exact recipe that was used to train it.
“However, we know that training these models requires a lot of tricks and hacks. I want to study these issues and propose better parameter-free optimization algorithms tailored for these applications.”
According to Orabona, AI is a branch of computer science that is moving at an incredible rate. He feels that only researchers with a solid foundation in the field can keep up with the breakneck speed of its countless advancements.
“The next few years will be pivotal in AI research, and I believe that being in the right place to study it can make a huge difference. To that end, I believe—in KAUST—I am in one of the few rare places in the world where the competencies and resources exist to train the next generation of machine learning models and AI.”
Related Persons
