Neural Networks

Mohammed A Shukri

Technical Consultant at KAUST (King Abdullah University of Science and Technology).

Principled Scaling of Deep Neural Networks

Soufiane Hayou, Postdoc, Simons Institute, UC Berkeley

-

B9 L4 R4225

Neural networks have achieved impressive performance in many applications such as image and speech recognition and generation. State-of-the-art performance is usually achieved via a series of engineered modifications to existing neural architectures and their training procedures. However, a common feature of these systems is their large-scale nature: modern neural networks usually contain Billions - if not 10's of Billions - of trainable parameters, and empirical evaluations (generally) support the claim that increasing the scale of neural networks (e.g. width and depth) boosts the model performance if done correctly. However, given a neural network model, it is not straightforward to address the crucial question `how do we scale the network?'. In this talk, I will show how we can leverage different mathematical results to efficiently scale neural networks, with empirically confirmed benefits.

Haozhe Liu

Ph.D. Student, Computer Science
Haozhe Liu is a PhD student at AI Initiative, King Abdullah University of Science and Technology (KAUST) under the supervision of Prof.Jürgen Schmidhuber.

Arnaud Dethise

Ph.D. Student, Computer Science
Arnaud Dethise is a PhD candidate in Computer Science under the guidance of Professor Marco Canini. He completed his Bachelor's degree in Engineering and Master's degree in Computer Science in 2015 and 2017 respectively, from UCLouvain, Louvain-la-Neuve, Belgium. Arnaud has a strong interest in machine learning applied to distributed systems and is currently pursuing his doctoral research in this field. His specific research interests include Neural Networks, trustworthiness, verifiability and explainability, and model privacy, with a focus on exactness and provable guarantees. Arnaud’s goals