Neural Networks

Approximation and Optimization for Neural Networks

-

B2/B3 L0 A0215

In this talk, we consider new connections between the approximation and optimization of neural networks. Instead of relying on excessive over-parametrization to achieve zero training loss, we identify good minima by comparison with established approximation bounds.

Mapping Properties of Neural Networks and Inverse Problems

-

B1 L4 R4214

This talk will explore the injectivity and universal approximation properties of neural networks, focusing on one-to-one mappings and the approximation of probability measures using compositions of invertible flow networks and injective layers, with applications in inverse problems, imaging, and generative models.

Mohammed A Shukri

Technical Consultant at KAUST (King Abdullah University of Science and Technology).

Principled Scaling of Deep Neural Networks

Soufiane Hayou, Postdoc, Simons Institute, UC Berkeley

-

B9 L4 R4225

Neural networks have achieved impressive performance in many applications such as image and speech recognition and generation. State-of-the-art performance is usually achieved via a series of engineered modifications to existing neural architectures and their training procedures. However, a common feature of these systems is their large-scale nature: modern neural networks usually contain Billions - if not 10's of Billions - of trainable parameters, and empirical evaluations (generally) support the claim that increasing the scale of neural networks (e.g. width and depth) boosts the model performance if done correctly. However, given a neural network model, it is not straightforward to address the crucial question `how do we scale the network?'. In this talk, I will show how we can leverage different mathematical results to efficiently scale neural networks, with empirically confirmed benefits.

Haozhe Liu

Ph.D. Student, Computer Science
Haozhe Liu is a Ph.D. candidate at King Abdullah University of Science and Technology (KAUST), supervised by Prof. Jürgen Schmidhuber.

Arnaud Dethise

Ph.D. Student, Computer Science
Arnaud Dethise is a PhD candidate in Computer Science under the guidance of Professor Marco Canini. He completed his Bachelor's degree in Engineering and Master's degree in Computer Science in 2015 and 2017 respectively, from UCLouvain, Louvain-la-Neuve, Belgium. Arnaud has a strong interest in machine learning applied to distributed systems and is currently pursuing his doctoral research in this field. His specific research interests include Neural Networks, trustworthiness, verifiability and explainability, and model privacy, with a focus on exactness and provable guarantees. Arnaud’s goals