neural network

Muhammad Esmat

Visiting student at, King Abdullah University of Science and Technology (KAUST).

Dynamics and Convergence of Weight Normalization for Training Neural Networks

Guido Montufar, Assistant Professor, Departments of Mathematics and Statistics, University of California, Los Angeles (UCLA)

-

B1 L3 R3119

We present a result on the convergence of weight normalized training of artificial neural networks. In the analysis, we consider over-parameterized 2-layer networks with rectified linear units (ReLUs) initialized at random and trained with batch gradient descent and a fixed step size. The proof builds on recent theoretical works that bound the trajectory of parameters from their initialization and monitor the network predictions via the evolution of a ''neural tangent kernel'' (Jacot et al. 2018). We discover that training with weight normalization decomposes such a kernel via the so called ''length-direction decoupling''. This in turn leads to two convergence regimes. From the modified convergence we make a few curious observations including a natural form of ''lazy training'' where the direction of each weight vector remains stationary.

Economic Model Predictive Control Strategies for Nonlinear Systems

-

KAUST

The first part of this talk provides a brief introduction of the EMPC (motivation, challenges, and solutions). The second part of this talk proposes a regret-based robust EMPC paradigm for nonlinear systems subject to unknown but bounded disturbance. The main motivation of the proposed work is the possible improvement of the economic performance when one considers the regret function as the objective function for the robust EMPC algorithm instead of the worst cost. The third part of this talk introduces an integrated framework that combines a Neural Network (NN) algorithm with an MPC scheme that can guarantee closed-loop stability in the presence of deception cyberattacks (e.g., min-max cyberattack). Both discrete-time and continuous-time nonlinear systems will be utilized throughout the talk to demonstrate the applicability and effectiveness of the proposed control methods. Finally, future research directions will be presented at the end of the talk.

Why Today’s Neural Networks Still Use the 1943 McCullosh and Pitt’s Neuron Model?

Prof. Moncef Gabbouj, Department of Computing Sciences, Tampere University

-

B1 L3 R3119

Operational Neural Networks (ONNs) are new generation network models targeting to address two major drawbacks of conventional Convolutional Neural Networks (CNNs): the homogenous network configuration and the “linear” neuron model that can only perform linear transformations over previous layer outputs. ONNs can perform any linear or non-linear transformation with a proper combination of “nodal” and “pool” operators.

Maxat Kulmanov defends his thesis proposal

1 min read ·
Maxat Kulmanov, a Ph.D. candidate under the supervision of Assistant Professor Robert Hoehndorf, defended his thesis proposal on July 31, 2018.

Mohammed Aljahdali

M.S. Student, Computer Science
Mohammed Khalid Aljahdali is a M.S./Ph.D. student in computer science program and member of the Software-defined Advanced Networked and Distributed Systems (SANDS) Lab under the supervision of Professor Marco Canini. Research Interests Mohammed's research interests include deep learning.

Maxat Kulmanov

Research Scientist, Bio-Ontology Research Group
Maxat Kulmanov is a Research Scientist in the Bio-Ontology Research Group at King Abdullah University of Science and Technology (KAUST). Research Interests Maxat's research interests include bioinformatics, knowledge representation and reasoning, machine learning, neural networks, semantic web, and algorithms. He is interested in knowledge discovery and data integration using artificial intelligence and semantic web technologies in biology and biomedicine. Professional Profile 2020-Present: Postdoctoral Research Fellow, Computational Bioscience Research Center, King Abdullah University of

Jun Chen

Ph.D. Student, Computer Science
Jun Chen is a Ph.D. candidate in the Computer Science Program under the supervision of Professor Mohamed Elhoseiny at King Abdullah University of Science and Technology (KAUST). Education and Early Career Jun obtained his bachelor's degree in Computer Science from Xi’an Jiaotong-Liverpool University in China, 2018. He joined KAUST as a Master student under the supervision of Prof. Robert Hoehndorf and received his master's degree in Computer Science in 2019. Research Interests Jun's research interests include machine learning, artificial intelligence, natural language processing, knowledge

Gian Marco Messa

Ph.D. Student, Computer Science
Gian Marco is a Ph.D. student at the Structural and Functional Bioinformatics Group (SFB) under the supervision of Professor Xin Gao at King Abdullah University of Science and Technology (KAUST). Gian Marco, upon finishing high school in Lecce (Italy), applied to the Politecnico of Milan in the year 2009 in Physics engineering. Due to personal issues, he came back to his hometown to complete the Bachelor's Degree at the University of Salento in Biotechnology. For his master's degree, seeking innovation and cutting edge knowledge, he moved to the University of Bologna, which offered an M. Sc