Abstract

The interplay between Pontryagin’s Minimum Principle and Bellman’s Principle of Optimalit

In this talk, I will present some work in progress on practical optimization methods for deep learning. We will start with a discussion of several empirical techniques that enable training of large-scale models in language and vision tasks, including weight decay, averaging, and schedulers. We will then look at a new approach that we call schedule-free due to its ability to work without a pre-defined time horizon. I will share some details about the theory for these methods, explain why they might be useful in practice and then shed some light on their limitations. This talk will be oriented towards people who already have some knowledge of optimization methods.

Artificial intelligence (AI) has focused on a paradigm in which intelligence inheres in a single, autonomous agent. Social issues are entirely secondary in this paradigm. When AI systems are deployed in social contexts, however, the overall design of such systems is often naive --- a centralized entity provides services to passive agents and reaps the rewards. Such a paradigm need not be the dominant paradigm for information technology. In a broader framing, agents are active, they are cooperative, and they wish to obtain value from their participation in learning-based systems. Agents may supply data and other resources to the system, only if it is in their interest to do so. Critically, intelligence inheres as much in the overall system as it does in individual agents, be they humans or computers. This is a perspective that is familiar in the social sciences, and a key theme in my work is that of bringing economics into contact with foundational issues in computing and data sciences. I'll emphasize some of the mathematical challenges that arise at this tripartite interface.

The seminars will be delivered in person by Jürgen Schmidhuber, Director of the KAUST AI Initiative, and will run weekly during the spring semester. The program will start with aspects of the theory of computation and delve into many topics not typically covered in a deep learning course. This is a truly unique opportunity to learn from one of the founders in artificial intelligence.

Machine learning assumes a pivotal role in our data-driven world. The increasing scale of models and datasets necessitates quick and reliable algorithms for model training. This dissertation investigates adaptivity in machine learning optimizers.

Abstract

Explore GPT Technology and LLM in KAUST!

The growth of digital cameras and data communication has led to an exponential increase in video production and dissemination. As a result, automatic video analysis and understanding has become a crucial research topic in the computer vision community. However, the localization problem, which involves identifying a specific event in a large volume of data, particularly in long-form videos, remains a significant challenge.

Conversational AI and Question-Answering systems (QASs) for knowledge graphs (KGs) are both emerging research areas: they empower users with natural language interfaces for extracting information efficiently and effectively. While Conversational AI simulates human-like conversations, its effectiveness is limited by the available training data. However, QASs retrieve the most up-to-date information from KGs by translating natural language queries into formal queries that the database engine can process. In this talk, we examine the characteristics of existing approaches for combining Conversational AI and QASs to create novel KG chatbots. We also introduce KGQAn, a universal QA system that can be applied to any KG without the need for customization.

In this work we focus our attention on distributed optimization problems in the context where the communication time between the server and the workers is non-negligible. We obtain novel methods supporting bidirectional compression (both from the server to the workers and vice versa) that enjoy new state-of-the-art theoretical communication complexity for convex and nonconvex problems.

Are commercial APIs from Microsoft, Google, and Amazon for NLP tasks any better than rule-based systems? Is documentation for LLMs accessible to non-expert users in the industry? Our work on creating a unified toolkit for evaluation (robustness gym) and reporting (interactive model cards) attempts to address these questions.

This seminar focuses on providing the audience with the context and scope of our internship program. The program is for the young and talented graduate students with an active interest in solving real-world problems. Some of the projects that will be presented in the seminar are actively developed in Elm and include domains such as computer vision, robotics and automation, healthcare, IoT, video analytics, and NLP. The seminar will serve as a launch pad to allow students to discuss their future interests and aspirations with the speaker. It will also enable them to develop a better awareness of domains more relevant to their future research aspirations.

Parameter-free online optimization is a class of algorithms that does not require tuning hyperparameters, yet they achieve the theoretical optimal performance. Moreover, they often achieve state-of-the-art performance too. An example would be gradient descent algorithms completely without learning rates. In this talk, I review my past and present contributions to this field. Building upon a fundamental idea connecting optimization, gambling, and information theory, I discuss selected applications of parameter-free algorithms to machine learning and statistics. Finally, we conclude with an overview of the future directions of this field.

Skoltech Applied AI center’s mission is to create AI models and frameworks for solving the problems of sustainable development of industry and economy. In my presentation, I will overview the current center's activities, applied and fundamental problem statements, and corresponding recent results.

This talk will focus on challenges and opportunities in designing middleware for HPC, AI (Deep/Machine Learning), and Data Science. We will start with the challenges in designing runtime environments for MPI+X programming models by considering support for multi-core systems, high-performance networks (InfiniBand and RoCE), GPUs, and emerging BlueField-2 DPUs. Features and sample performance numbers of using the MVAPICH2 libraries will be presented. For the Deep/Machine Learning domain, we will focus on MPI-driven solutions to extract performance and scalability for popular Deep Learning frameworks (TensorFlow and PyTorch), large out-of-core models, and Bluefield-2 DPUs.

Machine perception that corresponds to the ability to understand the visual world based on the input from sensors, such as cameras is one of the central problems in Artificial Intelligence. To this end, recent years have witnessed tremendous progress in various instance-level recognition tasks having real-world applications in e.g., robotics, autonomous driving and surveillance. In this talk, I will first present our recent results towards understanding state-of-the-art deep learning-based visual recognition networks in terms of their robustness and generalizability. Next, I will present our results on learning visual recognition models with limited human supervision. Finally, I will discuss moving one step further from instance-level recognition to understand visual relationships between object pairs.

KAUST Introduction on Machine Learning Bootcamp

As part of KAUST’s mission to disseminate skills a

Rising Stars in AI Symposium 2022 at KAUST

The "Rising Stars in AI Symposium" is geared towards yo

In this talk, I will discuss how the use of second-order information – e.g, curvature or covariance – can help in all three problems, yet with vastly different roles in each. First, I will present a noisy quadratic model, which qualitatively predicts scaling properties of a variety of optimizers and in particular suggests that second-order optimization algorithms would extend perfect scaling to much bigger batches. Second, I will show how we can derive and implement scalable and flexible Bayesian inference algorithms from standard second-order optimization algorithms. Third, I will describe a novel second-order algorithm that finds desired equilibria and save us from converging to spurious fixed points in two-player sequential games (i.e. bilevel optimization) or even more general settings. Finally, I will conclude how my research would pave the way towards intelligent machines that can learn from data and experience efficiently, reason about their own decisions, and act in our interests.

I will give a self-contained introduction to the theory of the neural network function class and its application to image classification and numerical solution of partial differential equations.

I will give a self-contained introduction to the theory of the neural network function class and its application to image classification and numerical solution of partial differential equations.

I will give a self-contained introduction to the theory of the neural network function class and its application to image classification and numerical solution of partial differential equations.

Abstract

The overarching goal of Prof.

Abstract

Most existing learning algorithms can be categorized into supervised, semi-supervised, an