In this work we focus our attention on distributed optimization problems in the context where the communication time between the server and the workers is non-negligible. We obtain novel methods supporting bidirectional compression (both from the server to the workers and vice versa) that enjoy new state-of-the-art theoretical communication complexity for convex and nonconvex problems.

Are commercial APIs from Microsoft, Google, and Amazon for NLP tasks any better than rule-based systems? Is documentation for LLMs accessible to non-expert users in the industry? Our work on creating a unified toolkit for evaluation (robustness gym) and reporting (interactive model cards) attempts to address these questions.

This seminar focuses on providing the audience with the context and scope of our internship program. The program is for the young and talented graduate students with an active interest in solving real-world problems. Some of the projects that will be presented in the seminar are actively developed in Elm and include domains such as computer vision, robotics and automation, healthcare, IoT, video analytics, and NLP. The seminar will serve as a launch pad to allow students to discuss their future interests and aspirations with the speaker. It will also enable them to develop a better awareness of domains more relevant to their future research aspirations.

Parameter-free online optimization is a class of algorithms that does not require tuning hyperparameters, yet they achieve the theoretical optimal performance. Moreover, they often achieve state-of-the-art performance too. An example would be gradient descent algorithms completely without learning rates. In this talk, I review my past and present contributions to this field. Building upon a fundamental idea connecting optimization, gambling, and information theory, I discuss selected applications of parameter-free algorithms to machine learning and statistics. Finally, we conclude with an overview of the future directions of this field.

Skoltech Applied AI center’s mission is to create AI models and frameworks for solving the problems of sustainable development of industry and economy. In my presentation, I will overview the current center's activities, applied and fundamental problem statements, and corresponding recent results.

This talk will focus on challenges and opportunities in designing middleware for HPC, AI (Deep/Machine Learning), and Data Science. We will start with the challenges in designing runtime environments for MPI+X programming models by considering support for multi-core systems, high-performance networks (InfiniBand and RoCE), GPUs, and emerging BlueField-2 DPUs. Features and sample performance numbers of using the MVAPICH2 libraries will be presented. For the Deep/Machine Learning domain, we will focus on MPI-driven solutions to extract performance and scalability for popular Deep Learning frameworks (TensorFlow and PyTorch), large out-of-core models, and Bluefield-2 DPUs.

Machine perception that corresponds to the ability to understand the visual world based on the input from sensors, such as cameras is one of the central problems in Artificial Intelligence. To this end, recent years have witnessed tremendous progress in various instance-level recognition tasks having real-world applications in e.g., robotics, autonomous driving and surveillance. In this talk, I will first present our recent results towards understanding state-of-the-art deep learning-based visual recognition networks in terms of their robustness and generalizability. Next, I will present our results on learning visual recognition models with limited human supervision. Finally, I will discuss moving one step further from instance-level recognition to understand visual relationships between object pairs.

KAUST Introduction on Machine Learning Bootcamp

As part of KAUST’s mission to disseminate skills a

Rising Stars in AI Symposium 2022 at KAUST

The "Rising Stars in AI Symposium" is geared towards yo

In this talk, I will discuss how the use of second-order information – e.g, curvature or covariance – can help in all three problems, yet with vastly different roles in each. First, I will present a noisy quadratic model, which qualitatively predicts scaling properties of a variety of optimizers and in particular suggests that second-order optimization algorithms would extend perfect scaling to much bigger batches. Second, I will show how we can derive and implement scalable and flexible Bayesian inference algorithms from standard second-order optimization algorithms. Third, I will describe a novel second-order algorithm that finds desired equilibria and save us from converging to spurious fixed points in two-player sequential games (i.e. bilevel optimization) or even more general settings. Finally, I will conclude how my research would pave the way towards intelligent machines that can learn from data and experience efficiently, reason about their own decisions, and act in our interests.

I will give a self-contained introduction to the theory of the neural network function class and its application to image classification and numerical solution of partial differential equations.

I will give a self-contained introduction to the theory of the neural network function class and its application to image classification and numerical solution of partial differential equations.

I will give a self-contained introduction to the theory of the neural network function class and its application to image classification and numerical solution of partial differential equations.

Abstract

The overarching goal of Prof.

Abstract

Most existing learning algorithms can be categorized into supervised, semi-supervised, an

Abstract

In this talk, I will talk about recent topics in generative modeling.

Abstract

In essence, science is about discovering regularities in Nature.

Abstract

An ontology is an explicit specification of a conceptualization of a domain.

Abstract

RNAs need to form specific secondary structures for their functions.

Abstract

Protein-protein interactions drive many important biological events such as infection, re

Abstract

Quantitative structure-property relationships (QSPRs) using machine learning tools to rel

Abstract

In this talk, I will introduce our recent progress on how deep learning models inspired u

Abstract

In safety-critical application domains, it is crucial to assess the attackability of the

Abstract

We propose a new optimization formulation for training federated learning models.

Abstract

Recent research showed that most of the existing machine learning algorithms are vulnerab