Federated learning (FL) is an emerging machine learning paradigm involving multiple clients, e.g., mobile phone devices, with an incentive to collaborate in solving a machine learning problem coordinated by a central server. FL was proposed in 2016 by Konecny et al. and McMahan et al. as a viable privacy-preserving alternative to traditional centralized machine learning since, by construction, the training data points are decentralized and never transferred by the clients to a central server. Therefore, to a certain degree, FL mitigates the privacy risks associated with centralized data collection. Unfortunately, optimization for FL faces several specific issues that centralized optimization usually does not need to handle. In this thesis, we identify several of these challenges and propose new methods and algorithms to address them, with the ultimate goal of enabling practical FL solutions supported with mathematically rigorous guarantees.

Datasets that capture the connection between vision, language, and affection are limited, causing a lack of understanding of the emotional aspect of human intelligence. As a step in this direction, the ArtEmis dataset was recently introduced as a large-scale dataset of emotional reactions to images along with language explanations of these chosen emotions.

Hydrogen is a carbon-free energy carrier that can be used to decarbonize various high-emitting sectors, such as transportation, power generation, and industry. Today, global hydrogen production is largely derived from fossil fuels such as natural gas and coal.

Differential Privacy (DP) allows for rich statistical and machine learning analysis, and is now becoming a gold standard for private data analysis. Despite the noticeable success of this theory, existing tools from DP are severely limited to regular datasets, e.g., datasets need to be or are assumed to be clean and normalized before performing DP algorithms.

The power system is facing unprecedented changes in operation and control as more and diverse sources and loads are being connected to this complex cyber-physical energy system.

Geospatial health data are essential to inform public health and policy. These data can be used to quantify disease burden, understand geographic and temporal patterns, identify risk factors and measure inequalities. In this talk, I will give an overview of statistical methods and computational tools for geospatial data analysis and health surveillance.

DNA Nanotechnology is a fascinating field that studies how to construct small biological structures entirely from DNA as a building material. The key insight is that DNA, if designed in a particular way, can construct complex 3D nanoscale structures entirely by means of self-assembly, governed by the base-pairing principle.

Traditional computing systems separate processors from memory, performing computation by shuttling data back and forth between these two units all the time. This bottleneck incurs limited processing speed and high power consumption in computing systems for deep learning models of ever-increasing complexity. Novel approaches and new principles are needed to revolutionize computing systems. Neuromorphic systems are proposed as a new computing architecture based on spiking neural networks analogous to the existing nervous systems.

We study the MARINA method of Gorbunov et al (ICML 2021) - the current state-of-the-art distributed non-convex optimization method in terms of theoretical communication complexity. Theoretical superiority of this method can be largely attributed to two sources: the use of a carefully engineered biased stochastic gradient estimator, which leads to a reduction in the number of communication rounds, and the reliance on {\em independent} stochastic communication compression operators, which leads to a reduction in the number of transmitted bits within each communication round.

Mobile computing proliferation is critically threatened by the breakdown of Dennard scaling, a law describing the area-proportional growth of integrated circuit power use.

The qualitative study of PDEs often relies on integral identities and inequalities. For example, for time-dependent PDEs, conserved integral quantities or quantities that are dissipated play an important role. In particular, if these integral quantities have a definite sign, they are of great interest as they may provide control on the solutions to establish well-posedness.

Dynamic programming is an efficient technique to solve optimization problems. It is based on decomposing the initial problem into simpler ones and solving these sub-problems beginning from the simplest ones.

The talk will discuss the challenging problem of designing Deep Neural Network systems that achieve high performance under low power envelopes, hindering their deployment in the embedded space.

The life sciences have invested significant resources in the development and application of semantic technologies to make research data accessible and interlinked, and to enable the integration and analysis of data. Utilizing the semantics associated with research data in data analysis approaches is often challenging. Now, novel methods are becoming available that combine symbolic methods and statistical methods in Artificial Intelligence. In my talk, I will show how to incorporate biological background knowledge in machine learning models for identification of gene-disease associations, genomic variants that are causative for heritable disorders, and to predict protein functions. The methods I describe are generic and can be applied in other domains in which biomedical ontologies and structured knowledge bases exist.

Available as dedicated hardware components into several mobile and server-grade processors, and recently included in infrastructure-as-a-service commercial offerings by several cloud providers, TEEs allow applications with high privacy and confidentiality demands to be deployed and executed over untrusted environments, shielding data and code from compromised systems or powerful attackers. After an  introduction to basic concepts for TEEs, I will survey some of our most recent contributions exploiting TEEs, including as defensive tools in the context of Federated Learning, as support to build secure cache systems for edge networks, as protection mechanisms in a med-tech/e-health context,  shielding novel environments (ie, WebAssembly), and more. Finally, I will highlight some of the lessons learned and offer open perspectives, hopefully useful and inspirational to future researchers and practitioners entering this exciting area of research.

The natural sciences such as biology, medicine, and chemistry are currently in a transformative stage. Progress in technologies for measuring and collecting data (sequences, images, and molecules) has exploded since the human genome project. In parallel, we have witnessed stunning advances in what can broadly be referred to as computational techniques. This includes data-driven analysis of data such as Machine learning and Artificial Intelligence. From an ML/AI standpoint, there is a renewed interest in classical” equation-based modeling, causal analysis, and generative probabilistic modeling techniques. BioAI refers to this “perfect storm” between Bio and AI.

Error feedback (EF), also known as error compensation, is an immensely popular convergence stabilization mechanism in the context of distributed training of supervised machine learning models enhanced by the use of contractive communication compression mechanisms, such as Top-k. First proposed by Seide et al (2014) as a heuristic, EF resisted any theoretical understanding until recently [Stich et al., 2018, Alistarh et al., 2018].

The overarching goal of Prof. Michels' Computational Sciences Group within KAUST's Visual Computing Center is enabling accurate and efficient simulations for applications in Scientific and Visual Computing. Towards this goal, the group develops new principled computational methods based on solid theoretical foundations.

A traditional goal of algorithmic optimality, squeezing out operations, has been superseded because of evolution in architecture. Algorithms must now squeeze memory, data transfers, and synchronizations, while extra operations on locally cached data cost relatively little time or energy. Hierarchically low-rank matrices realize a rarely achieved combination of optimal storage complexity and high-computational intensity in approximating a wide class of formally dense operators that arise in exascale applications.

In this thesis, we discuss some new developments in optimization inspired by the needs and practice of machine learning, federated learning, and data science. In particular, we consider seven key challenges of mathematical optimization that are relevant to modern machine learning applications, and develop a solution to each.

About 12 years ago, Bitcoin was created as the first form of decentralized money, with some of the properties of Nash's ideal money. The protocol proposes a novel probabilistic consensus mechanism, that has the potential to automatize and decentralize many other human activities. The Bitcoin network also provides the first decentralized clock, and has a rich statistical physics interpretation. We will explore the foundations of "Decentralization Theory" and explore what can be expected as future developments.

This talk will give an overview of the research of the Secure Next Generation Resilient Systems (SENTRY) lab (sentry.kaust.edu.sa) at KAUST. The transformation of critical infrastructures into cyber-physical systems contributes towards modernization allowing for better planning, more flexible control, system-wide optimization, etc. The security, however, of such systems presents significant challenges in controlling and maintaining secure access to critical system resources and services.

Classical imaging systems are characterized by the independent design of optics, sensors, and image processing algorithms. In contrast, computational imaging systems are based on a joint design of two or more of these components, which allows for greater flexibility of the type of captured information beyond classical 2D photos, as well as for new form factors and domain-specific imaging systems. In this talk, I will describe how numerical optimization and learning-based methods can be used to achieve truly end-to-end optimized imaging systems that outperform classical solutions.