Machine learning applications have sparked the development of specialized software frameworks and hardware accelerators. Yet, in today’s machine learning ecosystem, one important part of the system stack has received far less attention and specialization for ML: how we store and preprocess training data. This talk will describe the key challenges for implementing high-performance ML input data processing pipelines.

Recent research showed that most of the existing machine learning algorithms are vulnerable to various privacy attacks. An effective way for defending these attacks is to enforce differential privacy during the learning process. As a rigorous scheme for privacy preserving, Differential Privacy (DP) has now become a standard for private data analysis. Despite its rapid development in theory, DP's adoption to the machine learning community remains slow due to various challenges from the data, the privacy models and the learning tasks. In this talk, I will give a brief introduction on DP and use the Empirical Risk Minimization (ERM) problem as an example and show how to overcome these challenges in DP model. Particularly, I will first talk about how to overcome the high dimensionality challenge from the data for Sparse Linear Regression in the local DP (LDP) model. Then, I will discuss the challenge from the non-interactive LDP model and show a series of results to reduce the exponential sample complexity of ERM. Next, I will present techniques on achieving DP for ERM with non-convex loss functions. Finally, I will discuss some future research along these directions.

In this thesis defense, I will explore the unique characteristics of IoT traffic and examine IoT systems. The work is motivated by the new capabilities offered by modern Software Defined Networks (SDN) and blockchain technology. We evaluate IoT Quality of Service (QoS) in traditional networking. We obtain mathematical expressions to calculate end-to-end delay, and dropping. Then, we analyze IoT traffic load and propose an intelligent edge that can identify volumetric traffic and address them in real-time using an instantaneous detection method for IoT applications (IDIoT). This approach can easily detect a large surge and potential variation in traffic patterns for an IoT application, which may contribute to safer and more efficient operation of the overall system. Our results provide insight into the advantages of an intelligent edge serving as a detection mechanism.

GPUs have emerged as a popular choice for deep learning. To deal with ever-growing datasets, it is also common to use multiple GPUs in parallel for distributed deep learning. Although achieving cost-effectiveness in these clusters relies on efficient sharing, modern GPU hardware, deep learning frameworks, and cluster managers are not designed for efficient, fine-grained sharing of GPU resources. In this talk, I will present our recent works on efficient GPU resource management, both within a single GPU and across many GPUs in a cluster for hyperparameter tuning, training, and inference. The common thread across all our works is leveraging the interplay between short-term predictability and long-term unpredictability of deep learning workloads

In this talk I will present recent advancement in automated reasoning, in particular computer-supported theorem proving, for generating and proving software properties that prevent programmers from introducing errors while making changes in this software. When testing programs manipulating the computer memory, our initial results show our work is able to prove that over 80% of test cases are guaranteed to have the expected behavior.

In essence, science is about discovering regularities in Nature. It turns out that such regularities (laws) are written in the language of mathematics. In many cases, such laws are formulated and refined from fundamental “first principles.” Yet, in phenomenological areas such as biology, we have an abundance of data but lack “first principles.” Machine learning and deep learning, in particular, are remarkably successful in classification and prediction tasks. However, such systems, when trained on data, do not, as a rule, provide compact mathematical laws or fundamental first principles. Here we ask how we can identify interpretable compact mathematical laws from complex data-sets when we don’t have access to first principles. I will give an overview of this problem and provide some vignettes of our ongoing work in attacking this problem.

We present a novel large-scale dataset and accompanying machine learning models aimed at providing a detailed understanding of the interplay between visual content, its emotional effect, and explanations for the latter in language. In contrast to most existing annotation datasets in computer vision, we focus on the affective experience triggered by visual artworks and ask the annotators to indicate the dominant emotion they feel for a given image and, crucially, to also provide a grounded verbal explanation for their emotion choice.

In this talk, I will introduce our recent efforts on developing novel computational models in the field of biological imaging. I will start with the examples in electron tomography, for which I will introduce a robust and efficient scheme for fiducial marker tracking, and then describe a novel constrained reconstruction model towards higher resolution sub-tomogram averaging. I will then show our work on developing deep learning methods for super-resolution fluorescence microscopy.

"A picture is worth a thousand words", and by going beyond static images, interactive visualization has become crucial to exploring, analyzing, and understanding large-scale scientific data. This is true for many areas of science and engineering, such as high-resolution imaging in neuroscience or materials science, as well as in large-scale fluid simulations of the Earth’s atmosphere and oceans, or of trillion-cell oil reservoirs. However, the fact that the amount of data in data-driven sciences is increasing rapidly toward the petascale, and further, presents a tremendous challenge to interactive visualization and analysis. Nowadays, an important enabler of interactivity is often the parallel processing power of GPUs, which, however, requires well-designed customized data structures and algorithms. Furthermore, scientific data sets do not only get larger, they also get more and more complex, and thus have become very hard to interpret and analyze. In this talk, I will give an overview of the research of my group in large-scale scientific visualization, from data structures and algorithms that enable petascale visualization on GPUs, to novel visual abstractions for interactive analysis of highly complex structures in neuroscience, to novel mathematical techniques that leverage differential geometric methods for the detection and visualization of features in large, complex fluid dynamics data on curved surfaces such as the Earth.

Drawing the user's attention to important items in an image, a complex visualization, or a cluttered graphical user interface is a non-trivial challenge. In the context of visualization, our goal is to effectively attract the user's attention to relevant items in large and complex scenes, while keeping noticeable modifications of the image to a minimum. In this talk, I will give an overview of common highlighting methods and present results from my research on attention guidance in complex, dynamic visualizations.

Life at micron-scale is inaccessible to the naked eye. To aid the comprehension of nano- and micro-scale structural complexity, we utilize 3D visualization. Thanks to efficient GPU-accelerated algorithms, we witness a dramatic boost in the sheer size of structures that can be visually explored. As of today an atomistic model of the entire bacterial cell can be displayed interactively. On top of that, advanced 3D visualizations efficiently convey the multi-scale hierarchical architecture and cope with the high degree of structural occlusion which comes with dense packing of biological building blocks. To further scale up the size of life forms that can be visually explored, the rendering pipeline needs to integrate runtime construction of the biological structure. Assembly rules define how a body of a certain biological entity is composed. Such rules need to be applied on-the-fly, depending on where the viewer is currently located in the 3D scene to generate full structural detail for that part of the scene. We will review how to construct membrane-like structures, soluble protein distributions, and fiber strands through parallel algorithms, resulting in a collision-free biologically-valid scene. Assembly rules that define how a life form is structurally built need to be expressed in an intuitive way for the domain scientist, possibly directly in the three-dimensional space. Instead of placing one biological element next to another one for the entire biological structure by the modelers themselves, only assembly rules need to be specified and the algorithm will complete the application of those rules to form the entire biological entity. These rules are derived from current scientific knowledge and from all available experimental observations. The Cryo-EM Tomography is on rising and shows that we can already now reach the near-atomistic detail when employing smart algorithms. Our assembly rules extraction, therefore, needs to integrate with microscopic observations, to create an atomistic representation of specific, observed life forms, instead of generic models thereof. Such models then can be used in whole-cell simulations, and in the context of automated science dissemination.

In a nutshell, Resilient Computing is a new paradigm based on modelling, architecting and designing computer systems so that: they have built-in baseline defences; such defences cope with virtually any quality of threat, be it accidental faults, design errors, cyber-attacks, or unexpected operating conditions; provide incremental protection of, and automatically adapt to, a dynamic range of threat severity; provide sustainable operation.

I will review some background of generative adversarial modeling using GANs and discuss how our group has been using GANs for image editing.

State-of-the-art Natural Language Processing (NLP) systems nowadays are dominated by machine learning and deep learning models. However, most of these models often only work well when there are abundant labeled data for training. Furthermore, as these models typically have a large number of parameters, they require large compute resources to train. For the majority of languages in the world and the researchers working on these languages, however, abundant labeled data are a privilege and so do compute resources. How can we train generalizable NLP models that are effective even when labeled data are scarce and compute resources are limited? In this talk, I will present some of our solutions that leverage unsupervised or few-shot learning and readily available multilingual resources or multimodal data to improve machine translation and nuanced text classification such as news framing under these low resource settings.

The overarching goal of Prof. Michels' Computational Sciences Group within KAUST's Visual Computing Center is enabling accurate and efficient simulations for applications in Scientific and Visual Computing.

In this talk, I focus on the adoption of low latency persistent memory modules (PMMs). PMMs upend the long-established model of remote storage for distributed file systems. Instead, by colocating computation with PMM storage we can provide applications with much higher IO performance, sub-second application failover, and strong consistency. To demonstrate this, I present Assise, a new distributed file system, based on a persistent, replicated coherence protocol that manages client-local PMM as a linearizable and crash-recoverable cache between applications and slower (and possibly remote) storage.

In the first part of the talk, I will focus on ZygOS[SOSP 2017], a system optimized for μs-scale, in-memory computing on multicore servers. ZygOS implements a work-conserving scheduler within a specialized operating system designed for high request rates and a large number of network connections. ZygOS revealed the challenges associated with serving remote procedure calls (RPCs) on top of a byte-stream oriented protocol, such as TCP. In the second part of the talk, I will present R2P2[ATC 2019]. R2P2 is a transport protocol specifically designed for datacenter RPCs, that exposes the RPC abstraction to the endpoints and the network, making RPCs first-class datacenter citizens. R2P2 enables pushing functionality, such as scheduling, fault-tolerance, and tail-tolerance, inside the transport protocol, making it application-agnostic. I will show how using R2P2 allowed us to offload RPC scheduling to programmable switches that can schedule requests directly on individual cores.

This talk covers two research directions that address the shortcomings of existing network stacks. The first is on scalable software network stacks, solving problems in different components of operating systems and applications to allow a single server to handle data flows for tens of thousands of clients. The second is on Wide Area Network (WAN) congestion control, focusing on network-assisted congestion control schemes, where end-to-end solutions fail. The talk will conclude with a discussion of plans for future research in this area.

Biological systems are distinguished by their enormous complexity and variability. That is why mathematical modeling and computational simulation of those systems is very difficult, in particular thinking of detailed models which are based on first principles. The difficulties start with geometric modeling which needs to extract basic structures from highly complex and variable phenotypes, on the other hand also has to take the statistic variability into account. Moreover, the models of the processes running on these geometries are not yet well established, since these are equally complex and often couple many scales in space and time. Thus, simulating such systems always means to put the whole frame to test, from modelling to the numerical methods and software tools used for simulation. These need to be advanced in connection with validating simulation results by comparing them to experiments.

The overarching goal of Prof. Michels' Computational Sciences Group within KAUST's Visual Computing Center is enabling accurate and efficient simulations for applications in Scientific and Visual Computing. Towards this goal, the group develops new principled computational methods based on solid theoretical foundations. This talk covers a selection of previous and current work presenting a broad spectrum of research highlights ranging from simulating stiff phenomena such as the dynamics of fibers and textiles, over liquids containing magnetic particles, to the development of complex ecosystems and weather phenomena. Moreover, connection points to the growing field of machine learning are addressed and an outlook is provided with respect to selected technology transfer activities.

In this talk, I will give an overview of research done in the Image and Video Understanding Lab (IVUL) at KAUST. At IVUL, we work on topics that are important to the computer vision (CV) and machine learning (ML) communities, with emphasis on three research themes: Theme 1 (Video Understanding), Theme 2 (Visual Computing for Automated Navigation), Theme 3 (Fundamentals/Foundations).

In this talk, I will first give an overview of the research activities in Structural and Functional Bioinformatics Group (http://sfb.kaust.edu.sa). I will then focus on our efforts on developing computational methods to tackle key open problems in Nanopore sequencing. In particular, I will introduce our recent works on developing a collection of computational methods to decode raw electrical current signal sequences into DNA sequences, to simulate raw signals of Nanopore, and to efficiently and accurately align electrical current signal sequences with DNA sequences. I will further introduce their applications in biomedicine and healthcare.

The talk will present our decade-long efforts to build an integrated data-driven modeling system to study and predict the circulation and climate of the Arabian Peninsula at all scales. Starting from a general description of the Virtual Red Sea Initiative at its achievements so far, I will then outline our ongoing research under the KAUST Centre of Excellence for NEOM to develop new tools to seamlessly project and study the environment at the urban scales of NEOM. I will in particular discuss the involved scientific opportunities and challenges in terms of computational Sciences, including our extreme computational requirements, and the handling, analysis and visualization of very large datasets.

We propose a new optimization formulation for training federated learning models. The standard formulation has the form of an empirical risk minimization problem constructed to find a single global model trained from the private data stored across all participating devices. In contrast, our formulation seeks an explicit trade-off between this traditional global model and the local models, which can be learned by each device from its own private data without any communication. Further, we develop several efficient variants of SGD (with and without partial participation and with and without variance reduction) for solving the new formulation and prove communication complexity guarantees.