Transcription factors are an important family of proteins that control the transcription rate from DNAs to messenger RNAs through the binding to specific DNA sequences. Transcription factor regulation is thus fundamental to understanding not only the system-level behaviors of gene regulatory networks, but also the molecular mechanisms underpinning endogenous gene regulation. In this talk, I will introduce our efforts on developing novel optimization and deep learning methods to quantitatively understanding transcription factor regulation at network- and molecular-levels. Specifically, I will talk about how we estimate the kinetic parameters from sparse time-series readout of gene circuit models, and how we model the relationship between the transcription factor binding sites and their binding affinities.

Dynamic programming is an efficient technique to solve optimization problems. It is based on decomposing the initial problem into simpler ones and solving these sub-problems beginning from the simplest ones. A conventional dynamic programming algorithm returns an optimal object from a given set of objects. We developed extensions of dynamic programming which allow us (i) to describe the set of objects under consideration, (ii) to perform a multi-stage optimization of objects relative to different criteria, (iii) to count the number of optimal objects,(iv) to find the set of Pareto optimal points for the bi-criteria optimization problem, and (v) to study the relationships between two criteria.

The goal of our research is to understand the physical origin of these behaviors and trasform them into sustainable technologies that tackle contemporary problem of global interest, ranging from energy harvesting to clean water production, design of smart materials, biomedical applications, information security, artificial intelligence, global warming, and so on. Creating technologies from complex natural systems is a modern interdisciplinary research field that permeates many different scientific areas, ranging from physics to mathematics, to engineering and the theory of linguistics. This is a very challenging, yet very promising research. It involves the understanding of what we consider complex, which translates as something “involved, intricate, complicated, not easily understood or analyzed”. This sets the challenge of being able to understand the mechanisms of these systems and cross many different disciplines in order to constructively harness their properties into reproducible applications.

An important stream of research in computational design aims at digital tools which support users in realizing their design intent in a simple and intuitive way, while simultaneously taking care of key aspects of function and fabrication. Such tools are expected to shorten the product development cycle through a reduction of costly feedback loops between design, engineering and fabrication. The strong coupling between shape generation, function and fabrication is a rich source for the development of new geometric concepts, with an impact to the original applications as well as to geometric theory. This will be illustrated at hand of applications in architecture and fabrication with a mathematical focus on discrete differential geometry and geometric optimization problems.

We developed and expanded novel methods for representation learning, predicting protein functions and their loss of function phenotypes. We use deep neural network algorithm and combine them with symbolic inference into neural-symbolic algorithms. Our work significantly improves previously developed methods for predicting protein functions through methodological advances in machine learning, incorporation of broader data types that may be predictive of functions, and improved systems for neural-symbolic integration. The methods we developed are generic and can be applied to other domains in which similar types of structured and unstructured information exist. In future, our methods can be applied to prediction of protein function for metagenomic samples in order to evaluate the potential for discovery of novel proteins of industrial value.  Also our methods can be applied to the prediction of loss of function phenotypes in human genetics and incorporate the results in a variant prioritization tool that can be applied to diagnose patients with Mendelian disorders.

The thesis focuses on the computation of high-dimensional multivariate normal (MVN) and multivariate Student-t (MVT) probabilities. Firstly, a generalization of the conditioning method for MVN probabilities is proposed and combined with the hierarchical matrix representation. Next, I revisit the Quasi-Monte Carlo (QMC) method and improve the state-of-the-art QMC method for MVN probabilities with block reordering, resulting in a ten-time-speed improvement. The thesis proceeds to discuss a novel matrix compression scheme using Kronecker products. This novel matrix compression method has a memory footprint smaller than the hierarchical matrices by more than one order of magnitude. A Cholesky factorization algorithm is correspondingly designed and shown to accomplish the factorization in 1 million dimensions within 600 seconds. To make the computational methods for MVN probabilities more accessible, I introduce an R package that implements the methods developed in this thesis and show that the package is currently the most scalable package for computing MVN probabilities in R. Finally, as an application, I derive the posterior properties of the probit Gaussian random field and show that the R package I introduce makes the model selection and posterior prediction feasible in high dimensions.

In this seminar, I will present some of the work I have done on Continual Deep Learning, among the research topics at the Vision-CAIR group. Continual learning aims to learn new tasks without forgetting previously learned ones. This is especially challenging when one cannot access data from previous tasks and when the model has a fixed capacity as adopted in modern deep learning techniques.  Decreasing the gap towards human-level continual learning, we extended continual deep learning from multiple perspectives. The Hebb's learning theory from biology can be famously summarized as “Cells that fire together wire together.". Inspired by this theory from biology, we proposed Memory Aware Synapses (ECCV18) to quantify and reduce machine forgetting in a way that enables leveraging unlabeled data, which was not possible in former techniques. We later developed a Bayesian approach appearing at ICLR2020, where we explicitly modeled uncertainty parameters to orchestrates forgetting in continual learning. We showed in our ICLR2019 and ACCV18 works that task descriptors/ language can operate in continual learning visual tasks to improve learning efficiency and enable zero-shot task transfer. Beyond computer vision tasks, we recently developed an approach appearing at ICLR2020 we call "Compositional Language Continual Learning". We showed that disentangling syntax from semantics enables better compositional Seq2Seq learning and can significantly alleviate forgetting of tasks like machine translation.  In the talk, I will go over these techniques and shed some light on future research possibilities.

In this seminar, the science of MOCVD, the material science of InGaN, and the new-born InGaN-based red LED performance will be discussed. The three primary colors in light are RGB. Green and blue LEDs have been realized by using InGaN active region. The current red LEDs are based on AlGaAs or InGaP as the active region. If we can realize red LEDs by InGaN, it is possible to integrate RGB LEDs in a wafer. Such RGB integration is a breakthrough to develop the next displays, so-called, micro-LED displays that are the next after the OLED displays, and functional LED lightings.

This dissertation is devoted to the fabrication and electrical and optical characterization of a new class of III-nitride light-emitter known as superluminescent diode (SLD). SLD works in an amplified spontaneous emission (ASE) regime, and it combines several advantages from both LD and LED, such as droop-free, speckle-free, low-spatial coherence, broader emission, high-optical power, and directional beam. Here, SLDs were fabricated by a focused ion beam by tilting the front facet of the waveguide to suppress the lasing mode. They showed a high-power of 474 mW on c-plane GaN-substrate with a large spectral bandwidth of 6.5 nm at an optical power of 105 mW. To generate SLD-based white light, a YAG-phosphor-plate was integrated, and a CRI of 85.1 and CCT of 3392 K were measured. For the VLC link, SLD showed record high-data rates of 1.45 Gbps and 3.4 Gbps by OOK and DMT modulation schemes, respectively. Additionally, a widely single- and dual-wavelength tunability were designed using SLD-based external cavity (SLD-EC) configuration for a tunable blue laser source.

This talk presents a new classification method for functional data. We consider the case where different groups of functions have similar means so that it is difficult to classify them based on only the mean function. To overcome this limitation, we propose the second moment based functional classifier (SMFC). Here, we demonstrate that the new method is sensitive to divergence in the second moment structure and thus produces lower rate of misclassification compared to other competitor methods. Our method uses the Hilbert-Schmidt norm to measure the divergence of second moment structure. One important innovation of our classification procedure lies in the dimension reduction step. The method data-adaptively discovers the basis functions that best capture the discrepancy between the second moment structures of the groups, rather than uses the functional principal component of each individual group, and good performance can be achieved as unnecessary variability is removed so that the classification accuracy is improved. Consistency properties of the classification procedure and the relevant estimators are established. Simulation study and real data analysis on phoneme and rat brain activity trajectories empirically validate the superiority of the proposed method.

In this thesis, efficient solutions are sought out to fundamental problems in Electromagnetic (EM) imaging that determines the shape, location, and material properties of an (unknown) object of interest in an investigation domain from the scattered field measured away from it. The solution of an EM inverse scattering problem inherently poses two main challenges: (i) non-linearity, since the scattered field is a non-linear function of the material properties and (ii) ill-posedness, since the integral operator has a smoothing effect and the number of measurements is finite in dimension and they are contaminated with noise. The non-linearity is tackled incorporating a multitude of techniques (ranging from Born approximation (linear), inexact Newton (linearized) to complete non-linear iterative Landweber schemes) that can account for weak to strong scattering problems. The ill-posedness of the EM inverse scattering problem is circumvented by formulating the above methods into a minimization problem with a sparsity constraint, which assumes that the dimension of the unknown object relative to the investigation domain is much smaller. Numerical experiments, which are carried out using synthetically generated measurements, show that the images recovered by these sparsity-regularized methods are sharper and more accurate than those produced by existing methods. The methods developed in this work have potential application areas ranging from oil/gas reservoir engineering to biological imaging where sparse domains naturally exist.

This talk discusses a body of work on exploiting the DSP Blocks in modern FPGAs to construct high-performance datapaths including the concept of FPGA overlays. It outlines work that established FPGAs as a viable virtualized cloud acceleration platform, and how the industry has adopted this model. Finally, it discusses recent work on incorporating accelerated processing in network controllers and the emerging concept of in-network computing with FPGAs. These strands of work come together to demonstrate the value of thinking about computing beyond the CPU-centric view that still dominates.

Underwater wireless optical communication (UWOC) has attracted increasing interest for data transfer in various underwater activities, due to its order-of-magnitude higher bandwidth compared to conventional acoustic and radio-frequency (RF) technologies. Our studies pave the way for eventual applications of UWOC by relieving the strict requirements on PAT using UV-based NLOS. Such modality is much sought-after for implementing robust, secure, and high-speed UWOC links in harsh oceanic environments. This work was first started with the investigation of proper NLOS configurations. Path loss (PL) was chosen as a figure-of-merit for link performance. The effects of NLOS geometries, water turbidity, and transmission wavelength are evaluated by measuring the corresponding PL. The experimental results suggest that NLOS UWOC links are favorable for smaller azimuth angles, stronger water turbidity, and shorter transmission wavelength, as exemplified by the use of 375-nm wavelength. With the understanding of favorable NLOS UWOC configurations, we established a NLOS link consisting of an ultraviolet (UV) laser as the transmitter for enhanced light scattering and high sensitivity photomultiplier tube (PMT) as the receiver. A high data rate of 85 Mbit/s using on-off keying (OOK) in a 30-cm emulated highly turbid harbor water is demonstrated. Besides the underwater communication links, UV-based NLOS is also appealing to be the signal carrier for direct communication across wavy water-air interface. The trial results indicate link stability, which alleviates the issues brought about by the misalignment and mobility in harsh environments, paving the way towards real applications.

In this dissertation, we aim at theoretically studying and analyzing deep learning models. Since deep models substantially vary in their shapes and sizes, in this dissertation, we restrict our work to a single fundamental block of layers that is common in almost all architectures. The block of layers of interest is the composition of an affine layer followed by a nonlinear activation function and then lastly followed by another affine layer. We study this block of layers from three different perspectives. (i) An Optimization Perspective. We try addressing the following question: Is it possible that the output of the forward pass through the block of layers highlighted above is an optimal solution to a certain convex optimization problem? As a result, we show an equivalency between the forward pass through this block of layers and a single iteration of certain types of deterministic and stochastic algorithms solving a particular class of tensor formulated convex optimization problems.

Traditional backstepping approaches may struggle to asymptotically stabilize systems in pure feedback form, due to its inherent implicit equations. Approximation based designs only have a limited domain of validity and turn out sensitive to model uncertainty and disturbances. We propose a new design that circumvents the necessity of solving implicit algebraic equations by introducing new state variables. Additional augmentations to the backstepping  Lyapunov design lead to explicit expressions for the associated differential equations. The result is a dynamic state feedback, capable of asymptotically stabilizing the origin of a general class of nonlinear systems, based on just standard assumptions.

In this talk, I will present a line of work done at the Image and Video Understanding Lab (IVUL), which focuses on developing deep graph convolutional networks (DeepGCNs). A GCN is a deep learning network that processes generic graph inputs, thus extending the impact of deep learning to irregular grid data including 3D point clouds and meshes, social graphs, protein interaction graphs, etc. By adapting architectural operations from the CNN realm and reformulating them for graphs, we were the first to show that GCNs can go as deep as CNNs. Developing such a high capacity deep learning platform for generic graphs opens up many opportunities for exciting research, which spans applications in the field of computer vision and beyond, architecture design, and theory. In this talk, I will showcase some of the GCN research done at IVUL and highlight some interesting research questions for future work.

In this talk, I will use the Empirical Risk Minimization (ERM) problem as an example and show how to overcome these challenges. Particularly, I will first talk about how to overcome the high dimensionality challenge from the data for Sparse Linear Regression in the local DP (LDP) model. Then, I will discuss the challenge from the non-interactive LDP model and show a series of results to reduce the exponential sample complexity of ERM. Next, I will present techniques on achieving DP for ERM with non-convex loss functions. Finally, I will discuss some future research along these directions.

This talk is about building compilers for high-performance code generation. It has three parts. The first part is about Tiramisu (http://tiramisu-compiler.org/), a polyhedral compiler designed for generating highly efficient code for multicores and GPUs. It is the first polyhedral compiler that can match the performance of highly hand-optimized industrial libraries such as Intel MKL and cuDNN. The second part is about applying Tiramisu to accelerate deep learning (DNN) inference. In comparison to other DNN compilers, Tiramisu has two unique features: (1) it supports sparse DNNs; and (2) it can express and optimize general RNNs (Recurrent Neural Networks). The third part will present recent work on the problem of automatic code optimization. In particular, it will focus on using deep learning to build a cost model to explore the search space of code optimizations.

In this talk, I will discuss a wide range of ideas on studying computer science and doing research in computer science. Peter Wonka is the program chair of the computer science program. His research interests are deep learning, computer graphics, computer vision, and remote sensing.

Backstepping is a widely applicable control technique based on Lyapunov theory that under rather mild assumptions leads to families of control laws for a large class of nonlinear systems. Focusing on systems of ordinary differential equations, we introduce the basic concept (integrator backstepping), generalize it, among others, to systems in strict feedback form and pure feedback form, which all enjoy an inherent controllability property, captured in the system structure. The course ends with extending the setting to the adaptive backstepping case, resorting to the certainty equivalence principle and Barbalat's lemma. The course is furnished by a series of exercises to let the students gather experience on tailored examples.

Structured media provide the momentum compensation for the scattering of waves. It is well-known that nano-scale modulations of the refractive index may lead to a temporal and spatial control over light propagation. Yet, also engineering the gain and loss profile uncovers analogous shaping effects. However, only the interplay between both the refractive index and gain and loss modulations introduces unidirectionality in light management. Thus, non-Hermitian optics has become one of the most fertile grounds in optics. A generalized Hilbert transform allows tailoring the two quadratures of the complex permittivity to design periodic or disordered non-Hermitian media, holding either global or local unidirectionality following arbitrary vector fields to tailor the flow of light. The method allows restricting the permittivity within realistic values rendering it suitable for applications.

In this talk, Tareq will present his research contributions and future directions to advance some critical IoT-enabling technologies: sensing, localization, and communications. He will demonstrate how we take advantage of structure in sensed-data and sensor arrays. This structure can help mitigate sensing uncertainties, improve localization accuracy, and enhance the performance of communication systems, all while reducing the computational overhead.  The Internet of Things (IoT) has ushered a new era in many fields including retail, medicine, agriculture, and the automotive industry. In fact, it is projected that by 2025, one trillion IoT devices will be deployed worldwide: the equivalent of 1000 devices per person. To reach such a scale, major advancements are needed in various IoT-enabling technologies.

Functional data analysis is a very active research area due to the overwhelming existence of functional data. In the first part of this talk, I will introduce how functional data depth is used to carry out exploratory data analysis and explain recently-developed depth techniques. In the second part, I will discuss spatio-temporal statistical modeling. It is challenging to build realistic space-time models and assess the validity of the model, especially when datasets are large. I will present a set of visualization tools we developed using functional data analysis techniques for visualizing covariance structures of univariate and multivariate spatio-temporal processes. I will illustrate the performance of the proposed methods in the exploratory data analysis of spatio-temporal data.

Backstepping is a widely applicable control technique based on Lyapunov theory that under rather mild assumptions leads to families of control laws for a large class of nonlinear systems. Focusing on systems of ordinary differential equations, we introduce the basic concept (integrator backstepping), generalize it, among others, to systems in strict feedback form and pure feedback form, which all enjoy an inherent controllability property, captured in the system structure. The course ends with extending the setting to the adaptive backstepping case, resorting to the certainty equivalence principle and Barbalat's lemma. The course is furnished by a series of exercises to let the students gather experience on tailored examples.

In this talk, we represent recent advances of radio frequency (RF) sensing technology for healthcare, specifically in monitoring human activities inside homes, retirement facilities, and hospitals. Sensing technologies and data analytics are considered powerful tools in efficient indoor monitoring of human activities.  Monitoring of activities of daily living (ADL) can identify falls, which are considered as the leading cause of fatal and non-fatal injuries for people aged 65 and over. It can also detect variants in activity patterns and changes in routines and lifestyle as well as the state of physical, cognitive, and psychological health of the person. In addition to monitoring ADL, RF-based gesture recognition using hands and arms is shown to be an important contactless technology for Man-Machine-Interface (MMI). Adding to the indoor applications, RF-based vital sign monitoring has vast medical use, as respiration and heartbeats are essential diagnostic barometers for many health problems. More recently, RF sensors have also been proposed for gait analysis for rehabilitation and timely diagnosis of many neurological, orthopedic and medical conditions. Changes in gait patterns can also be precursors of falls. In this talk, we present successful examples in each of the above application areas and discuss pertinent open problems worthy of investigations.