From my experience, I will try to answer doubts and dilemmas PhD students are often faced with, in their path towards a degree. Namely, I'll discuss how advisors, colleagues, peers, reviewers and so forth, fit in the universe of a PhD student, and I will end sharing my own definition of 'excellence', as an objective to pursue.

Skoltech Applied AI center’s mission is to create AI models and frameworks for solving the problems of sustainable development of industry and economy. In my presentation, I will overview the current center's activities, applied and fundamental problem statements, and corresponding recent results.

In the big data era, it is necessary to rely on distributed computing. For distributed optimization and learning tasks, in particular in the modern paradigm of federated learning, specific challenges arise, such as decentralized data storage. Communication between the parallel machines and the orchestrating distant server is necessary but slow. To address this main bottleneck, a natural strategy is to compress the communicated vectors. I will present EF-BV, a new algorithm which converges linearly to an exact solution, with a large class of deterministic or random, biased or unbiased compressors.

Random fields are popular models in statistics and machine learning for spatially dependent data on Euclidian domains. However, in many applications, data is observed on non-Euclidian domains such as street networks. In this case, it is much more difficult to construct valid random field models. In this talk, we discuss some recent approaches to modeling data in this setting, and in particular define a new class of Gaussian processes on compact metric graphs.

This talk will focus on challenges and opportunities in designing middleware for HPC, AI (Deep/Machine Learning), and Data Science. We will start with the challenges in designing runtime environments for MPI+X programming models by considering support for multi-core systems, high-performance networks (InfiniBand and RoCE), GPUs, and emerging BlueField-2 DPUs. Features and sample performance numbers of using the MVAPICH2 libraries will be presented. For the Deep/Machine Learning domain, we will focus on MPI-driven solutions to extract performance and scalability for popular Deep Learning frameworks (TensorFlow and PyTorch), large out-of-core models, and Bluefield-2 DPUs.

Machine perception that corresponds to the ability to understand the visual world based on the input from sensors, such as cameras is one of the central problems in Artificial Intelligence. To this end, recent years have witnessed tremendous progress in various instance-level recognition tasks having real-world applications in e.g., robotics, autonomous driving and surveillance. In this talk, I will first present our recent results towards understanding state-of-the-art deep learning-based visual recognition networks in terms of their robustness and generalizability. Next, I will present our results on learning visual recognition models with limited human supervision. Finally, I will discuss moving one step further from instance-level recognition to understand visual relationships between object pairs.

In this thesis, we discuss a few fundamental and well-studied optimization problem classes: decentralized distributed optimization (Chapters 2 to 4), distributed optimization under similarity (Chapter 5), affinely constrained optimization (Chapter 6), minimax optimization (Chapter 7), and high-order optimization (Chapter 8). For each problem class, we develop the first provably optimal algorithm: the complexity of such an algorithm cannot be improved for the problem class given. The proposed algorithms show state-of-the-art performance in practical applications, which makes them highly attractive for potential generalizations and extensions in the future.

In this talk, I will discuss communication compression and aggregation mechanisms for curvature information in order to reduce these costs while preserving theoretically superior local convergence guarantees.

This talk presents some perspectives on doing research in computer science and publishing through the peer review process.

Federated learning (FL) is an emerging machine learning paradigm involving multiple clients, e.g., mobile phone devices, with an incentive to collaborate in solving a machine learning problem coordinated by a central server. FL was proposed in 2016 by Konecny et al. and McMahan et al. as a viable privacy-preserving alternative to traditional centralized machine learning since, by construction, the training data points are decentralized and never transferred by the clients to a central server. Therefore, to a certain degree, FL mitigates the privacy risks associated with centralized data collection. Unfortunately, optimization for FL faces several specific issues that centralized optimization usually does not need to handle. In this thesis, we identify several of these challenges and propose new methods and algorithms to address them, with the ultimate goal of enabling practical FL solutions supported with mathematically rigorous guarantees.

Datasets that capture the connection between vision, language, and affection are limited, causing a lack of understanding of the emotional aspect of human intelligence. As a step in this direction, the ArtEmis dataset was recently introduced as a large-scale dataset of emotional reactions to images along with language explanations of these chosen emotions.

Hydrogen is a carbon-free energy carrier that can be used to decarbonize various high-emitting sectors, such as transportation, power generation, and industry. Today, global hydrogen production is largely derived from fossil fuels such as natural gas and coal.

Differential Privacy (DP) allows for rich statistical and machine learning analysis, and is now becoming a gold standard for private data analysis. Despite the noticeable success of this theory, existing tools from DP are severely limited to regular datasets, e.g., datasets need to be or are assumed to be clean and normalized before performing DP algorithms.

The power system is facing unprecedented changes in operation and control as more and diverse sources and loads are being connected to this complex cyber-physical energy system.

Geospatial health data are essential to inform public health and policy. These data can be used to quantify disease burden, understand geographic and temporal patterns, identify risk factors and measure inequalities. In this talk, I will give an overview of statistical methods and computational tools for geospatial data analysis and health surveillance.

DNA Nanotechnology is a fascinating field that studies how to construct small biological structures entirely from DNA as a building material. The key insight is that DNA, if designed in a particular way, can construct complex 3D nanoscale structures entirely by means of self-assembly, governed by the base-pairing principle.

Traditional computing systems separate processors from memory, performing computation by shuttling data back and forth between these two units all the time. This bottleneck incurs limited processing speed and high power consumption in computing systems for deep learning models of ever-increasing complexity. Novel approaches and new principles are needed to revolutionize computing systems. Neuromorphic systems are proposed as a new computing architecture based on spiking neural networks analogous to the existing nervous systems.

We study the MARINA method of Gorbunov et al (ICML 2021) - the current state-of-the-art distributed non-convex optimization method in terms of theoretical communication complexity. Theoretical superiority of this method can be largely attributed to two sources: the use of a carefully engineered biased stochastic gradient estimator, which leads to a reduction in the number of communication rounds, and the reliance on {\em independent} stochastic communication compression operators, which leads to a reduction in the number of transmitted bits within each communication round.

Mobile computing proliferation is critically threatened by the breakdown of Dennard scaling, a law describing the area-proportional growth of integrated circuit power use.

The qualitative study of PDEs often relies on integral identities and inequalities. For example, for time-dependent PDEs, conserved integral quantities or quantities that are dissipated play an important role. In particular, if these integral quantities have a definite sign, they are of great interest as they may provide control on the solutions to establish well-posedness.

Dynamic programming is an efficient technique to solve optimization problems. It is based on decomposing the initial problem into simpler ones and solving these sub-problems beginning from the simplest ones.

The talk will discuss the challenging problem of designing Deep Neural Network systems that achieve high performance under low power envelopes, hindering their deployment in the embedded space.

The life sciences have invested significant resources in the development and application of semantic technologies to make research data accessible and interlinked, and to enable the integration and analysis of data. Utilizing the semantics associated with research data in data analysis approaches is often challenging. Now, novel methods are becoming available that combine symbolic methods and statistical methods in Artificial Intelligence. In my talk, I will show how to incorporate biological background knowledge in machine learning models for identification of gene-disease associations, genomic variants that are causative for heritable disorders, and to predict protein functions. The methods I describe are generic and can be applied in other domains in which biomedical ontologies and structured knowledge bases exist.