Towards Efficient AI Hardware: Software-Hardware Co-Design for In-Memory Computing Accelerators

Overview

The exponential growth of artificial intelligence (AI) applications has driven the demand for efficient, scalable, and energy-aware hardware capable of supporting deep neural networks (DNNs) across a wide range of platforms, from cloud servers to edge devices. In-memory computing (IMC) has emerged as a promising paradigm to overcome the inefficiencies of traditional von Neumann architectures by performing matrix-vector multiplications directly within memory arrays, thereby significantly reducing data movement and energy consumption. However, the effective deployment of AI models to IMC accelerators requires comprehensive co-optimization across the software–hardware stack and holistic co-design methodologies that jointly optimize software and hardware parameters to accommodate the diverse needs of modern AI workloads.

This dissertation addresses two key challenges in software–hardware co-design for IMC-based neural network accelerators: (1) the need to develop generalized IMC hardware that can efficiently support multiple neural network models, and (2) the need for automated frameworks that jointly optimize model parameters, quantization schemes, and IMC hardware configurations for workload-specific deployments. To tackle the first challenge, a joint hardware–workload co-optimization framework is proposed to generate generalized IMC hardware that supports multiple neural network models with minimal performance degradation. The framework incorporates a four-phase genetic algorithm, enhanced with Hamming-distance-based sampling, improving convergence speed and design diversity. To address the second challenge, this thesis introduces Compute-In-Memory Neural Architecture Search (CIMNAS), a unified framework that explores a vast multi-dimensional search space to generate highly efficient IMC designs tailored to specific workloads. It performs joint optimization across three domains: neural network architecture, quantization schemes, and IMC hardware configurations, exploring a broad hardware parameter space across device, circuit, architecture, and system levels.

The proposed approaches demonstrate significant improvements in hardware efficiency, scalability, and inference accuracy and are validated through extensive simulation results and comparisons with state-of-the-art frameworks. Together, these contributions advance the field of automated software–hardware co-design for IMC applications, enabling next-generation design methods for energy-efficient and generalizable AI accelerators.