Deep neural networks (DNNs) are revolutionizing computing, necessitating an integrated approach across the computing stack to optimize efficiency. In this talk, I will explore the frontier of DNN optimization, spanning algorithms, software, and hardware. We'll start with hardware-aware neural architecture search, demonstrating how tailoring DNN architectures to specific hardware can drastically enhance performance.