Demystifying Adversarial Patch Attacks and Defenses in the Physical World
In this talk, I will introduce a series of adversarial patch attacks in face recognition systems and autonomous driving cars, and present our recent studies in developing a zero-shot and patch-agnostic defense framework.
Overview
Adversarial patch attacks have become one of the most ubiquitous and urgent threats to state-of-the-art computer vision systems that are widely deployed in real-world platforms, such as critical surveillance cameras and autonomous vehicles. Specifically, these attacks are launched by embedding adversarial perturbations into small and purposely located patches, allowing the adversary to directly disrupt or mislead model predictions in digital and physical environments, such as hiding and altering target objects to evade vision-based detection systems.
In this talk, I will reveal the prior knowledge in adversarial patches by characterizing the causality, generation, and effectiveness, and illustrate the perturbations through modelling and visualization. Specifically, I will introduce a series of adversarial patch attacks in face recognition systems and autonomous driving cars that adopt diverse mechanisms to generate the patches and successfully spoof various systems in real-world scenarios. Then, I will present our recent studies in developing a zero-shot and patch-agnostic defense framework that can detect, localize, and purify adversarial patches and restore original images while outperforming existing defense solutions.
Presenters
Brief Biography
Dr. Tao Ni (Tony) is an Assistant Professor of Computer Science in the Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division at King Abdullah University of Science and Technology (KAUST), where he is also affiliated with the KAUST Center of Excellence for Generative AI. His research lies at the intersection of cybersecurity and AI and focuses on the security and trustworthiness of AI-driven systems, from large language and vision-language models and the autonomous agents built on them to the embodied systems, such as robots, extended reality (XR) devices, and IoT platforms, in which these models increasingly perceive, decide, and act. His work spans AI safety and AI for security (adversarial attacks and defenses, LLM/VLM security, and agentic AI security), the security of AI-driven cyber-physical and embodied systems (robotics, XR, and IoT security), and low-power mobile computing. Through his work, he reveals how AI models and the physical systems they control can be manipulated, eavesdropped on, or hijacked through adversarial inputs, side channels, and compromised sensors. He also develops practical, system-level defenses, ranging from zero-shot protection against adversarial patches to secure and efficient on-device LLM inference, that safeguard these systems without sacrificing efficiency. His broader goal is to make the coming generation of agentic and embodied AI systems secure and trustworthy by design, so that they can be safely deployed at scale in everyday devices and mission-critical settings.
Dr. Ni has collaborated extensively with leading research institutions in the United States, Europe, Singapore, and Hong Kong, including National University of Singapore, George Mason University, TU Delft, and City University of Hong Kong. These collaborations have produced research at the leading venues in computer security, mobile computing, and machine learning, including IEEE S&P, ACM CCS, USENIX Security, NDSS, ACM MobiCom, ACM MobiSys, ACM MobiHoc, ICML, NeurIPS, and ACM UIST, as well as in journals such as IEEE TDSC, IEEE TMC, and IEEE TSC. At KAUST, he teaches Computer Systems and AI Security and contributes to the KAUST Academy's Master of Engineering (MEng) in Cybersecurity program delivered for Saudi Arabia’s Ministry of Interior, helping build national capacity in cybersecurity.
In recognition of his research contributions, Dr. Ni was named an ACM MobiSys Rising Star in 2024 and an ACM/IEEE AIoTSys Rising Star in 2025, where he also received the Best Presentation Award. His work has received international recognition, including the Springer Cybersecurity Best Practical Paper Award (2024) for his ACM CCS 2023 paper, the Best Paper Award at ML4CS (2026), the People's Choice ASSET Poster Award at ACM MobiSys (2024), and City University of Hong Kong’s Outstanding Research Thesis Award (2025) for his doctoral dissertation. He serves as an Associate Editor of IEEE Transactions on Services Computing (TSC) and IEEE Transactions on Consumer Electronics (TCE), on the Editorial Board of IEEE Network Magazine, and on the Distinguished Reviewer Board of ACM Transactions on Sensor Networks (TOSN), and he regularly serves on the program committees of top-tier cybersecurity and mobile computing conferences, including USENIX Security, NDSS, ACM MobiSys, ACM MobiCom, ACM SenSys, RAID, and ACSAC.
Before joining the KAUST faculty in 2025, Dr. Ni was a postdoctoral researcher at the City University of Hong Kong (CityUHK). He earned his Ph.D. in Computer Science from CityUHK in 2024, a Master of Computing from the Australian National University (ANU) in 2020, and a Bachelor of Engineering in Electrical Engineering from Shanghai Jiao Tong University in 2018.
Aligned with Saudi Vision 2030 and the Kingdom's target to lead in AI and cybersecurity, Dr. Ni continues to advance the security of agentic and embodied AI systems, fostering interdisciplinary collaboration across security, machine learning, and robotics, and translating rigorous attack and defense research into practical protections for the AI-driven systems at the heart of the Kingdom’s digital transformation.
