Towards Trustworthy Acceleration for AI Systems

Overview

AI systems increasingly rely on acceleration throughout inference: context compression shortens inputs, key-value (KV) cache reuse avoids recomputing attention states, and speculative decoding lets a small model draft tokens for a larger model to verify. These techniques aim to reduce computation, memory, latency, and cost while preserving task utility. But do they also preserve safety and security?

I will begin with a security-utility asymmetry in lossy speculative decoding: as speedups increase, safety and security degrade earlier and faster than coding and reasoning performance. I will present a theoretical framework that explains this asymmetry and guides the design of systems that improve security while retaining efficiency. Turning to KV cache reuse, I will show how attackers can exploit it to carry adversarial influence across contexts and hijack other users’ outputs. Finally, I will share my vision for trustworthy acceleration: understanding which computations can be safely approximated or reused during inference optimization, and designing efficient AI systems with security as a first-class design objective.