Extreme Computing Universals
This talk redefines "extreme" computing as operating under severe resource constraints rather than just massive scale, outlining universal algorithmic, hardware, and system-level strategies to overcome these challenges, illustrated by KAUST success stories.
Overview
“Extreme” in computing means more than large in scale. It can indicate constraints of real-time, low power, low memory capacity or bandwidth per core, or low available concurrency. Some universals in dealing with extremes include: reside high on the memory hierarchy (e.g., by blocking or processing on the fly), reduce synchrony in frequency and/or in span (e.g., by performing extra flops), reduce communication in number and/or volume of messages (e.g., by exploiting extra memory), employ dynamic scheduling and balancing (e.g., by runtime systems based on DAGs), avoid over-resolving with respect to output accuracy requirements (e.g., adapt precision, fidelity, and inner tolerances), reformulate applications before computing (e.g., with smarter bases or discretizations), co-design algorithms with hardware (e.g., specialized heterogeneity in processing, memory, and networking elements), exploit the “right to re-order” (e.g., linearization vs partitioning, lagged evaluations, colorings), exploit hierarchical or multiple alternative versions of the same system, exploit data sparsity to meet “curse of dimensionality” with “blessing of low rank”, take resilience into algorithms, relieving hardware and systems, look “over the transoms” for optimizations beyond optimized components, code to specialized back-ends while presenting high-level APIs to users, consider “science per Joule.” Most are classical but have new significance within and beyond classical simulation. We provide examples from KAUST success stories and welcome others.
Presenters
Brief Biography
David Keyes is a professor in the Applied Mathematics and Computational Sciences, Computer Science, and Mechanical Engineering programs. He served as a founding dean of the Mathematical and Computer Sciences and Engineering Division from 2009 to 2012 and as the director of the strategic initiative and ultimately the Research Center in Extreme Computing from 2013 to 2024. He is also an adjunct professor and former Fu Foundation Chair Professor of Applied Physics and Applied Mathematics at Columbia University, and a faculty affiliate of several laboratories of the U.S. Department of Energy.
Professor Keyes is Fellow of the Society for Industrial and Applied Mathematics (SIAM), the American Mathematical Society (AMS), and of the American Association for the Advancement of Science (AAAS). He is the recipient of the SIAM Prize for Distinguished Service to the Profession (2011), the Distinguished Faculty Teaching Award of Columbia University (2008), the Sidney Fernbach Award of IEEE Computer Society (2007), and the ACM Gordon Bell Prize (1999), and the Prize for Teaching Excellence in the Natural Sciences of Yale University (1991) .
Keyes graduated summa cum laude in Aerospace and Mechanical Sciences with a certificate in Engineering Physics from Princeton in 1978 and earned a doctorate in Applied Mathematics from Harvard in 1984. He was a Research Associate in Computer Science at Yale University 1984-1985, and has had decadal research appointments at the Institute for Computer Applications in Science and Engineering (ICASE), NASA-Langley Research Center, and the Institute for Scientific Computing Research (ISCR), Lawrence Livermore National Laboratory.
