Heterogeneity in Hardware: Opportunities and Challenges for Software and Applications (SC21 Panel)
-
- Bilel Hadri, Computational Scientist, Supercomputing Lab, KAUST
SC21 Saint Louis MO USA
Overview
Heterogeneity in Hardware: Opportunities and Challenges for Software and Applications
SC21 Panel
Time: Tuesday, 16 November 2021, 10:30am - 12pm CST
Location: 225-226 (America's Center, St. Louis, MO USA)
Abstract
With the end of Moore’s Law, the community has witnessed new hardware trends to increase performance. Today, it is not only the traditional x86 and accelerators that are part of computing systems, but also ARM, FPGAs and dedicated processors for DL workloads that equip now pioneering HPC systems. By the end of this decade, we are moving towards an era of extreme scale with “extreme heterogeneity”. This may result in systems built from a custom aggregation of components that may take us further away from the one hardware fits all paradigm.
The panel discusses the latest hardware evolution and the impact on HPC applications. The challenges reside in the increasing complexity of the underlying hardware. This urges to consider hardware/software co-design to facilitate the adoption of emerging technologies. The opportunities lie in designing new programming models, algorithmic innovations and performance tools to pursue the quest of scientific discovery.
Agenda
With the end of Moore’s Law, the scientific community has witnessed new hardware trends to increase performance with emerging technologies. Today, it is not only the traditional x86 and hardware accelerators that are part of computing systems, but also ARM, FPGAs and dedicated processors for Deep Learning workloads that equip now pioneering HPC systems. By the end of this decade, it is clear that we are moving towards an era of extreme scale with “extreme heterogeneity”. This may result in systems built from a custom aggregation of components that may take us further away from the one hardware fits all paradigm.
How do we program these complex heterogenous hardware?
How do we ensure high productivity for developers of scientific software and applications?
How do we sustain performance of legacy codes that have survived previous waves of technology evolution?
The goal of the proposed panel is to discuss the latest hardware evolution (e.g., hardware support for low precisions, memory technology, power consumption) and how this may impact the development of HPC software libraries and applications (e.g., in terms of productivity, portability, scalability). The challenges reside in the increasing complexity of the underlying hardware. This urges to consider hardware/software co-design to facilitate the adoption of emerging technologies in software and applications. The opportunities lie in designing new programming models, algorithmic innovations and performance tools to pursue the quest of scientific discovery.
The attendees are invited to join the expert panelists, to learn from their experiences and discuss how to address the following questions moving forward:
- What is wrong with homogeneous hardware at the first place?
- How much programming efforts will heterogeneous hardware require? Will users need to make radical changes to their practices, methods, tools, and techniques to be able to exploit forthcoming modern resources and solve their scientific workflow more efficiently?
- How can the hardware complexity be further abstracted?
- Isn’t it time to consolidate programming models by largely adopting standard?
- How innovative algorithms can actually ease the programming of heterogeneous hardware?
- Why performance tools are even more critical in identifying bottleneck in heterogeneous architectures?
- How to develop sustainable competencies around selecting, implementing, and managing new technologies to support diverse workload?
- Are the current and projected developments of HPC systems and software aligned with the needs of scientific community?
The panel will be structured as follows:
- While waiting for the panel to start on time, the audience will be invited to complete a live short survey and suggest ideas they would like to be addressed during the panel.
- Each panel member is invited for a short presentation (8 minutes maximum), to discuss and answer questions about Heterogeneity in Hardware: Opportunities and Challenges for Software and Applications from their own perspectives. (50min)
- Then the interactive discussion with the audience will start by reviewing the feedbacks from the live short survey, followed by a Q&A session with microphones and social media interactions using SC21 web-application and Twitter. (40min).
List of Speakers
The list of confirmed panelists has been chosen to cover distinctive expertise coming from diverse institution types (i.e., world leading HPC companies, universities and national labs), geographical locations (USA, France, China and Japan), demographic characteristics (young talented researchers along with senior experts) and gender equity. We will expose antagonist perspectives in the context of hardware technology and software development. The panelists are experts in hardware and software with strong industrial and academic backgrounds. They are well known for creating excitement and drawing a large audience at different HPC related conferences.
|
Anima Anandkumar holds dual positions in academia and industry. She is a Bren professor at Caltech CMS department and a director of machine learning research at NVIDIA. She has spearheaded the development of tensor algorithms that are central to effectively processing multidimensional and multimodal data, and for achieving massive parallelism in large-scale AI applications. She is recipient of several awards such as the Alfred. P. Sloan Fellowship, NSF Career Award, Faculty fellowships from Microsoft, Google and Adobe, Young Investigator Awards from the Army research office and Air Force office of sponsored research, and Women in AI by Venturebeat. |
|
Laura Grigori is a senior research scientist at INRIA in France, where she leads the Alpines group, a joint group between INRIA and the J.L. Lions Laboratory, Sorbonne University, in Paris. She leads several projects on preconditioning, communication avoiding algorithms and associated numerical libraries for large scale parallel/multicore machines. After postdoctoral research at the University of California, Berkeley and the Lawrence Berkeley National Laboratory, she became a researcher for INRIA in 2004, and became the head of the Alpines project in 2013. In 2020 Grigori was named a SIAM Fellow "for contributions to numerical linear algebra, including communication-avoiding algorithms". In 2021, she joined the SIAM Council as a Member-at-Large. |
|
Robert W. Wisniewski is an ACM Distinguished Scientist, IEEE Senior Member, the CTO and Chief Architect for High Performance Computing and a Senior Principal Engineer at Intel Corporation. He is the technical lead and PI for Aurora, the supercomputer to be delivered to Argonne National Laboratory that will achieve greater than an exaflop of computation. Before coming to Intel, he was the chief software architect for Blue Gene Research and manager of the Blue Gene and Exascale Research Software Team at the IBM T.J. Watson Research Facility, where he was an IBM Master Inventor and led the software effort on Blue Gene/Q, which received the National Medal of Technology and Innovation, was the most powerful computer in the world in June 2012, and occupied 4 of the top 10 positions on Top 500 list. |
|
Piotr Luszczek is a Research Assistant Professor at the Innovative Computing Laboratory in University of Tennessee, Knoxville's Department of Electrical Engineering and Computer Science. Piotr earned MSc in Computer Science from AGH University of Science and Technology in Krakow, Poland, and PhD in Computer Science from the University of Tennessee Knoxville. Piotr's research interests include benchmarking, numerical linear algebra for high-performance computing, automatic performance tuning for hardware accelerators, and stochastic models for performance. Piotr has over a decade or experience developing high performance numerical software for large scale, distributed memory systems with multicore processors and GPU accelerators. His main research interest include benchmarking and automated performance tuning. Currently, Piotr serves as co-PI for the ECP xSDK project that has a main goal to improve access to the world-class software on the Exascale machines. In the course of his career, he achieved Google Scholar h-index of 33 and i10-index of 91. |
|
James Lin is the co-founder and has been the vice director of High Performance Computing Center at Shanghai Jiao Tong University, one of the leading university-level supercomputer center in China, since 2012. His current research interests include performance analysis at micro-architectural level for emerging many-core processors, and large-scale applications on supercomputers. He has served as a steering committee for IEEE CLUSTER and a standing committee member for CCF TCHPC. He also served as a TPC member and reviewer in many HPC conferences and journals, including SC, IPDPS, and Transaction on Computer (TC). He is a senior member of the ACM and CCF. |
|
Satoshi Matsuoka is the director of RIKEN R-CCS, the top-tier HPC center in Japan which operates the K Computer and will host its successor Supercomputer Fugaku, and he is a Specially Appointed Professor at Tokyo Tech since 2018. He had been a Full Professor at the Global Scientific Information and Computing Center (GSIC), Tokyo Institute of Technology, since 2000, where he has been the leader of the TSUBAME series of supercomputers that have won many accolades such as world #1 in power-efficient computing. Satoshi Matsuoka also leads various major supercomputing research projects in areas such as parallel algorithms and programming, resilience, green computing, and convergence of Big Data/AI with HPC. He has written over 500 articles, chaired numerous ACM/IEEE conferences, and has won many awards, such as the ACM Gordon Bell Prize in 2011 and the highly prestigious 2014 IEEE-CS Sidney Fernbach Memorial Award. |