Heterogeneity in Hardware: Opportunities and Challenges for Software and Applications
With the end of Moore’s Law, the community has witnessed new hardware trends to increase performance. Today, it is not only the traditional x86 and accelerators that are part of computing systems, but also ARM, FPGAs and dedicated processors for DL workloads that equip now pioneering HPC systems. By the end of this decade, we are moving towards an era of extreme scale with “extreme heterogeneity”. This may result in systems built from a custom aggregation of components that may take us further away from the one hardware fits all paradigm.
The panel discusses the latest hardware evolution and the impact on HPC applications. The challenges reside in the increasing complexity of the underlying hardware. This urges to consider hardware/software co-design to facilitate the adoption of emerging technologies. The opportunities lie in designing new programming models, algorithmic innovations and performance tools to pursue the quest of scientific discovery.
With the end of Moore’s Law, the scientific community has witnessed new hardware trends to increase performance with emerging technologies. Today, it is not only the traditional x86 and hardware accelerators that are part of computing systems, but also ARM, FPGAs and dedicated processors for Deep Learning workloads that equip now pioneering HPC systems. By the end of this decade, it is clear that we are moving towards an era of extreme scale with “extreme heterogeneity”. This may result in systems built from a custom aggregation of components that may take us further away from the one hardware fits all paradigm.
How do we program these complex heterogenous hardware?
How do we ensure high productivity for developers of scientific software and applications?
How do we sustain performance of legacy codes that have survived previous waves of technology evolution?
The goal of the proposed panel is to discuss the latest hardware evolution (e.g., hardware support for low precisions, memory technology, power consumption) and how this may impact the development of HPC software libraries and applications (e.g., in terms of productivity, portability, scalability). The challenges reside in the increasing complexity of the underlying hardware. This urges to consider hardware/software co-design to facilitate the adoption of emerging technologies in software and applications. The opportunities lie in designing new programming models, algorithmic innovations and performance tools to pursue the quest of scientific discovery.
The attendees are invited to join the expert panelists, to learn from their experiences and discuss how to address the following questions moving forward:
- What is wrong with homogeneous hardware at the first place?
- How much programming efforts will heterogeneous hardware require? Will users need to make radical changes to their practices, methods, tools, and techniques to be able to exploit forthcoming modern resources and solve their scientific workflow more efficiently?
- How can the hardware complexity be further abstracted?
- Isn’t it time to consolidate programming models by largely adopting standard?
- How innovative algorithms can actually ease the programming of heterogeneous hardware?
- Why performance tools are even more critical in identifying bottleneck in heterogeneous architectures?
- How to develop sustainable competencies around selecting, implementing, and managing new technologies to support diverse workload?
- Are the current and projected developments of HPC systems and software aligned with the needs of scientific community?
The panel will be structured as follows:
- While waiting for the panel to start on time, the audience will be invited to complete a live short survey and suggest ideas they would like to be addressed during the panel.
- Each panel member is invited for a short presentation (8 minutes maximum), to discuss and answer questions about Heterogeneity in Hardware: Opportunities and Challenges for Software and Applications from their own perspectives. (50min)
- Then the interactive discussion with the audience will start by reviewing the feedbacks from the live short survey, followed by a Q&A session with microphones and social media interactions using SC21 web-application and Twitter. (40min).
 Report for DOE ASCR Basic Research Needs Workshop on Extreme Heterogeneity, 2018
List of Speakers
The list of confirmed panelists has been chosen to cover distinctive expertise coming from diverse institution types (i.e., world leading HPC companies, universities and national labs), geographical locations (USA, France, China and Japan), demographic characteristics (young talented researchers along with senior experts) and gender equity. We will expose antagonist perspectives in the context of hardware technology and software development. The panelists are experts in hardware and software with strong industrial and academic backgrounds. They are well known for creating excitement and drawing a large audience at different HPC related conferences.
Anima Anandkumar holds dual positions in academia and industry. She is a Bren professor at Caltech CMS department and a director of machine learning research at NVIDIA. She has spearheaded the development of tensor algorithms that are central to effectively processing multidimensional and multimodal data, and for achieving massive parallelism in large-scale AI applications. She is recipient of several awards such as the Alfred. P. Sloan Fellowship, NSF Career Award, Faculty fellowships from Microsoft, Google and Adobe, Young Investigator Awards from the Army research office and Air Force office of sponsored research, and Women in AI by Venturebeat.
Laura Grigori is a senior research scientist at INRIA in France, where she leads the Alpines group, a joint group between INRIA and the J.L. Lions Laboratory, Sorbonne University, in Paris. She leads several projects on preconditioning, communication avoiding algorithms and associated numerical libraries for large scale parallel/multicore machines. After postdoctoral research at the University of California, Berkeley and the Lawrence Berkeley National Laboratory, she became a researcher for INRIA in 2004, and became the head of the Alpines project in 2013. In 2020 Grigori was named a SIAM Fellow "for contributions to numerical linear algebra, including communication-avoiding algorithms". In 2021, she joined the SIAM Council as a Member-at-Large.
Patricia Damkroger is vice president and general manager of the High Performance Computing organization in the Data Platforms Group at Intel Corporation. She leads Intel’s global technical and HPC business and is responsible for developing and executing strategy, building customer relationships and defining a leading product portfolio for technical computing workloads, including emerging areas such as high-performance data analytics, HPC in the cloud and artificial intelligence. An expert in the HPC field, Damkroger has more than 27 years of technical and managerial expertise both in the private and public sectors. Prior to joining Intel in 2016, she was the associate director of computation at the U.S. Department of Energy’s Lawrence Livermore National Laboratory.
Jack Dongarra holds an appointment at the University of Tennessee, Oak Ridge National Laboratory, and the University of Manchester. He specializes in numerical algorithms in linear algebra, parallel computing, use of advanced-computer architectures, programming methodology, and tools for parallel computers. He was awarded the IEEE Sidney Fernbach Award in 2004; in 2008 he was the recipient of the first IEEE Medal of Excellence in Scalable Computing; in 2010 he was the first recipient of the SIAM Special Interest Group on Supercomputing's award for Career Achievement; in 2011 he was the recipient of the IEEE Charles Babbage Award; and in 2013 he received the ACM/IEEE Ken Kennedy Award. He is a Fellow of the AAAS, ACM, IEEE, and SIAM and a foreign member of the Russian Academy of Science and a member of the US National Academy of Engineering.
James Lin is the co-founder and has been the vice director of High Performance Computing Center at Shanghai Jiao Tong University, one of the leading university-level supercomputer center in China, since 2012. His current research interests include performance analysis at micro-architectural level for emerging many-core processors, and large-scale applications on supercomputers. He has served as a steering committee for IEEE CLUSTER and a standing committee member for CCF TCHPC. He also served as a TPC member and reviewer in many HPC conferences and journals, including SC, IPDPS, and Transaction on Computer (TC). He is a senior member of the ACM and CCF.
Satoshi Matsuoka is he director of RIKEN R-CCS, the top-tier HPC center in Japan which operates the K Computer and will host its successor Supercomputer Fugaku, and he is a Specially Appointed Professor at Tokyo Tech since 2018. He had been a Full Professor at the Global Scientific Information and Computing Center (GSIC), Tokyo Institute of Technology, since 2000, where he has been the leader of the TSUBAME series of supercomputers that have won many accolades such as world #1 in power-efficient computing. Satoshi Matsuoka also leads various major supercomputing research projects in areas such as parallel algorithms and programming, resilience, green computing, and convergence of Big Data/AI with HPC. He has written over 500 articles, chaired numerous ACM/IEEE conferences, and has won many awards, such as the ACM Gordon Bell Prize in 2011 and the highly prestigious 2014 IEEE-CS Sidney Fernbach Memorial Award.