R-CCS Cafe - Special Edition (Jul 26, 2024)

Title:
High Performance Kernel Code Generation Using Generative AI
Abstract:
Generative Artificial Intelligence (AI) technologies, such as GPT and Llama, have shown promise in facilitating code generation across a variety of programming languages. However, the domain of high-performance scientific computing, which demands specialized expertise, presents unique challenges that have led to mixed results in terms of both performance and correctness when applying Generative AI. This presentation will delve into our experiments with employing Generative AI to develop established high-performance computing kernels, such as AXPY, GEMV, and GEMM. We examine the deployment of these AI models across various parallel programming models and languages, including C++ (with OpenMP, OpenMP Offload, OpenACC, CUDA, HIP), Fortran (utilizing OpenMP, OpenMP Offload, OpenACC), Python (via numpy, Numba, pyCUDA, cuPy), and Julia (through Threads, CUDA.jl, AMDGPU.jl). Our analysis aims to assess the efficacy and correctness of Generative AI in generating scientific computing kernels, as well as its adaptability to the specialized requirements of high-performance scientific computing. Through this exploration, we intend to illuminate the potential of Generative AI as a tool for innovation within scientific computing, highlighting its capabilities and identifying its challenges that need to be overcome to fully leverage its potential.
This is a joint work with Pedro Valero-Lara, William Godoy, Prasana Balaprakash, and Jeffery Vetter at ORNL.