GPU Programming with CUDA Training Course

Introduction

• An overview of CUDA.
• Comparing CUDA with OpenCL and SYCL.
• Review of CUDA features and architecture.
• Setting up the development environment.

Getting Started

• Creating a new CUDA project using Visual Studio Code.
• Exploring the project structure and associated files.
• Compiling and executing the program.
• Displaying output using printf and fprintf.

CUDA API

• Understanding the function of the CUDA API within host programs.
• Querying device information and capabilities using the CUDA API.
• Managing device memory allocation and deallocation via the CUDA API.
• Transferring data between host and device using the CUDA API.
• Launching kernels and synchronizing threads with the CUDA API.
• Handling errors and exceptions using the CUDA API.

CUDA C/C++

• Understanding the role of CUDA C/C++ in device programs.
• Writing kernels in CUDA C/C++ to execute on the GPU and manipulate data.
• Using CUDA C/C++ data types, qualifiers, operators, and expressions.
• Leveraging CUDA C/C++ built-in functions, including math, atomic, and warp operations.
• Utilizing CUDA C/C++ built-in variables such as threadIdx, blockIdx, and blockDim.
• Employing CUDA C/C++ libraries like cuBLAS, cuFFT, and cuRAND.

CUDA Memory Model

• Distinguishing between host and device memory models.
• Utilizing CUDA memory spaces, including global, shared, constant, and local.
• Working with CUDA memory objects such as pointers, arrays, textures, and surfaces.
• Applying CUDA memory access modes, such as read-only, write-only, and read-write.
• Understanding the CUDA memory consistency model and synchronization mechanisms.

CUDA Execution Model

• Distinguishing between host and device execution models.
• Defining parallelism using CUDA threads, blocks, and grids.
• Using CUDA thread functions, such as threadIdx, blockIdx, and blockDim.
• Employing CUDA block functions, including __syncthreads and __threadfence_block.
• Utilizing CUDA grid functions, such as gridDim, gridSync, and cooperative groups.

Debugging

• Identifying common errors and bugs in CUDA programs.
• Inspecting variables, breakpoints, and call stacks using the Visual Studio Code debugger.
• Debugging CUDA programs on Linux using CUDA-GDB.
• Detecting memory errors and leaks with CUDA-MEMCHECK.
• Debugging and analyzing CUDA programs on Windows using NVIDIA Nsight.

Optimization

• Understanding factors that influence CUDA program performance.
• Improving memory throughput using CUDA coalescing techniques.
• Reducing memory latency with CUDA caching and prefetching techniques.
• Optimizing memory access and bandwidth using CUDA shared and local memory techniques.
• Measuring and enhancing execution time and resource utilization through CUDA profiling tools.