GPU Programming with CUDA Training Course

Introduction

• Introduction to CUDA
• Comparison: CUDA vs OpenCL vs SYCL
• Overview of CUDA features and architecture
• Setting up the Development Environment

Getting Started

• Creating a new CUDA project in Visual Studio Code
• Exploring project structure and files
• Compiling and executing the program
• Displaying output using printf and fprintf

CUDA API

• Understanding the role of the CUDA API in the host program
• Querying device information and capabilities via the CUDA API
• Allocating and deallocating device memory using the CUDA API
• Copying data between host and device with the CUDA API
• Launching kernels and synchronizing threads through the CUDA API
• Handling errors and exceptions using the CUDA API

CUDA C/C++

• Understanding the role of CUDA C/C++ in the device program
• Writing kernels for GPU execution and data manipulation in CUDA C/C++
• Using CUDA C/C++ data types, qualifiers, operators, and expressions
• Utilizing CUDA C/C++ built-in functions, such as math, atomic, and warp operations
• Using CUDA C/C++ built-in variables, such as threadIdx, blockIdx, and blockDim
• Working with CUDA C/C++ libraries, such as cuBLAS, cuFFT, and cuRAND

CUDA Memory Model

• Distinguishing between host and device memory models
• Utilizing CUDA memory spaces, including global, shared, constant, and local
• Using CUDA memory objects, such as pointers, arrays, textures, and surfaces
• Applying CUDA memory access modes, such as read-only, write-only, and read-write
• Understanding the CUDA memory consistency model and synchronization mechanisms

CUDA Execution Model

• Distinguishing between host and device execution models
• Defining parallelism using CUDA threads, blocks, and grids
• Using CUDA thread functions, such as threadIdx, blockIdx, and blockDim
• Utilizing CUDA block functions, such as __syncthreads and __threadfence_block
• Employing CUDA grid functions, such as gridDim, gridSync, and cooperative groups

Debugging

• Identifying common errors and bugs in CUDA programs
• Inspecting variables, breakpoints, and call stacks using the Visual Studio Code debugger
• Debugging CUDA programs on Linux with CUDA-GDB
• Detecting memory errors and leaks using CUDA-MEMCHECK
• Debugging and analyzing CUDA programs on Windows with NVIDIA Nsight

Optimization

• Understanding factors that influence CUDA program performance
• Improving memory throughput with CUDA coalescing techniques
• Reducing memory latency using CUDA caching and prefetching techniques
• Optimizing memory accesses and bandwidth using CUDA shared memory and local memory techniques
• Measuring and improving execution time and resource utilization with CUDA profiling tools