GPUs in Artificial Intelligence

A graphics processing unit (GPU) is a processor designed to perform many similar numerical operations in parallel. In artificial intelligence training, GPUs accelerate the repeated matrix and tensor computations used to calculate model outputs, measure errors and update parameters. They complement rather than completely replace central processing units (CPUs), which commonly coordinate the overall program and data pipeline.

Why GPUs are suited to AI training

Training neural networks involves large numbers of similar arithmetic operations that can be executed simultaneously. A GPU's parallel architecture and high memory bandwidth are well suited to this workload.

• Matrix multiplication is central to the training of many neural networks and can be divided into numerous parallel operations.
• Convolutional operations used in computer vision can also be computed efficiently in parallel.
• GPUs process batches of training examples concurrently, improving computational throughput.
• Specialised lower-precision arithmetic can reduce computation time and memory use when supported by suitable hardware and training methods.

Role in the training cycle

The model's data and parameters are represented as multidimensional arrays called tensors. Once these tensors are transferred to GPU memory, software frameworks issue computational operations, or kernels, to the GPU.

• During the forward pass, the GPU applies successive mathematical operations to produce predictions.
• During backpropagation, it calculates gradients showing how model parameters contributed to the error.
• During optimisation, these gradients are used to update model parameters repeatedly across many batches.
• The CPU commonly manages program control, data preparation and input-output tasks, while the GPU performs the computation-intensive tensor operations.

Scaling and limitations

Large models may require multiple GPUs, but effective scaling depends on memory capacity, communication speed and software design.

• Data parallelism places copies of a model on different GPUs and assigns each GPU a different subset of a training batch.
• Model parallelism divides a model across GPUs when it cannot fit into the memory of one device.
• Exchanging parameters, gradients or intermediate results between GPUs can become a communication bottleneck.
• GPU memory capacity and bandwidth can limit model size and batch size.
• Training on GPUs can require substantial electrical power and cooling infrastructure.
• GPUs may offer limited advantage for small tasks or workloads dominated by sequential operations and data movement.

How UPSC asks this