Mini-Batch Gradient Descent

Mini-batch gradient descent offers a practical balance between full-batch methods and single-sample SGD. It splits the data into small subsets and performs parameter updates based on each subset. This provides both computational efficiency and reduced noise compared with single-example updates. In deep learning, mini-batch methods are among the most commonly used approaches because they align well with hardware acceleration and stable training. The choice of batch size often influences model behavior more than many practitioners initially expect.