Technical GlossaryDeep Learning
Batch Normalization
A technique that normalizes intermediate activations at the mini-batch level to accelerate training and provide partial regularization.
Batch normalization helps reduce distribution shift inside deep networks and makes optimization more stable. By normalizing activations at the mini-batch level, it can enable training with higher learning rates. It may also introduce a mild regularization effect. It has played an important role in many architectures, from CNNs to pre-Transformer models.
You Might Also Like
Explore these concepts to continue your artificial intelligence journey.
