Gradient

The gradient is the vector that points in the direction of steepest increase for a multivariable function. Each component contains derivative information with respect to a different parameter. In machine learning, the gradient is used to understand how the loss function behaves and to determine the direction in which parameters should be updated. For this reason, the gradient acts like a compass for optimization. Nearly all modern training algorithms rely on this information directly or indirectly.