Saddle Point

A saddle point is one of the most important yet often misunderstood structures on optimization surfaces. It is neither a true minimum nor a true maximum; in some directions it curves upward, in others downward. In high-dimensional problems, saddle points may actually be more common than local minima. This can cause optimization algorithms to slow down or become trapped in unstable regions. The concept of the saddle point is especially useful for understanding the behavior of deep learning loss landscapes.