Line Search

Line search is used in optimization to determine how far to move once a promising direction has been chosen. Knowing the right direction alone is not enough: steps that are too large can create instability, while steps that are too small can slow learning dramatically. For this reason, step-size selection has a major impact on optimization performance. In advanced optimization algorithms, line search plays an important role in achieving more reliable and controlled convergence. Strong optimization requires both a good direction and a good step size.