Gradient Boosting

Gradient Boosting trains weak learners sequentially, with each new model focusing on reducing the errors made by earlier ones. It is well known for achieving strong performance, especially on tabular datasets. Thanks to its flexible loss framework, it can be applied to both regression and classification. However, it requires careful tuning of hyperparameters such as learning rate and tree depth to avoid overfitting.