Tanh Activation

A zero-centered activation function that maps inputs into the range from -1 to 1.

The tanh activation behaves similarly to sigmoid in terms of saturation, but its zero-centered output can provide some optimization advantages. It was frequently used in older RNNs and sequence models. Because it carries both negative and positive values symmetrically, it can offer a more balanced representation in some hidden-state settings. Still, because of the vanishing-gradient issue in deep architectures, its use is more limited in many modern networks.