Glossary Library

Technical GlossaryDeep Learning

Swish Activation

A modern activation function that multiplies the input by a sigmoid to create a smooth nonlinear transformation.

Swish is one of the modern activation functions that has attracted attention for its smoothness and expressive behavior. Because it does not shut off completely in the negative region and provides smooth transitions, it can outperform ReLU in some settings. It is often considered a strong alternative in studies examining deep optimization dynamics.

You Might Also Like

Explore these concepts to continue your artificial intelligence journey.

Glossary Cover

yapay-zeka-temelleri

Optimization

The process of systematically improving model parameters according to an objective function in order to increase performance.

Glossary Cover

matematik-istatistik-optimizasyon

Gradient Descent

A fundamental optimization method that updates parameters in the opposite direction of the gradient to minimize a loss function.

Glossary Cover

matematik-istatistik-optimizasyon

Stochastic Gradient Descent

An optimization approach that updates parameters using single examples or small subsets instead of the full dataset at each step.