# SwiGLU Activation: SiLU + GLU = Modern FFN'in Kalbi — Shazeer 2020'den Llama-3'e

> Source: https://sukruyusufkaya.com/learn/llm-muhendisligi/swiglu-activation-shazeer-2020-llama-3
> Updated: 2026-08-18T11:14:36.771Z
> Category: LLM Mühendisliği
> Module: Modül 10: Transformer Block — Bloğun Anatomisi
**TLDR:** SwiGLU activation function'ın anatomi: SiLU (Sigmoid-weighted Linear Unit) base + Gated Linear Unit mechanism. Shazeer 2020 'GLU Variants Improve Transformer'. ReLU/GeLU karşılaştırma, niye modern modellerin tercihi. FFN dimensions (d_ff = 8/3 × d_model Llama-3 tercihi), parameter math, Llama-3 implementation.