# AdamW + Learning Rate Schedule: Modern LLM Optimization'ın Matematik Anatomisi

> Source: https://sukruyusufkaya.com/learn/llm-muhendisligi/adamw-optimizer-learning-rate-schedule
> Updated: 2026-08-11T17:46:18.053Z
> Category: LLM Mühendisliği
> Module: Modül 11: Pre-training Dynamics + Optimizer Math
**TLDR:** Modern LLM optimization: SGD'den Adam'a, Adam'dan AdamW'ye evrim. Loshchilov 2019 weight decay decoupling. Momentum (β1=0.9) + variance estimate (β2=0.95) intuition. Learning rate schedules: cosine decay, linear decay, warmup gerekli. Gradient clipping, mixed precision training, hyperparameter pitfalls.