# Derivatives, Gradients, and Matrix Calculus: The Math of Backprop from Scratch

> Source: https://sukruyusufkaya.com/en/learn/llm-muhendisligi/turev-gradient-matrix-calculus-backprop
> Updated: 2026-05-13T13:00:22.832Z
> Category: LLM Mühendisliği
> Module: Module 1: The AI Engineer's Mathematical Arsenal
**TLDR:** Derivatives from scalar to vector to matrix. Jacobian, Hessian, chain rule, numerator vs denominator layout. Why the derivative of softmax + cross-entropy is so elegant. Manual backprop computation compared with PyTorch autograd.