# Reverse-mode vs Forward-mode Autodiff: JVP, VJP, Dual Numbers, and Which to Use When in LLMs

> Source: https://sukruyusufkaya.com/en/learn/llm-muhendisligi/reverse-mode-forward-mode-autodiff-jvp-vjp
> Updated: 2026-05-13T13:00:23.722Z
> Category: LLM Mühendisliği
> Module: Module 2: Before PyTorch — NumPy and Autodiff from Scratch
**TLDR:** The two fundamental modes of automatic differentiation: forward-mode (Jacobian-vector product, dual numbers) and reverse-mode (vector-Jacobian product, backprop). Mathematical comparison, computational complexity, JAX's jvp/vjp/grad/hessian, which scenario requires which mode in LLMs.