# DPO: Direct Preference Optimization — Rafailov 2023, Cheaper Rebirth of RLHF

> Source: https://sukruyusufkaya.com/en/learn/llm-muhendisligi/dpo-direct-preference-optimization-rafailov-2023
> Updated: 2026-05-13T11:17:37.339Z
> Category: LLM Mühendisliği
> Module: Module 15: RLHF + DPO — Alignment & Preference Optimization
**TLDR:** DPO (Rafailov 2023): mathematical reformulation of RLHF — no reward model, no RL. Direct preference loss. Llama-3 RLHF replacement. Math derivation, implementation simpler than PPO, comparable quality. Turkish DPO practical: $1K cost 8B model alignment.