Reinforcement Fine-Tuning (RFT), DPO and GRPO: LLM Customization Methods in 2026

TL;DR — In 2026, fine-tuning is not a single method but a family of methods you choose based on the data you have: SFT for labeled reference outputs, DPO/ORPO/KTO for preference pairs, and RFT (reinforcement fine-tuning) for tasks with a verifiable reward. RFT makes the model generate candidate outputs, scores them with an external evaluator, and updates the policy by quality — it learns from reward rather than imitating references like SFT. With GRPO and RULER-style reward models, you can improve agent behavior without hand-writing reward functions or large labeled sets, sometimes from as few as ~10 examples. LoRA/QLoRA make all of this affordable. If you train on personal or customer data in Turkey, plan for KVKK from day one: lawful basis, purpose limitation, and the risk of models memorizing and leaking personal data — prefer synthetic/anonymized data and self-hosting.

One of the most common sentences I hear in the field is: "We want a model of our own." Behind that sentence usually hide three different needs that all arrive under the same name. Sometimes you want the model to produce a format consistently, sometimes to know your company's product knowledge, and sometimes to perform a specific task not like a generic assistant but to your standards. These three needs are solved with different tools, and one of the most expensive mistakes is trying to solve a "behavior" problem with a "knowledge" tool, or vice versa.

By 2026, foundation models are so strong that in many scenarios a good prompt and the right context are enough without any fine-tuning. And yet fine-tuning did not die; it matured. We now fine-tune not to "make the model smarter" but for three concrete things: to lock in behavior, to guarantee format and tone, and to reduce latency and cost. The number of scenarios where a small fine-tuned model is far cheaper and faster than bolting a long system prompt onto a giant model and paying for thousands of tokens per request keeps growing.

In this post I map the 2026 landscape of fine-tuning: how to choose a method by the data you hold, why RFT is talked about so much, how techniques like GRPO deliver results from little data, and how to reconcile all of this with the reality of KVKK in Turkey. At the end I leave you a method-selection table and a project checklist so you have a compass when you sit down at the table.

The right question when choosing a fine-tuning method is not "which one is more modern?" but "what data do I have?" That single question routes you to the right family.

1. SFT — Supervised Fine-Tuning. If you have "input → correct output" pairs, you are here. The model learns to imitate the reference answers you provide. Classic, robust, predictable. If you want a support assistant to answer in a specific format and tone, a few hundred good examples of SFT are often enough. Weakness: the model is only as good as what you show it; you must label the "correct answer" yourself, and it will not easily go beyond the style you demonstrated.

2. Preference-based methods — DPO / ORPO / KTO. If you have preference pairs like "this answer is better than that one," this family is for you. DPO (Direct Preference Optimization) converts preference data directly into a loss function instead of training a reward model and then running RL. It tries to achieve RLHF-like results with far simpler, more stable training. ORPO merges SFT and preference alignment in a single step and needs no separate reference model. KTO (Kahneman-Tversky Optimization) does not even require paired preferences; it can work with one-off "this was good / this was bad" signals, which are much easier to collect in a real product. This family shines on subjective tasks where "there is no single right answer but some answers are clearly better": style, helpfulness, safety, brand voice.

3. RFT — Reinforcement Fine-Tuning. If you can measure programmatically whether an answer is good, you are here. Does the code compile and pass tests? Is the math result correct? Does the generated JSON match the schema? Does the SQL return the expected rows? On these "verifiable reward" tasks, RFT can beat every other method, because it does not make the model memorize a reference answer — it gives the model a goal and lets it discover the path to that goal itself.

Keep this triad in mind: labeled references → SFT; preference pairs → DPO/ORPO/KTO; verifiable reward → RFT. The rest of this post is largely the detail of that distinction.

To summarize the difference between SFT and RFT in one sentence: SFT tells the model "do it like this" and shows an example; RFT tells the model "achieve this" and gives a score. The difference looks small but changes the outcome radically.

The RFT loop runs like this. First the model generates several candidate outputs for an input. Then an external evaluator — a test suite, a validation function, or a reward model — gives each candidate a reward (a numeric score). Then the policy (the model) is updated to make high-reward behavior more likely and low-reward behavior less likely. This loop repeats thousands of times and the model masters the task by trial and error.

The critical point: in RFT you do not show the model the "correct answer." You only give it a measure of "how good each answer is." That is why RFT is so powerful on tasks where humans struggle to describe the solution but can easily evaluate the outcome. Example: describing step by step how to "write the best reply to this customer email" is hard, but scoring "did the reply resolve the question, follow policy, and match the tone" is comparatively easy.

The beauty of RFT is that the model can discover solution paths you never imagined. Imitation-based SFT caps you at the ceiling of your own examples; RFT can climb up to the reward ceiling. The price is a training process that demands care and a well-designed reward signal. A poorly designed reward leads to "reward hacking": the model raises the metric but misses the real goal. The most common mistake I see in the field is defining the reward too narrowly; the model exploits that narrow metric and you end up with a "technically successful but useless" model.

Under the RFT and RLHF umbrella, there are several answers to "how do we update the model against reward?" These are not rivals but tools with different trade-offs.

RLHF (Reinforcement Learning from Human Feedback). The classic recipe: humans rank outputs, a reward model is trained from those rankings, then the policy is optimized with RL against that reward model. The alignment that made ChatGPT what it is is largely this. Powerful but multi-stage, expensive, and prone to instability.

PPO (Proximal Policy Optimization). The classic optimization engine of RLHF. It updates the policy in cautious steps, penalizing it for straying too far from the reference. Very powerful, but it requires a separate value model and is demanding in memory and tuning.

DPO (Direct Preference Optimization). As above; it skips the reward model and RL loop and optimizes preference data directly. Stable, cheap, easy to implement. It needs preference data, not verifiable rewards.

GRPO (Group Relative Policy Optimization). The star of 2026. It removes PPO's need for a separate value model. The idea is elegant: generate a group of candidates for the same input, use the group's average reward as a baseline, and judge each candidate as "better/worse than average" against it. No expensive value model needed; in-group comparison does the job of advantage estimation. This makes GRPO both lighter and highly effective on verifiable-reward tasks. A large part of the leap in the latest generation of reasoning models comes from this family.

RLOO (REINFORCE Leave-One-Out). A similar simplification: for each sample it computes the baseline from the rewards of the other samples in the group. Lightweight and surprisingly effective; attractive for small teams.

My practical advice: if you start with preference data, start with DPO, because it gives the fastest result with the fewest surprises. If you have a verifiable reward and genuinely want behavior that "improves through experience," lean toward GRPO. Put PPO on the table only if you have an experienced RL team and serious infrastructure.

What makes GRPO truly exciting is the practice that emerges when it combines with reward models. The biggest obstacle in classic RL was having to hand-write a reward function for every new task. For most real-world tasks, writing a mathematical definition of "good" is nearly impossible.

This is where RULER-style reward models come in. In this approach, a large language model (LLM) is used as a judge to score candidate outputs on a relative scale: "which of the answers in this group performed the task better?" So instead of writing a reward function by hand, you provide a rubric that describes the task and use the model's own evaluation. This maps beautifully onto GRPO's group-relative logic, since you are already doing in-group relative ranking.

The practical result is striking: some platforms (for example Predibase) reported meaningful improvements with such setups from as few as ~10 labeled examples. Read it correctly: this does not mean "train a model from scratch with 10 examples." We are talking about aligning a strong base model to a specific task with a well-designed reward signal and very few human labels. The leverage here is the combination of scarce human labels with abundant verifiable/rubric-based reward. Training agents "through experience" means exactly this: the agent keeps attempting the task, gets reward from each attempt, and masters it over time — just as an intern learns by doing the job.

Still, be cautious. These reported results are task-dependent; they are wonderful on verifiable or clearly-rubriced tasks, while on fuzzy subjective tasks the reward model can be noisy. Always validate with an independent test set.

None of the methods above would be accessible to most teams if you had to update all model weights. LoRA (Low-Rank Adaptation) is important precisely for this reason. LoRA freezes the giant model's weights and trains only small, low-rank matrices that are added. The number of trained parameters drops to a tiny fraction of the base model; memory and cost fall dramatically, and you can even train separate small adapters for multiple tasks and swap them on top of the same base model.

QLoRA takes this a step further: it quantizes the base model to low precision (like 4-bit) and trains the LoRA adapters on top of that quantized model. The result is that even sizable models can be fine-tuned on a single modest GPU. This brings even compute-hungry methods like RFT/GRPO within reach of small teams' budgets. In practice, a typical 2026 fine-tuning project is almost always LoRA/QLoRA-based rather than full-weight.

This is the budget line I rescue most often at the consulting table. The three are not alternatives; they solve different problems, and most mature systems use all three together.

• Start with prompt engineering. The cheapest, fastest, most reversible lever. Most needs are solved with a good system prompt, a few examples (few-shot), and clear instructions. Before rushing to fine-tune, make sure you have truly hit the ceiling of the prompt.
• RAG (Retrieval-Augmented Generation) is a knowledge problem. If you want to bring in information the model doesn't know — current, organization-specific, or frequently changing — use RAG. Product catalogs, regulations, yesterday's price list: trying to memorize these into the model via training is both expensive and fragile. With RAG the knowledge stays external, updatable, and auditable.
• Fine-tuning is a behavior, format, tone, latency, and cost problem. Fine-tune when you want to change how the model behaves, what format it produces, what voice it speaks in — or to internalize long prompts so you cut the per-request token cost.

A simple rule: Missing knowledge → RAG. Behavior/format/cost problem → fine-tuning. Before both → prompt. In real systems the order is usually: prototype with prompt, wire in knowledge with RAG, and only move to fine-tuning when you hit the behavior/consistency/cost wall.

Data / need you have	Suitable method	Why
Input → correct output pairs (labeled)	SFT	Learns to imitate references, predictable
"Answer A better than B" preference pairs	DPO / ORPO	Stable, cheap, no RL loop needed
One-off good/bad signals (no pairs)	KTO	Easiest signal to collect in production
Programmatically measurable correct result (code, math, JSON, SQL)	RFT (GRPO/RLOO)	Climbs to the reward ceiling by trial and error
Hard to write reward by hand, but a rubric is describable	GRPO + RULER-style reward model	Improves through experience with few human labels
Need current/organization-specific knowledge	Not fine-tuning → RAG	Knowledge stays external, updatable
Fast, reversible behavior tweak	Not fine-tuning → prompt engineering	Cheapest lever
Budget/hardware-constrained fine-tuning	LoRA / QLoRA (on top of the above)	Parameter-efficient, possible on one GPU

My field experience is clear: the success of fine-tuning projects is eighty percent determined in data preparation, not in model choice. A few principles:

Quality before quantity. 200 carefully prepared examples almost always beat 5,000 sloppy ones. Contradictory labels, inconsistent format, and noisy examples pull the model the wrong way. Interrogate every example: "is this exactly the behavior I want the model to learn?"

Freeze the format. The input/output format in training must be identical to the format in production. System prompt, delimiters, role tags — all must be consistent. Even the smallest format difference between training and inference silently degrades performance.

Collect preference and reward data from production. For DPO, you can derive preference pairs from real users' thumbs up/down signals or from humans comparing two answers side by side. For RFT, write your validation functions first; the reward signal is the heart of the project.

Set aside the test set up front. Before training, reserve a clean evaluation set the model will never see during training. Otherwise you fool yourself.

The most-skipped step of fine-tuning is serious evaluation. Training the model and saying "wow, nice answer" is hope, not engineering. Build a real evaluation:

• Task-specific automatic metrics: pass rate on verifiable tasks, format compliance, exact match. On subjective tasks use an LLM-judge rubric, but calibrate it against a human sample.
• Side-by-side (A/B) comparison: compare the fine-tuned model against base model + good prompt blindly on the same test set. See whether fine-tuning actually wins.
• Regression check: make sure the model does not lose old abilities while improving on the new task (see catastrophic forgetting below).
• Safety and bias: fine-tuning can weaken safety alignment. Re-run harmful-content and bias tests before and after training.

When you intensively teach the model a new task, it is normal for it to forget part of its old abilities. This is called "catastrophic forgetting." You might aggressively fine-tune on a customer-classification task and then notice the model's general fluency or reasoning has regressed.

Ways to mitigate: use parameter-efficient methods like LoRA (forgetting is reduced because base weights are frozen); keep the learning rate low; don't run too many epochs (over-training increases forgetting); and mix some general/diverse examples into the training data to "remind" the model of its old skills (replay). Most importantly: measure old abilities with your regression test set after every training run.

The cost of fine-tuning is not just GPU hours. See the whole picture: data collection and labeling labor, experiment rounds, evaluation infrastructure, and — most importantly — maintenance. When the base model is upgraded you may need to repeat your fine-tuning; maintaining a fine-tuned model is like maintaining a software component.

The upside is real too: a well fine-tuned small model runs far cheaper and faster than sending a long system prompt to a giant model on every request; latency drops, token cost drops, and you can keep the data in your own infrastructure. The decision must be made on total cost at scale, not the cost of a single request. At low volume, prompt+RAG almost always wins; on a high-volume, narrow, repetitive task, fine-tuning passes the break-even point long ago.

This is where technical excitement meets legal reality, and the part most neglected in the field. Training a model on personal or customer data is not ordinary data processing under KVKK; it triggers several risks at once.

Lawful basis and purpose limitation. Is the purpose for which you collected the data the same as model training? If you collected support records "to provide support," using the same data "to train a model" is a new processing purpose and may require a separate lawful basis (explicit consent or a legitimate-interest assessment). Address this up front in your privacy notice and business processes.

Memorization and leakage risk. Models can memorize rare and repeated sequences in the training data. If you train a model on raw personal data, there is a real risk that the model later leaks that personal data as output with an appropriate prompt. Under KVKK this can amount to a data breach, and "unlearning" it from a model is technically very hard. The strongest defense is to cut the risk at the source.

Prefer synthetic and anonymized data. Wherever possible, train on anonymized or synthetic data instead of raw personal data. Well-constructed anonymization both lightens the KVKK burden and lowers memorization risk. Synthetic data generation matured in 2026; training on examples that mimic the real distribution but belong to no one is feasible for most tasks.

Data residency and self-hosting. Doing training and inference on a SaaS abroad raises the question of cross-border data transfer. If you work with sensitive data, training open-source models on your own infrastructure (self-hosting) keeps data residency under control and eases auditing. In scenarios within the scope of the EU AI Act (high-risk uses), governance of training data, traceability, and documentation are already becoming mandatory; designing this from the start together with KVKK is far cheaper than patching it later.

Right to erasure and governance. When a person requests deletion of their data, what will you do if that data is embedded in the model's weights? You must have an answer: either never train on personal data, or define a process such as retraining/adapter-swapping. Set up governance before training.

Use this list as a compass when you sit down at the table:

1. Name the problem correctly. Is it a knowledge problem (→ RAG), a behavior/format/cost problem (→ fine-tuning), or solvable with a prompt? The wrong tool choice is the most expensive mistake.
2. Truly test the prompt and RAG ceiling. Before moving to fine-tuning, prove these cheap levers are not enough.
3. Pick the method by the data. Labeled reference → SFT; preference → DPO/ORPO/KTO; verifiable reward → RFT/GRPO. Use the selection table.
4. Build the evaluation set first. Define a clean test set and clear metrics before training.
5. Prioritize data quality. Few but rigorous. Keep the format identical between training and production.
6. Plan KVKK from the start. Lawful basis, purpose limitation, preference for anonymized/synthetic data, self-hosting decision, deletion governance.
7. Start with LoRA/QLoRA. Move to full-weight updates only if you truly need to.
8. Start small, measure, iterate. Begin with a small sample set and a single method to see the break-even point.
9. Measure catastrophic forgetting. Verify with regression tests that old abilities are preserved.
10. Plan for maintenance. Accept from the outset the process and cost of repeating fine-tuning on base-model upgrades.
11. Be prepared for reward hacking. In RFT, verify with independent evaluation that the metric represents the real goal.
12. Re-test safety alignment. Repeat harmful-content and bias tests after fine-tuning.

If after all this you are asking "so what do I do tomorrow?", my recommendation is to proceed in this concrete order. First pick a single narrow, high-volume, well-defined task — not the whole assistant, but for example just "assign the incoming support email to the right category and generate a template reply." Do the break-even math for this task: how many requests per day, how many tokens per request, and how much does the fine-tuned small model save compared with the giant model? If the numbers are not convincing, fine-tuning is not the right tool anyway.

Then look at the data: do you have labeled examples, preference pairs, or a verifiable success criterion? This determines the method. If you have a verifiable criterion (is the category correct, does the reply follow policy), start with a GRPO-based RFT setup on few examples; otherwise start with SFT on a few hundred rigorous examples, or DPO on preferences collected from production. Use LoRA/QLoRA, start small, and always blind-compare the fine-tuned model against base model + good prompt on that reserved test set.

And place your KVKK decisions as a layer beneath this entire technical flow: is the data you train on personal, can you anonymize it, where do you train, and what will you do when a deletion request arrives? Put the answers to these questions in writing before you train the model. Fine-tuning is a powerful lever in 2026; but its power appears when you use it on the right problem, with the right data, the right method, and the right legal footing. On the wrong problem it is merely an expensive and hard-to-maintain burden. The difference is determined by those few clear questions you ask before you begin.