Reinforcement Fine-Tuning (RFT): GRPO, DPO, and Reasoning Models (2026)

TL;DR — Reinforcement Fine-Tuning (RFT) is a family of fine-tuning methods that strengthens a model's reasoning by rewarding correct outcomes on verifiable tasks. The biggest misconception I see in the field is this: teams say "let's just train the model on our own data" and dump everything into fine-tuning. Yet the sequence that actually works in 2026 is clear: Prompt → RAG → Fine-tune → Distill. Use fine-tuning for durable behavior, schema, and tone; use RAG for knowledge that changes or is proprietary. You get the highest return by running a thin LoRA/QLoRA adapter on a strong base model together with retrieval (RAG), not as a replacement for it. Below, I explain the difference between PPO, DPO, and GRPO, how to design verifiable rewards, the Turkish/KVKK and on-prem dimension, and a practical decision checklist — all from the field.

I wrote this not as a glossary of definitions but as if I were speaking to a technical team sitting across from me at the consulting table. Because concepts like RFT, GRPO, and DPO became popular so fast over the last year that in most projects the link between terminology and real benefit got severed. My goal is to restore that link.

I describe Reinforcement Fine-Tuning in its simplest form like this: instead of telling the model "this answer is right, that one is wrong," we score the answer the model produces with a reward signal and push the model toward behaviors that earn high reward. In classic supervised fine-tuning (SFT), we show the model "if the input is this, the output is exactly this." In RFT, we give a signal of "how good was your output." The difference looks subtle but the consequences are very different.

I like to explain this distinction with an exam analogy. SFT is like making a student memorize the answer key to solved problems. The student imitates seen problems nicely but struggles on a slightly different one. RFT is like having the student produce their own solution and then checking "did this solution reach the correct outcome." The student strengthens the reasoning paths that lead to the right answer. This is exactly why RFT shines on tasks that require reasoning and multi-step thinking.

But there's a critical condition here: verifiability. For RFT to genuinely work, you have to be able to measure automatically and reliably whether an answer is correct.

"

The golden rule of RFT: If you cannot score the correctness of an answer with a machine, consistently, then your reward signal is noisy — and the model will learn that noise.

Examples of verifiable tasks from the field:

• Math and numerical reasoning: The result is either right or wrong. Checking is easy.
• Code generation: Does it pass the test suite or not? Does it compile? Are the unit tests green?
• Structured output (schema) compliance: Does it conform to the JSON schema, are the fields the right type, are required fields filled?
• Rule-based document extraction: For tasks like pulling the VAT amount from an invoice, the correct value is known and comparable.
• Classification and labeling: If there is a definitive correct label, the reward is clear.

By contrast, on tasks like "write a more empathetic email to the customer," correctness is subjective. You can still use RFT there, but you'll have to produce the reward through a human preference or a judge model — which makes things harder and pushes us toward methods like DPO. I'll get to those shortly.

In short: make RFT your first choice on tasks where "correct" is objectively measurable. If correctness is subjective, try the cheaper routes first.

These three acronyms intimidate people. Yet their core ideas are surprisingly intuitive. Think of all three as three different strategies answering the same question: "How do we push the model toward the answers that humans, or a metric, prefer?"

PPO — Proximal Policy Optimization

PPO is the classic heavyweight of reinforcement learning. The logic is this: the model produces an answer, a reward model (or a verifier) scores that answer, and then the model's parameters are updated according to that score. The "proximal" part matters: it has a brake that stops the model from drifting too much on each update. That way the model walks toward good answers without falling apart entirely.

PPO is powerful but it has a price. You typically have to train a separate reward model, then train the main model with that reward model. That means juggling two models, a value function, and sensitive hyperparameters all at once. From what I've seen in the field, PPO makes sense for teams that want maximum flexibility and have serious engineering resources; for small teams it's usually too heavy.

DPO — Direct Preference Optimization

DPO emerged almost as a reaction to PPO's complexity, and it has a simplicity I love in the field. In DPO you don't train a separate reward model. Instead you have preference pairs: two answers to the same question, one "preferred," the other "not preferred." DPO optimizes the model directly to move toward the preferred answer and away from the other.

The beauty of DPO is that its data format fits human intuition very well. Saying "this answer is better than that one" is much easier than saying "this answer is exactly this," and it's usually more scalable. That's why DPO is my first choice on subjective tasks like tone, style, safety, and style alignment. If there's no verifiable "correct answer" but you can say "which one is better," DPO is very efficient.

GRPO — Group Relative Policy Optimization

GRPO is the method that popularized recent reasoning models. Its idea is very elegant: instead of carrying a separate, expensive value function/reward model as in PPO, it has the model produce a group of answers for the same question (say 8 or 16 different attempts). Then it scores these answers with a verifier and evaluates each answer relative to the group's average. Answers above average are rewarded; those below are penalized.

I explain this with a classroom analogy: GRPO grades the student's exam paper not against an absolute cutoff but relative to the class average. It asks, "Was this attempt better than my other attempts on the same question?" This relativity eliminates the need to train a separate value function, both simplifying the process and making it very effective on verifiable tasks.

GRPO especially shines in domains that are automatically verifiable like math, code, and logic — because you can assign solid scores to the answers within a group. Much of the leap in reasoning models over the past year rests on this approach.

Let me summarize all three in one sentence each:

• PPO: The most flexible, most powerful, but heaviest and most engineering-intensive route.
• DPO: The simple, efficient route for subjective alignment, working with preference pairs.
• GRPO: The elegant route that grows reasoning on verifiable tasks with group-relative reward, requiring no value model.

In my consulting I say this again and again: the fate of RFT projects is decided not in the algorithm but in the reward design. Setting up GRPO or PPO correctly is relatively easy; the truly hard part is turning the concept of a "good answer" into a function that a machine can score.

My non-negotiables for a good verifiable reward design:

1. Determinism. The reward function must give the same score to the same answer every time. If there's randomness, the model learns the noise.

2. Resistance to manipulation (reward hacking). The model is lazy; if it finds a way to earn the reward without deserving it, it will. The classic example: when you say "reward if the answer is correct," the model might learn to just guess the correct result and skip the reasoning. Or if you reward length, the model starts producing needlessly long, empty text.

"

I've adopted a rule in the field: when you design a reward function, ask yourself, "If I were this model, how would I collect this reward without earning it?" Every loophole you find, the model will find too.

3. Graded signal. If possible, don't make the reward just 0/1. Rewarding partial correctness (for instance, correct format but wrong value, or 3 of 4 steps correct) speeds up learning. But you must design these gradations so they can't be gamed either.

4. Separating format and content reward. A technique often used in reasoning models: rewarding, on one hand, that the answer is in the correct format (e.g., thinking steps inside a certain tag, the final answer in a separate place), and on the other hand that the content is correct — separately. This makes the model both orderly and correct.

Tools I use as verifiers in the field: unit test runners (for code), symbolic/numeric comparators (for math), JSON schema validators (for structured output), regular expressions and rule engines (for rule-based extraction), and when needed a "judge model" (LLM-as-a-judge, for subjective tasks — but use this carefully, because the judge model has its own biases too).

Now let's come to the cost side, because the reason most teams give up on fine-tuning is hidden right here. Retraining all of a model's parameters (full fine-tuning) demands enormous GPU memory, long duration, and high cost. For most corporate projects in Turkey, this picture is off-putting from the start.

This is exactly where LoRA (Low-Rank Adaptation) enters, and it's the method that most often made me say "so it's actually doable" in the field. LoRA's idea is exceptionally elegant: you freeze the base model's weights and don't touch them at all. Instead, you add small, trainable, low-rank matrices to the model. During training, only these small matrices get updated. The result: strong adaptation with minimal compute.

I put it this way: you're not rebuilding a massive model from scratch; you're adding a thin "behavior layer" on top. Against a model with billions of parameters, you train perhaps a few million. Memory needs drop, training speeds up, and best of all: you can attach and detach different LoRA adapters onto the same base model for different tasks.

QLoRA takes this idea a step further: by quantizing the base model (i.e., holding the weights at lower precision, such as a 4-bit representation), it lowers memory needs even further, then trains the LoRA adapters on top of that. This lets you fine-tune, on a much more modest GPU, a model that would otherwise require very expensive hardware. For teams in Turkey working with their own hardware and a limited budget, QLoRA practically opens a door.

My practical approach, which I call the "LoRA ladder":

1. Start with a strong base model. You can't rescue a weak model with LoRA; LoRA specializes a good base, it doesn't fix a bad one.
2. Start with a thin adapter (low rank). Starting small and growing if needed is better than bloating from the start.
3. Run the adapter together with retrieval. The highest return is here: knowledge comes from RAG, behavior/schema/tone from the adapter.
4. Consider a separate adapter per task. Instead of forcing a single giant model to do everything, lightweight, swappable adapters make maintenance easier.

This is the most debated question at my consulting table. My clear answer is: the two are not rivals, they are partners. But confusing which is for what is the trap projects fall into most often.

I frame the distinction like this:

• Fine-tuning changes "how it behaves." Durable behavior, output schema, tone, style, the jargon of a domain, hitting a certain format every time — these are fine-tuning jobs. You shape the model's "personality" and "reflexes."
• RAG changes "what it knows." Knowledge that changes, updates, is confidential or organization-specific — these are RAG jobs. The price list, product catalog, internal policy documents, yesterday's regulatory change... Burying these into fine-tuning is a big mistake, because when the knowledge changes you'd have to retrain the model.

"

Simple litmus test: If the knowledge changes every six months, use RAG. If the behavior is durable, fine-tune. If you're unsure, it's probably RAG.

I frequently share this table with my clients. Which method, for what, what kind of data it needs, and what it costs:

Method	For what	Data needed	Relative cost
Prompt engineering	Fast behavior tuning, first attempt	Only well-written instructions	Very low
RAG (retrieval)	Changing/confidential/org-specific knowledge	Document base + good indexing	Low–medium
SFT (supervised fine-tuning)	Fixed behavior, format, basic specialization	Input–output example pairs	Medium
LoRA / QLoRA	Economic, portable specialization	SFT data + strong base model	Low–medium
DPO	Tone, style, safety, subjective alignment	Preference pairs (good/bad answer)	Medium
GRPO / PPO (RFT)	Reasoning on verifiable tasks	Verifier + a large pool of questions	Medium–high

As you read this table, keep in mind: as you go down, both cost and complexity rise. So start from the top.

The sequence I advocate in 2026 is exactly this, and it works in almost every project. Climb the ladder step by step; don't move to a lower rung before the upper one is exhausted.

Rung 1 — Prompt. Everything starts with a good prompt. A surprising number of problems labeled "we need fine-tuning" are actually solved with a well-written system prompt, a few examples (few-shot), and a clear output format. This is the cheapest, fastest rung. Every gain you get here is a saving on the later rungs.

Rung 2 — RAG. If the prompt isn't enough and the problem is usually "the model doesn't know our knowledge," add retrieval. Feed the model's answers with your own documents, catalogs, and policies. The vast majority of corporate projects in the field are solved at these two rungs, with no fine-tuning needed at all.

Rung 3 — Fine-tune. If RAG isn't enough either — that is, if the problem is not knowledge but behavior — then it's time to fine-tune. I summarize this threshold as "when RAG isn't enough." If the model accesses the knowledge but can't hit the format, can't catch the tone, can't do your domain's reasoning — a thin LoRA/QLoRA adapter comes into play. And I emphasize again: place this adapter not instead of RAG but alongside it. This is the highest-return configuration.

Rung 4 — Distill. If your fine-tuned, strong but perhaps large and expensive model works well, the final step is to distill it: transferring the big model's behavior to a smaller, faster, cheaper model. This is done to lower production cost and reduce latency. But you only do this once you have a working, high-quality "teacher" model in hand.

The philosophy at the core of this ladder is: always start with the cheapest and simplest solution, and climb to the next level of complexity only when you truly need to. The most expensive mistakes I've seen in the field were in teams that skipped this ladder and jumped straight to the third rung.

In fine-tuning projects the algorithms are thought to be the shiny part, but data makes up eighty percent of the work. A bitter truth from the field: with bad data, even the best algorithm produces a bad model. "Garbage in, garbage out" holds mercilessly in this domain.

What I look for in quality fine-tuning data:

• Representativeness. The data must reflect the inputs the model will encounter in the real world. If you train only on easy examples, the model collapses in hard situations.
• Consistency. If you give contradictory outputs to the same type of input, the model learns confused. A labeling guide and consistency checks are essential.
• Cleanliness. Duplicate, erroneous, leaky (test data mixing into training) examples mislead the model. Cleaning the data is boring but it's the highest-return work.
• Balance. If one class or one behavior type is overly dominant in the data, the model neglects the others.
• Sufficient but not excessive volume. LoRA's beauty is that most tasks can be solved with a few thousand good examples. A few thousand careful examples beat tens of thousands of mediocre ones.

On the reward design side, I remind you again of what I discussed in the previous section: your reward must reflect the behavior you target exactly and must not be gameable. The reward function is the project's hidden contract; it's where you tell the model what you actually want. If you say it wrong, the model does what the reward says, not what you say.

After fine-tuning a model, saying "it looks good" isn't enough; you have to prove it. And more importantly, you must be sure that while improving the model in one place you haven't broken it in another. I call this regression, and it's one of the most insidious traps in the field.

My evaluation discipline is as follows:

1. Keep a separate test set. You must have an evaluation set the model never saw in training and that reflects the real world. Testing with the training data is fooling yourself.

2. Be vigilant against overfitting. The model can memorize the training examples and collapse on new ones. If training accuracy rises while validation accuracy falls, the alarm is ringing. LoRA gives an advantage here because, since fewer parameters are trained, the tendency to memorize is relatively lower — but caution is still needed.

3. Run regression tests. Note down the tasks the model did well before fine-tuning, and re-test them after fine-tuning. While fine-tuning for reasoning, you may have broken the model's general abilities, its style, or its safety. Losing old abilities while gaining new ones (catastrophic forgetting) is a real risk.

4. Look not just at the average but at the tail. A model whose average accuracy has risen but that collapses on the most critical 5% of hard cases can be a disaster in production. Do the error analysis by hand, example by example.

5. Don't neglect human evaluation. Automated metrics guide you, but especially on subjective tasks, having real humans review a few hundred examples is invaluable.

"

Fine-tuning a model is easy; proving you didn't break it is hard. Invest in your evaluation infrastructure as much as your training infrastructure.

Now let me come to the dimension I encounter most as someone working in Turkey and that general sources rarely discuss.

Turkish language adaptation. Global models speak Turkish better and better, but in a corporate context there are still gaps: industry-specific Turkish jargon, formal correspondence style, the subtleties of legal-administrative language, the fine errors created by Turkish's agglutinative structure. This is a classic "behavior" problem where fine-tuning shines. To permanently embed Turkish corporate style and terminology into the model, a thin LoRA adapter goes a long way. But be careful: the organization's current knowledge (prices, regulations, internal procedures) should still stay in RAG; only the language and style behavior should go into fine-tuning.

KVKK and data privacy. This is the backbone of corporate projects in Turkey. If you work with data containing personal information, of a trade-secret nature, or subject to regulation, sending that data to an external API is often either prohibited or very risky. This is where an advantage of fine-tuning, and especially of LoRA/QLoRA, emerges:

• On-prem (in-house) training. Thanks to LoRA/QLoRA's low hardware needs, you can do the fine-tuning on the organization's own servers without taking the data outside. The data never leaves the building. From a KVKK standpoint, this is very close to the gold standard.
• Data locality and auditability. When you train on your own infrastructure, you can fully control which data is used how, log it, and prove it when needed.
• Portability of adapters. A LoRA adapter is a small file; storing, versioning, and managing it in-house is easy. The base model stays fixed, and the sensitive behavior lives in the adapter.

Let me give a practical warning in the KVKK context too: if your training data contains personal information, burying it into fine-tuning can leave a permanent trace. As much as possible, anonymize and mask. Try to teach the behavior pattern, not the personal data itself. Personal information that changes or may need to be deleted belongs not in fine-tuning but in the RAG layer — because deleting a record in RAG is easy, whereas "undoing" a piece of information from fine-tuned weights is very hard in practice. This ties directly to KVKK principles like the "right to be forgotten."

Cost and compute. In Turkey, GPU access and cost are still a serious constraint. QLoRA's greatest gift is here: it makes possible, on modest hardware, a fine-tuning that would otherwise look inaccessible. For a team with a limited budget, the "highest-return fine-tuning" is a thin QLoRA adapter placed on a strong open base model and fed by RAG. This configuration attends to KVKK, budget, and quality all at once.

Now let me reduce all of this to a practical checklist I use at my consulting table. Ask these in order on your next project.

Step 1 — Is the problem really fine-tuning?

• Did I try the problem with a good system prompt and a few examples?
• If the problem is "the model doesn't know," did I try RAG first?
• Is the problem a durable behavior/schema/tone problem? If yes, proceed to fine-tuning.

Step 2 — Is the task verifiable?

• Can I measure an answer's correctness with a machine, consistently?
• If yes → RFT (GRPO/PPO) is a strong candidate.
• If no, but I can say "which is better" → DPO.
• Neither, I only have example pairs → SFT.

Step 3 — Algorithm and method selection.

• Verifiable reasoning, limited engineering resources → GRPO.
• Maximum flexibility, strong team, complex reward → PPO.
• Subjective alignment, preference data → DPO.
• Basic specialization → SFT, and make it economic with LoRA/QLoRA.

Step 4 — Data and reward.

• Is my data representative, consistent, clean, and balanced?
• Did test data leak into training?
• Is my reward function resistant to manipulation (reward hacking)?
• Does the reward measure exactly the behavior I want?

Step 5 — Configuration and infrastructure.

• Did I configure the adapter together with RAG (not instead of)?
• Did I start with a strong base model?
• Does KVKK require on-prem/in-house training? Does QLoRA make it possible?
• Did I anonymize/mask personal data?

Step 6 — Evaluation and safety.

• Do I have a separate, realistic test set?
• Did I prove, with regression testing, that I preserved old abilities?
• Am I monitoring validation curves against overfitting?
• Did I look not at the average but at the hard tail cases too?
• Did I do human review on critical tasks?

See this list not as bureaucracy but as a seatbelt. When you advance by marking every item "yes," your fine-tuning project will have avoided nearly all the traps into which the failed projects I've seen in the field fell.

Finally, let me leave a concrete action suggestion: In your next meeting, ask your team a single question — "Is what we're trying to solve a knowledge problem or a behavior problem?" The answer to this single question will tell you directly whether to do RAG or fine-tuning, and will save you weeks of wasted effort. Climb the ladder in order, measure what's verifiable, and always start with the thinnest adapter.