SFT, DPO and RFT: Which Fine-Tuning Method to Choose in 2026?

TL;DR — The question I hear most often at my corporate consulting table these days is this: "Should we fine-tune our model, or is prompt engineering enough?" The answer is always "it depends" — but in this piece I want to make that "it depends" concrete. In short: push prompting and RAG first; for most problems, the need ends right there. If you genuinely need fine-tuning, SFT (Supervised Fine-Tuning) should almost always be your first stop — it learns from reference outputs, its setup is relatively simple, and on its own it handles roughly 60-75% of the job. If what you need is preference-based alignment — sharpening the model's sense of "this answer is better than that one" — DPO (Direct Preference Optimization) comes into play; it offers a lightweight alternative to RLHF without needing a separate reward model. But it works with static, offline preference pairs, which makes it the wrong tool for multi-step agentic tasks that need to learn from dynamic tool-execution results. When the correct output is hard to label by hand but the correctness of a generated output can be automatically verified (math, code generation, tool-use planning), RFT (Reinforcement Fine-Tuning) takes the stage: the model generates candidate outputs, receives a reward from a verifier, and updates its policy accordingly — it can produce meaningful gains with as few as a dozen examples. In practice, the strongest results I see come from applying RFT on top of SFT, pushing into failure modes that demonstrations simply don't cover. In Turkey, there's one more layer to add to this picture: if you're going to train on customer data, the KVKK (Turkish data protection law) framework needs to be at the table from day one — not bolted on at the end.

A few years ago, when you said "fine-tuning," the team across the table usually understood one specific thing: showing the model example input-output pairs and getting it used to that behavior. That's no longer the case. There are now three distinct fine-tuning philosophies, each demanding a different kind of data, each requiring different infrastructure, and each solving a different class of problem. In the organizations I consult for — banking, e-commerce, manufacturing, public institutions — I frequently see these three methods getting mixed up. One team says "let's do DPO" but actually only has good examples on hand, not preference pairs. Another team says "let's try RFT" but has no mechanism to automatically verify the correctness of their task outcomes. This confusion isn't cheap: choosing the wrong method costs time, budget, and — worse — the team's trust in fine-tuning as a discipline.

The question comes up more often in 2026 because all three methods have matured in production environments, most major model providers now offer them via API, and with the rise of agentic systems, the complaint "our model completes the task 50% of the time and gets stuck the other 50%" has become far more visible. In this piece I'm going to treat the three methods not as a technical deep-dive but as a decision-making guide: which method fits which problem, what data you need, how costly it is, and — most importantly — in what order you should proceed.

I'm putting this section first for a reason: in a meaningful share of the projects I've seen in the field, fine-tuning turned out to be unnecessary. When teams say "our model doesn't know our jargon" or "the responses don't match our tone," their first instinct is fine-tuning. Yet most of the time the fix is much cheaper and much faster, hiding in one of two places:

• Prompt engineering: Clarifying system instructions, adding few-shot examples, templating the output format — these are generally the first tools to try, and they cost zero training. At one client organization, we solved the complaint "our model never writes like us" with a system prompt containing three good examples; fine-tuning was never needed.
• RAG (Retrieval-Augmented Generation): If the model "doesn't know" something, the problem usually isn't in the parameters — it's in access. Frequently changing information like an up-to-date product catalog, internal procedures, or regulatory text shouldn't be "memorized" into the model via fine-tuning, because you'd need to retrain every time it changes. Instead, setting up a vector database and a solid retrieval layer so the model can reach the right document at the right moment is both cheaper and more sustainable.

I'd recommend taking fine-tuning seriously in three cases: (1) the model can't consistently reproduce a specific behavioral pattern — tone, format, decision logic — and you can't fix it with prompting after three or four attempts; (2) the task involves so many edge cases or rules that they simply won't fit inside a prompt; (3) you need to move to a smaller, specialized model for latency or cost reasons (fine-tuning a small model to do the same job far more cheaply than repeatedly calling a large model with long prompts). If none of these three conditions apply, think twice before investing in fine-tuning.

"

A field observation: of every ten projects that arrive with the request "let's do fine-tuning," roughly six of them actually needed a better prompt architecture or a stronger RAG pipeline. Fine-tuning is a powerful tool when used in the right place — but it isn't the answer to every problem.

Once you've decided fine-tuning is warranted, the next question is which method to choose. Let me explain all three through the same lens: what does the model learn from?

SFT — Learning from demonstration

Supervised Fine-Tuning is the most classic and most intuitive method. You have input-output pairs: "this question deserves this answer," "this request should be handled this way." You show the model these examples, and it learns to imitate them. Technically, you're updating the model's weights to increase the probability that it produces these "gold standard" outputs.

The most practical characteristic of SFT is this: you train on successful trajectories. If you have a customer service bot, a code assistant, or a document summarizer, and you can collect examples where the system historically produced "good" results, you can use those directly as SFT data. You don't need to design a complex reward function, build a preference pair, or write a verifier — you just need to be able to say "this is a good example."

In my experience, SFT handles 60-75% of the job with even a few hundred to a few thousand well-curated examples. This isn't a random figure — it's a threshold I see repeated in the field over and over. SFT largely resolves format consistency, tone alignment, and baseline task performance. The remaining 25-40% tends to hide in fine distinctions — things the model "knows" but doesn't apply consistently, or situations where more than one answer is correct and the question becomes which one is better — which is exactly what brings us to DPO and RFT.

When should you choose SFT? Almost always as your first stop, whether or not you plan to apply DPO or RFT afterward. Without a solid SFT foundation underneath, both of these methods behave far more fragilely. Jumping straight to DPO or RFT without SFT is like building a wall without laying a foundation first.

DPO — Learning from preference

Direct Preference Optimization emerged as a lighter alternative to RLHF (Reinforcement Learning from Human Feedback). In the classic RLHF pipeline, you first train a separate "reward model," then use that reward model to update the actual model through reinforcement learning — a two-stage process that's computationally expensive and hard to keep stable. DPO eliminates that separate reward model entirely: it takes your preference pairs ("answer A is better than answer B") and converts them directly into a loss function, optimizing the model directly.

The key word here is directly. DPO updates the model's weights to make the preferred output more likely relative to the rejected output — without an intermediate reward model or a reinforcement-learning loop. This makes it a significantly simpler, cheaper, and more stable training process compared to RLHF.

When is DPO the right tool? When you have data labeled as "which of these two answers is better" and your goal is to improve the model's alignment — its match with human preference. Typical scenarios: making response tone kinder/shorter/safer, teaching the model which of two possible answers better fits the brand voice, or suppressing harmful/inappropriate outputs.

But there's a critical boundary here, and I see teams overlook it frequently: DPO works with a static, offline preference dataset. You collect preference pairs in advance, run training once, and the model learns from these fixed pairs. This works fine for single-step or short-interaction tasks. However, in multi-step, agentic tasks — scenarios where the model calls a tool, observes the result, decides on the next step, and perhaps backtracks — DPO's static structure falls short. Because in these tasks, what counts as "correct" needs to be learned from dynamic tool-execution results, not from a preference pair fixed in advance. The model needs to learn what happens when it calls a tool, and whether that result moves the task forward or backward — a dynamic that simply can't be captured by a pre-prepared "A is better than B" pair. This is exactly the point where DPO is the wrong tool, and RFT steps in.

RFT — Learning from reward

Reinforcement Fine-Tuning is the most "active" of the three methods. In SFT and DPO, the model is essentially a passive student — it learns the examples (or preferences) shown to it. In RFT, the model generates its own outputs, these candidate outputs are scored by an external evaluator or verifier (a reward signal), and the model updates its policy — its decision-making strategy — based on that reward signal. This loop repeats: generate, evaluate, update.

The power of RFT lies here: in some tasks, labeling the "correct answer" by hand is very difficult or very expensive, but automatically verifying whether a generated output is correct is entirely possible. Consider this: writing a thousand examples of "how the model should solve this math problem" would be enormously laborious, but checking whether the model's generated solution reaches the correct result can be done programmatically in seconds. The same logic applies to:

• Mathematical reasoning: The correctness of the result can be checked programmatically.
• Code generation: Whether the generated code passes the test suite, compiles, or produces the expected output — all automatically verifiable.
• Tool-use planning: Whether an agent chose the right sequence of tools and completed the task can be verified through an environment simulation.

Another important property of RFT is its data efficiency. With a well-designed reward function or verifier in place, RFT can produce meaningful gains with as few as a dozen examples — because the model isn't memorizing a fixed dataset, it's exploring on its own based on the reward signal. This is a tremendous advantage in scenarios where data collection is expensive but verification is cheap.

How should RFT be used most effectively? The approach I've seen work best in the field, and the one I recommend, is this: apply RFT as a layer on top of SFT. First, establish the model's baseline behavior, format, and task understanding through SFT — this gets you 60-75% of the way — then use RFT to push into the failure modes that demonstrations don't cover, i.e., scenarios that never appeared in your example set. A concrete illustration of this: on agent-performance benchmarks like tau-bench or SWE-bench, we see agents' task completion rates stuck in the 40-60% range despite a strong SFT foundation. In other words, the model has learned the basic behavior, but still fails roughly half the time in complex, multi-step, real-world scenarios. This gap is one that SFT data cannot close by its nature — because demonstrations only cover a limited set of scenarios — and this is precisely where RFT, by learning from the model's own attempts and real tool-execution outcomes, begins to close that gap.

I want to underline this point under its own heading, because it's one of the most common mistakes I see in the field. A team has an agent system — say, one that processes a customer request by calling multiple APIs in sequence, making decisions based on intermediate results — and to improve this system's performance, they turn to DPO. The reasoning seems to make sense: "let's compare good and bad outputs from the model and build preference pairs so it learns to do better."

The problem is this: in an agentic task, what counts as "good" or "bad" doesn't emerge from the quality of a single output — it emerges from a chain of interdependent decisions. When the model picks the wrong tool at step three, steps four and five inherit that error. DPO's static preference pairs cannot capture this cascading effect — because the pair you give to DPO is a pre-fixed, frozen comparison saying "in this scenario, answer A is better than answer B." But in the real world, the outcome of a tool call is variable; calling the same tool at the same step can sometimes return a different result, and the environment's state can change. The model needs to learn from this dynamic feedback, whereas DPO only gives it frozen "this is preferred over that" information.

RFT, by contrast, is designed precisely to fill this gap: the model actually calls the tool (or does so in a simulated environment), sees the real outcome, that outcome is evaluated for whether it moved the task forward, and the model updates its policy based on this live feedback. This is why, in multi-step, tool-using, environment-interacting systems, RFT is structurally the more appropriate tool compared to DPO.

I'd recommend thinking of this as a flow. Ask yourself the following questions in order:

1. Can you solve the problem with prompt engineering or RAG? If yes, you don't need fine-tuning at all. If no, move to step 2.
2. Do you have "good" examples (demonstrations), but no preference pairs or verifier? Start with SFT. This should be your first fine-tuning step in nearly every scenario.
3. After SFT, does the model handle the core task but remain inconsistent on finer distinctions — tone, preference, which of two correct answers is more appropriate? If you have or can collect preference pairs and the task is single-step or a short interaction, consider DPO.
4. Is the task multi-step, agentic, tool-using, and can you automatically verify correct vs. incorrect output (a test suite, a simulation environment, a math check, a rule engine)? If yes, add RFT on top of SFT.
5. Do you lack a verification mechanism and can't feasibly build one? Then RFT isn't a good fit for now; continue with SFT and/or DPO while building your verification infrastructure in parallel.

We can also lay this out as a table:

Situation	Recommended method	Why
Model can't match jargon/tone, prompting doesn't fix it	SFT (after prompting/RAG has been tried)	Demonstration examples are a direct fix
Two plausible answers exist, which is better isn't clear, task is single-step	DPO	Preference pairs directly optimize for this
Agent performs a multi-step task, calls tools, results are dynamic	RFT (on top of SFT)	Needs to learn from dynamic feedback
Task like math/code is verifiable but hard to label	RFT	Automatic verifier gives strong signal with few examples
Information isn't current / changes frequently	Neither SFT, DPO, nor RFT	Use RAG
A rarely repeated, one-off request	None	Prompt engineering is sufficient

These three methods differ significantly in their data and cost profiles. When helping clients with budget planning, I use roughly this framework — exact figures vary by project, but the proportions tend to hold:

Dimension	SFT	DPO	RFT
Data type	Input-output pairs (demonstrations)	Preference pairs (A > B)	Candidate output + reward/verification signal
Data collection difficulty	Moderate (past successful examples can be reused)	High (requires generating two options and comparative labeling)	Low-moderate (fewer examples needed if verifier is good)
Typical example count	Hundreds-thousands	Hundreds-thousands	Dozens-hundreds (with a quality verifier)
Infrastructure complexity	Low	Moderate	High (reward model/verifier + training loop)
Training stability	High, predictable	Moderate, needs careful hyperparameter tuning	Variable, depends on verifier quality
Main cost driver	Data curation	Comparative labeling	Verifier/environment development + compute

The practical takeaway here is: SFT starts with a relatively low barrier because its data-collection cost is essentially "compiling the good examples you already have." DPO's cost rises because generating preference pairs generally requires producing two (or more) outputs and having them comparatively labeled — meaning human evaluator time. In RFT, the primary cost isn't the number of examples but getting the verifier right. Building a good verifier — a test suite, a simulation environment, a rule engine — requires engineering investment, but once built, it can produce a strong signal with very few examples. That's why I position RFT as the method that "does a lot with little data, but requires infrastructure."

Whichever method you choose — SFT, DPO, or RFT — you also need to decide how you'll train. Updating all of a model's parameters (full fine-tuning) is both very costly and requires massive GPU memory — in most enterprise scenarios, this makes the budget and timeline unrealistic. This is where LoRA (Low-Rank Adaptation) and its even lighter cousin QLoRA (Quantized LoRA) come in.

LoRA's logic is simple: you freeze the model's original weights and add small, low-rank "adapter" matrices to each layer. During training, only these small adapters get updated — typically just a few percent of the model's total parameters. This drastically reduces training time, GPU memory requirements, and storage cost. QLoRA takes this one step further: it loads the base model into memory at low precision (e.g., 4-bit, quantized), while still training the adapters at full precision. This makes it possible to fine-tune models on a single consumer-grade GPU that previously would only have been feasible on large cloud clusters.

In my consulting work, I recommend LoRA/QLoRA as the near-default option, especially in these situations:

• Budget and GPU access are limited,
• You need multiple task/department-specific "expert" model variants (you can train a separate small adapter for each and swap them on top of the same base model),
• You want rapid iteration — adapter training completes much faster than full fine-tuning,
• You want to largely preserve the model's original capabilities while fine-tuning just one specific behavior.

I generally recommend full fine-tuning only when a truly fundamental change in the model's behavior is needed, when you have a very large and diverse dataset, and when you have the budget and infrastructure to cover that cost. But realistically, in the large majority of the enterprise projects I've encountered, LoRA/QLoRA more than covers the need.

The most overlooked part of fine-tuning — and, in my view, the most critical part — is evaluation. Whichever method you choose, if you can't give a concrete answer to "did our model actually get better?", what you're doing is an act of faith, not engineering.

Evaluation for SFT is relatively straightforward: on a held-out test set, you compare the model's generated outputs against reference outputs — either with automated metrics (task-based accuracy, format compliance) or with human evaluation. Evaluation for DPO is somewhat more nuanced: you need to check how often the model produces the preferred output, and also whether there's regression in general quality and safety metrics — i.e., making sure "alignment" gains haven't come at the cost of general capability.

For RFT, evaluation is intertwined with the training process itself — because a verifiable reward mechanism already sits at the center of the training loop. The critical question here is: is your verifier actually reliable? If the verifier is flawed or can easily be "gamed" (reward hacking), the model learns to game the verifier instead of learning the correct behavior. That's why, in RFT projects, I spend a significant portion of my time not on model training but on testing the robustness of the verifier: does the verifier correctly flag genuinely correct solutions? Can it distinguish solutions that are "wrong but got lucky" from truly correct ones? Does it miss edge cases?

A practical recommendation: before starting RFT, run a "validator validation" — compare your verifier's judgments against human evaluation on a small sample. Compare the scores your verifier assigns against the scores an expert human assigns. If there isn't strong agreement between them, you need to improve the verifier before starting RFT — otherwise the model will learn, very effectively, something you never intended.

To make this concrete, let me share a composite example close to a real profile. Picture a mid-sized Turkish bank's or e-commerce company's customer service operation. The process generally unfolds like this:

Stage 1 - Prompting and RAG: In the first stage, we support the existing large language model with a RAG system connected to the company's product catalog and procedure documents. This resolves the majority of information-based questions like "what's the return process" or "what are the campaign terms." No fine-tuning is needed at all.

Stage 2 - SFT: We use the customer service team's historical, highly-rated conversation records (rated well in customer satisfaction surveys) — with personal data anonymized or masked — as SFT data. At this stage, the model learns the company's tone, standard response patterns, and escalation rules. This handles the bulk of baseline task performance.

Stage 3 - DPO (optional): If certain scenarios involve "two different response styles that are both correct, but one is more appropriate than the other" (for example, two responses conveying the same information, but one being more empathetic and the other more terse), a round of DPO can be applied using "this response is better than that one" judgments collected from customer representatives.

Stage 4 - RFT (in an agentic scenario): The company's customer service bot no longer just answers questions — it now calls an order-lookup API, initiates the return process, and accesses the shipment tracking system — that is, it has become a multi-step agent. Here, the SFT-trained agent may get stuck on a meaningful share of tasks in benchmark testing: calling the wrong API, misinterpreting an intermediate result, or failing to take steps in the correct order. At this point, we build a test/simulation environment (a mock order database, controlled API responses) with a mechanism that automatically verifies the agent's real task-completion success, and apply RFT on top of SFT. The result is an agent that can make correct decisions even in scenarios that never appeared in the demonstration data but that it encounters in simulation.

The biggest benefit of this staged approach is that you get a measurable gain at every step, and you only move to the next, more expensive step when it's genuinely needed.

When I look at fine-tuning projects in Turkey, there's a topic that stays in the shadow of the technical discussion but is at least as critical: KVKK (Turkey's Law on the Protection of Personal Data). If you're going to use customer conversation records, support tickets, or sales calls as SFT, DPO, or RFT data, that data very likely contains personal data — sometimes special-category personal data as well.

A few critical points I flag repeatedly in the field:

• Purpose limitation: One of KVKK's core principles is that data cannot be used for a purpose other than the one it was collected for. If a customer service record was collected for "providing service," using it for "training a model" may require a separate legal basis (explicit consent, a legitimate-interest analysis, etc.). Clarify this assessment with your legal and compliance team before the project begins.
• Anonymization and masking: Masking or fully removing direct identifiers — name, national ID number, phone, address — before data enters the fine-tuning pipeline should be standard practice. But be careful: anonymization isn't always as simple as it looks — a person can become re-identifiable from context (re-identification risk). I'd recommend treating anonymization not as a one-time "delete the name" step but as a process requiring its own risk assessment.
• Data minimization: The instinct to "use all the data I have" for fine-tuning is both risky under KVKK and technically unnecessary — a few hundred to a few thousand quality examples are already far more valuable for SFT than your entire database. Using only a genuinely necessary, representative subset both eases compliance and improves model quality.
• Third-party model providers: If you're running fine-tuning through a cloud-based API hosted with a provider abroad, this may constitute a cross-border data transfer and can trigger the additional obligations under Article 9 of KVKK (adequacy decision, standard contractual clauses, explicit consent, etc.). Clarifying this before the project scope is set is far cheaper than stopping midway and starting over.
• Retention and deletion: How long training data will be retained, and whether the raw data will be deleted after the model is trained, connects directly to KVKK's retention-limitation principle. Put these policies in writing.

The practical approach I recommend to clients is this: before starting a fine-tuning project, bring the technical team and the compliance/legal team to the same table and run a "data use sign-off" process. This is far less costly than hitting a legal blocker near the end of the project.

To wrap up, I want to share a simple checklist I use in the field:

• Have I genuinely tried solving the problem with prompt engineering or RAG?
• Do I have quality, representative demonstration examples on hand (for SFT)?
• Is this a scenario requiring a preference distinction, or is there a single correct answer (do I need DPO)?
• Is the task multi-step/agentic, does it need to learn from dynamic tool results (do I need RFT)?
• Do I have — or can I build — a mechanism to automatically verify correct vs. incorrect output (test suite, simulation, rule engine)?
• Should I proceed with LoRA/QLoRA or full fine-tuning — what does my budget and infrastructure allow?
• Have I kept my evaluation set completely separate from my training data?
• Have I tested my verifier's (if any) reliability against human evaluation?
• If I'm using customer data, have I clarified purpose limitation, anonymization, data minimization, and cross-border transfer with legal/compliance under KVKK?
• Am I following a staged path — SFT first, then DPO or RFT on top if needed — or did I jump straight to the most complex method?

That last item is really the heart of this piece. The most expensive mistake in the fine-tuning world is starting directly with the most complex method. I've seen teams skip SFT and try to jump straight into RFT, spend hours tuning hyperparameters on an unfounded training process, and eventually circle back and start with SFT anyway. The right order is almost always the same: push prompting and RAG first, lay a solid foundation with SFT, then, depending on the need, refine alignment with DPO or step into the verifiable, demonstration-resistant task classes with RFT. Get this sequence right, and you protect your budget, your team's time, and — perhaps most importantly — your team's trust in the model you've built.