Prompt Optimization with DSPy: From Hand-Writing Prompts to Programmable Pipelines

TL;DR — DSPy is an open-source framework originating from Stanford; it treats LLM interactions as programmable modules instead of hand-tuned prompt strings. With typed signatures you build multi-step pipelines and automatically optimize the prompts (and optionally model weights) against task-specific metrics. On structured tasks, benchmarks report a 10-40% quality gain over hand-written prompts; with more than 25,000 GitHub stars it is used in production by teams such as Cursor, Mistral, and Databricks. In this piece I explain why hand-writing prompts does not scale, the signature/module/optimizer concepts, real code examples, metric definition, a Turkish task example, when to reach for DSPy and when to stay away, and KVKK-compliant production practice.

Let me confess: for years I wrote prompts by hand, tuned them by hand, and reworked all of them from scratch at every model update. When I took over a system at one client with more than 40 prompts, I found them sitting in a Word file, unversioned, untested, in a "no one knows who changed it last" state. When we wanted to switch to a new model, fear fell over everyone, because no one knew where which prompt would break. That day I clearly saw that prompt engineering had to stop being an "art" and become "engineering." DSPy fills exactly this gap.

Writing prompts by hand works great at small scale. One task, one prompt, a few tries, and it is done. But as the system grows, this approach collapses. Let me gather the reasons under a few headings.

First, fragility. A hand-written prompt is shaped by the current model, the current examples, and the intuition of the person who wrote it. When you change the model (and a new model comes out every few months now), the prompt's behavior changes and often gets worse. If you have dozens of prompts, every model transition turns into a nightmare.

Second, unmeasurability. To the question "is this prompt better?" most teams answer by eyeballing, trying a few examples. This is not scientific. You can only tell whether a prompt is truly better with a defined metric and an evaluation set. In the hand process this discipline is almost never established.

Third, the composition problem. Real applications are not a single prompt; they are a chain of multiple steps — first rewrite the question, then retrieve documents, then generate the answer, then verify. When you wire these steps together with hand prompts, improving one step can break another, and optimizing the whole becomes impossible.

"

An observation from the field: the hidden cost of hand prompt tuning is not time, it is timidity. Teams become afraid to touch a working prompt; no improvement is made for fear it "breaks." The system freezes. Perhaps DSPy's biggest benefit is removing this fear: when there is a metric and optimization, making a change becomes safe.

DSPy's core idea is surprisingly simple but powerful: instead of telling the LLM what to do with a long text, you define what you want as a program and leave how the prompt is written to the framework. That is, you say "input is this, output should be that"; DSPy builds and optimizes the prompt to achieve it.

We can liken this to traditional software. In the old days we dealt with machine code; then compilers came and we wrote in a high-level language, the compiler handled the low level. DSPy can be thought of as a kind of compiler for LLM programming: you write the intent, DSPy compiles the prompt. That is why the DSPy team describes the framework as "programming, not prompting."

In practice there are three fundamental building blocks: Signature, Module, and Optimizer. Let us unpack them one by one.

Signature (typed signature): declaratively defines a task's inputs and outputs. For example "question -> answer" or "document -> summary." A signature says not what the prompt will do, but what it takes in and gives out. This is like a function signature: you specify the types and fields, the framework handles the rest.

Module: the component that runs the signature with a particular reasoning strategy. The best known are Predict (plain prediction), ChainOfThought (think step by step), ReAct (reason using tools). You can run the same signature with different modules; for example, you can try a question-answer task first with Predict, then with ChainOfThought, and measure which is better.

Optimizer: here is the magical part. The optimizer automatically improves the prompts (few-shot examples, instructions) and optionally the model weights based on your examples and the metric you defined. That is, you define the metric, the optimizer searches for the best prompt. Systematic search replaces hand trial-and-error.

The power of this trio is in how they interlock. With the signature you say what you want, with the module you choose how it will think, and with the optimizer you make this best-fit to your data.

Let us make it concrete. With the DSPy 2.x API, a simple question-answer module looks like this:

import dspy

# 1) Define the model
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

# 2) Signature: input -> output declaration
class QuestionAnswer(dspy.Signature):
    """Answer the question briefly and correctly."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="one-sentence answer")

# 3) Module: choose a reasoning strategy
qa = dspy.ChainOfThought(QuestionAnswer)

# 4) Run
result = qa(question="How does predictive maintenance differ from preventive maintenance?")
print(result.answer)

Notice: there is no hand-written prompt anywhere like "you are an expert, behave this way, think step by step." The signature states the intent, the ChainOfThought module adds the reasoning style, and DSPy builds the prompt itself in the background. When you want to change models, you only change the first line; the program stays the same.

The real gain is in optimization. DSPy offers several optimizers; each improves the prompt with a different strategy. Let me summarize the most used:

The usage pattern is as follows. First you prepare a training set and a metric. Then you apply the optimizer to your program:

from dspy.teleprompt import MIPROv2

# metric: is the answer correct? (simple example)
def accuracy(example, prediction, trace=None):
    return example.answer.lower() in prediction.answer.lower()

optimizer = MIPROv2(metric=accuracy, auto="light")
optimized_qa = optimizer.compile(
    qa,
    trainset=train_set,
)

The compile call here turns dozens of hours of hand trial-and-error into a systematic search. On structured tasks this approach is reported to deliver a 10-40% quality gain over hand-written prompts on benchmarks.

The reality that most surprises first-time DSPy users is this: the hard part is not learning DSPy, it is defining a good metric. The optimizer is only as smart as your metric. If you define the metric wrong, the optimizer runs toward the wrong target.

A metric is a function that converts how "good" an output is into a number. In simple tasks this can be a match check (is the answer correct?). But in real tasks it often needs to be more nuanced: is the summary both accurate and short? Did the classification land on the right label? Is the generated text in the desired format? In some cases the metric itself can be an LLM — the "LLM-as-a-judge" approach, where a model scores the output.

"

Warning: define your metric as close as possible to the real success of the job. Easy metrics like "word overlap" are easy to measure but may not reflect real quality. A wrong metric pushes the optimizer to game the metric rather than the real job. This is the most expensive mistake I see in the field.

Let me show with a concrete example how DSPy works on Turkish tasks. Say we want to sort an e-commerce company's customer messages into three classes: "return," "shipping," "product info." If you wrote a prompt by hand, you would try to add the subtleties of Turkish (suffixes, idioms, slang) one by one into the prompt, and still stumble on new phrasings. With DSPy, you learn from examples instead:

class ClassifyRequest(dspy.Signature):
    """Classify the customer message as return, shipping, or product info."""
    message: str = dspy.InputField()
    label: str = dspy.OutputField(desc="return | shipping | product_info")

classifier = dspy.Predict(ClassifyRequest)

# Optimize with labeled Turkish examples
from dspy.teleprompt import BootstrapFewShot
opt = BootstrapFewShot(metric=lambda e, p, trace=None: e.label == p.label)
trained = opt.compile(classifier, trainset=turkish_examples)

The critical point here is that you build the metric and the evaluation set with your own real Turkish data. DSPy's power is that it captures the pattern in the data from examples, without you having to describe the grammatical features of Turkish by hand. When a new slang phrase arrives, it is enough to add a few examples and recompile; you do not need to rewrite the prompt by hand.

DSPy is powerful but not a hammer to hit every nail. To be honest, in some cases it brings unnecessary complexity. Let me clarify when to reach for it and when to stay away.

Reach for DSPy: if the task is structured (classification, extraction, multi-step reasoning, RAG); if you have a defined success metric and evaluation data; if you need to optimize multiple prompts together; if you change models often and do not want to rework prompts each time; if you want to measure and improve quality systematically.

Stay away from DSPy: if a one-off, simple prompt is enough (not worth setting up a whole framework); if you have no evaluation data or clear metric (the optimizer has nothing to feed on); if the task is entirely creative and subjective with no measurable target; if your team's Python and LLMOps maturity is not there yet. This last item matters: DSPy requires an engineering discipline; if your team is not there yet, start simple and measured first.

Taking DSPy to production is not just writing code; it is setting up a process. Let me share the practices I recommend to teams in Turkey.

Prompt and program versioning. DSPy's beauty is that it takes the prompt out of being a text file and embeds it in code. Thus your prompts are versioned with git, go through code review, and their change history is kept. The "no one knows who changed it last" situation disappears.

Integrate evaluation into CI processes. On every code change, let an evaluation run automatically against the metric you defined. When a developer changes something, if quality drops, let CI break. This is the LLM-world equivalent of software testing and makes frozen systems safely changeable.

Freeze the optimized program. Every time you run the optimizer the result may change slightly. In production, save and freeze the optimized program (together with the learned few-shot examples and instructions); do not do uncontrolled re-optimization. Evaluate and approve a new optimization as a separate version before releasing it.

"

Practical advice: think of your DSPy program like a microservice. Let it have an input, an output, a test, a version, and monitoring. Once this discipline is established, model changes stop being a dreaded event and turn into a routine update.

DSPy's heart is evaluation data, and this data often comes from real customer messages, call recordings, support requests. This is exactly where KVKK comes in, and most teams skip it.

If your evaluation and optimization set contains personal data (name, phone, address, national ID, email), processing this data falls under KVKK. Moreover, this data is often sent to an LLM provider, even to a model abroad. This means risk in terms of both data processing and cross-border transfer.

Practical measures: PII masking. Before putting data into the evaluation set, mask personal data or replace it with fake data; use placeholders like "[NAME], [PHONE]" instead of "Ahmet Yilmaz, 0532...". Your metric usually does not need this information. Data minimization. Do not carry more fields than needed for evaluation. Local model option. For sensitive tasks, consider a locally running open-weight model to avoid sending data outside; DSPy is model-agnostic, which makes this switch easy. Notice and legal basis. Establish with your advisor the appropriate legal basis and notice text for processing customer data for this purpose.

Remember: DSPy's metric-focused discipline is actually a friend of KVKK. A well-defined evaluation set is also documentation of which data is processed and why. If you build compliance into the design from the start, you get both a better system and a process ready for audit. The same discipline helps with the EU AI Act too: if your LLM system operates in a high-risk use area (for example hiring, credit scoring, public services), the EU AI Act requires technical documentation, transparency, and human oversight. DSPy's metric-focused evaluation records, program versioning, and CI outputs become a natural part of exactly this documentation; that is, the answers to "how did we test, at what quality, with which version" are ready at hand. If you are building an application that touches the EU market, design this compliance infrastructure not afterward but from the start, while you set up your DSPy pipeline.

In a single-step question-answer, DSPy's benefit is only partly visible; the real difference emerges in multi-step systems. Today most enterprise applications are not a single LLM call. A typical RAG (retrieval-augmented generation) pipeline works like this: transform the user's question into a better search query, retrieve relevant documents, generate the answer with the retrieved documents, and finally verify whether the answer is consistent with the documents. Each of these four steps requires a separate prompt and, when managed by hand, turns into a nightmare.

In DSPy you build this chain as a single program and optimize the whole together. A simplified example:

class WriteQuery(dspy.Signature):
    """Turn the user question into an effective search query."""
    question: str = dspy.InputField()
    query: str = dspy.OutputField()

class GenerateAnswer(dspy.Signature):
    """Answer the question based on the retrieved documents."""
    context: str = dspy.InputField()
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

class RAG(dspy.Module):
    def __init__(self):
        self.write_query = dspy.ChainOfThought(WriteQuery)
        self.generate = dspy.ChainOfThought(GenerateAnswer)

    def forward(self, question):
        query = self.write_query(question=question).query
        docs = dspy.Retrieve(k=3)(query).passages
        context = "\n".join(docs)
        return self.generate(context=context, question=question)

The beauty here is this: when you apply the optimizer to this RAG program, both the query-writing step and the answer-generation step improve at once, end-to-end against the metric. This is nearly impossible with hand prompts; while fixing one step you break another. That is exactly why DSPy's compositional structure changes the game in multi-step systems. The biggest quality jumps I have seen in the field came when we optimized not a single prompt but an entire pipeline.

To position DSPy correctly you need to know what you are comparing it with. The answer to the frequently asked "should I use it instead of LangChain" is actually no; the two do different things. LangChain (and similar frameworks) focus on wiring components together, on orchestration — you still write the content of the prompts. DSPy, on the other hand, focuses on optimizing the content of the prompts. In fact you can use the two together.

The table below compares three approaches along the core axis:

The last row of this table is critical: DSPy's price is the obligation to prepare evaluation data. But this is not a flaw, it is actually a virtue; it forces you to measure quality. You cannot improve what you do not measure.

Let me share a concrete example. At one client there was a classifier that automatically routed incoming support emails; it ran with a long hand-written prompt and its accuracy was around 78% on an evaluation set. When they wanted to switch to a new model, the prompt's accuracy dropped to 71% and no one could figure out why. The team fiddled with the prompt by hand for weeks, got marginal improvements, but could not achieve a consistent result.

We moved the process to DSPy. First we prepared a 150-example evaluation set cleansed of PII. We modeled the task with a simple classification signature. We defined the metric clearly: the rate of landing on the correct label, plus the cost weight of misclassification (classifying an "urgent" request as "general" was more expensive than the reverse). Then we optimized with MIPROv2.

The result was that accuracy on the same new model rose from 71% to 86%. But for me the real gain was not the number; it was that the team was no longer afraid to change models. When the next model came, all they did was change one line, recompile, and see the result on the evaluation set. A dreaded process that used to take weeks turned into a half-day routine operation. This is DSPy's real return: the quality gain is nice, but the real value is the process becoming predictable and safe.

Let me be honest: DSPy has a learning curve. A team used to writing prompts by hand will at first think, "why are we going to all this trouble, we could just write the prompt and be done." This resistance is natural, and the way to overcome it is to show a small win early.

The adoption approach I recommend is this. First, apply DSPy not to the whole system but to a single painful task; start with a small victory. Second, show the "before/after" comparison on the same evaluation set, with numbers — what convinces people is not theory but the difference in their own data. Third, make metric definition a team sport; because a good metric requires the contribution of the person who knows the job best (the product owner, the domain expert); this is not just data science's job.

"

Practical advice: the point people struggle with most while learning DSPy is looking for the "magic" in the optimizer. Yet most of the value comes from defining a good signature and a good metric. The optimizer solves a well-defined problem; it does not rescue a poorly defined one. Spend most of your time on these two definitions.

If your team's maturity is not yet suited to Python and evaluation discipline, do not rush. DSPy is a powerful tool but requires discipline. First establish a clean evaluation culture on a single task; DSPy sits on top of that culture much more easily. In the field I have seen a few teams jump into DSPy before they were technically ready and leave it half-done; had those same teams first established a measurement discipline, the transition would have been much healthier.

From what I see in the field, the work teams most underestimate is writing signatures. They think "input is a question, output is an answer, done." Yet the signature is where you tell DSPy your intent, and the clearer it is, the better the optimizer works. A good signature has a few secrets.

First, field descriptions. Add a short but clear description to each input and output field. A description like answer: str = dspy.OutputField(desc="one sentence, cite the source") tells the model your expectation and narrows the optimizer's search space. Second, the docstring. The signature's docstring is actually the essence of the task; instead of "answer the question," write a sentence that reflects the spirit of the task, like "answer the question based only on the given context, without making things up." Third, the right granularity. Do not load too much work onto a single signature; splitting a complex task into several signatures makes both optimization and debugging easier.

Another important point is output types. DSPy 2.x supports structured outputs (lists, dictionaries, even Pydantic models). If your output needs to be in a specific structure (for example, a JSON object), declare it as a type in the signature; that way you do not have to beg by hand "please return JSON." That is exactly the power of typed signatures: you delegate the structure guarantee to the framework.

If DSPy's heart is the metric, the ground that metric runs on is the evaluation set. And here the most common fallacy is thinking "the more examples the better." The truth is: 50 clean, carefully chosen examples are better than 5000 noisy ones. Quality comes before quantity.

When building a good evaluation set, pay attention to these. Coverage: let your set reflect the diversity of the real world; include not only easy examples but also hard and edge cases. Because the optimizer improves whatever is in your set; if you feed it easy examples, you get a system good on easy examples but bad on hard ones. Label quality: let your labels be consistent and correct; contradictory labels confuse the optimizer. Separation: separate the training and test sets; evaluating the model on examples it has seen is fooling yourself.

"

Practical advice: keep your evaluation set alive. Regularly collect the real examples where the model erred in production and add them to the set. That way your set represents the real world's difficulties better over time, and each optimization becomes more accurate than the last. This turns DSPy from a static tool into a continuously learning system.

In Turkish tasks this is even more critical. Because of Turkish's rich morphology, slang, and regional differences, it is essential that your evaluation set comes from real Turkish user data. Translating and using a set prepared for English misses the subtleties of real Turkish usage. A clean Turkish set collected from your own field, from your own users, is more valuable than any off-the-shelf solution.

An often-skipped topic: optimization itself makes LLM calls and spends money. Especially optimizers that do Bayesian search like MIPROv2 call the model many times while searching for the best prompt. So think of optimization not as a production call but as a training step: infrequent, controlled, and budgeted. In the development phase, start with light settings like auto="light", but do a more comprehensive search for the final production version. Also, a common pattern is to use a cheaper model as a "teacher" during optimization and run the optimized program on a smaller, cheaper model; this both preserves quality and lowers production cost. In the field, the key to keeping the budget under control is to run optimization not frequently but in a planned way as meaningful data accumulates.

As I close this piece, I want you to have an applicable plan in hand. Think of the transition to DSPy in three stages.

First, pick a single structured task where you suffer the most — preferably a classification or extraction task, because defining a metric is easiest there. For this task, prepare a small evaluation set of 50-200 labeled examples and cleanse it of PII.

Second, model the task in DSPy with Predict or ChainOfThought, define a simple metric, and do the first optimization with BootstrapFewShot. Compare the result with your existing hand-written prompt on the same evaluation set. See the difference as a number; that number is what will convince your team.

Third, once you see it works, institutionalize the process: put the DSPy program in git, run evaluation in CI, freeze the optimized version, and make model changes without fear from now on. Once you establish this loop on one task, scaling to other tasks becomes much faster.

Prompt engineering started as an art, but as it matures it is turning into an engineering discipline. DSPy is the most concrete tool of this transformation. Moving from the comfortable ambiguity of hand-writing prompts to a measurable and repeatable process seems laborious at first; but once you cross that threshold, you will not want to go back.