What Is Logistic Regression? A Guide to Classification and the Sigmoid Function

What is logistic regression? Logistic regression is a statistical machine learning model that estimates the probability of an observation belonging to a given class and assigns the observation to one of two categories based on that probability. Despite having "regression" in its name, its core job is classification; its output is not a number but a probability and the decision derived from it.

Questions like "Is this email spam or not?", "Will this customer churn?", "Will this loan be repaid?" all share one thing: the answer is one of two options. Logistic regression is designed precisely to predict such binary decisions from data on a probability basis. This guide covers what logistic regression is, how it works with the sigmoid function, what the odds ratio means, the difference from linear regression, and how it is used across industries.

Definition

Logistic Regression

A statistical machine learning model used for binary classification that estimates the probability of an observation belonging to a given class. It takes a linear combination of the inputs, converts it into a probability between 0 and 1 with the sigmoid function, and assigns the observation to a class by applying a threshold; its coefficients are interpretable through the odds ratio.

Also known as: Logistic Regression, logit model, classification model

Logistic regression is one of the most fundamental and widely used classification models in machine learning. Its importance comes from three properties combined: interpretability, speed, and the ability to work with little data. While modern deep learning architectures behave like "black boxes" with millions of parameters, logistic regression clearly shows how each input affects the outcome.

This transparency is a critical advantage in regulated industries. When a bank must justify a loan rejection or a physician must explain a risk score, "the model said so" is not enough; it must be known which factor influenced the decision and in which direction. Because logistic regression naturally meets this need, it remains the first baseline model set up in most projects, even when more powerful models are available. For the broader picture of machine learning, see the what is an algorithm and what is deep learning guides.

Logistic regression can be thought of in two steps. First, like a linear equation, the model multiplies each input by a coefficient and sums them to produce a raw score (the logit). This score can theoretically take any value from minus to plus infinity; it cannot be interpreted as a probability. In the second step, this raw score is passed through the sigmoid function and converted into a probability between 0 and 1.

How to

Steps of a logistic regression prediction

The core steps the model follows from inputs to the final class decision.

1. 1

   Weight the inputs

   Each feature (age, income, past behavior) is multiplied by a coefficient and summed to compute a raw score (logit).

2. 2

   Convert to probability with the sigmoid

   The raw score is passed through the sigmoid function and turned into a probability between 0 and 1.

3. 3

   Apply a threshold

   The probability is compared with a decision threshold (usually 0.5); above it one class is chosen, below it the other.

4. 4

   Learn the coefficients by training

   The coefficients are optimized (maximum likelihood) to best fit the actual outcomes in the training data.

The model's learning process consists entirely of finding these coefficients from data. During training, the model searches for the coefficients that maximize the agreement between the probabilities it predicts and the true labels. This is why logistic regression is both mathematically sound and explainable in its results.

At the heart of logistic regression lies the sigmoid function. The sigmoid function is an S-shaped curve that squeezes any number it takes as input — however large or small — into a value between 0 and 1. Very large positive scores approach 1, very large negative scores approach 0; when the score is zero, the output is exactly 0.5.

Why is this transformation necessary? Because by definition a probability must lie between 0 and 1, while the raw output of a linear equation can spill outside this range. The sigmoid function converts the raw score into a meaningful probability, enabling interpretable outputs like "this observation belongs to the positive class with 82% probability." The decision can then be made with a confidence level rather than a hard "yes/no", which brings the flexibility to tune the threshold to business needs.

What makes logistic regression not just a prediction tool but also an explanation tool is the concept of the odds ratio. Odds is the ratio of the probability of an event happening to the probability of it not happening: if an event's probability is 80%, the odds are 0.80 / 0.20 = 4, meaning the event is four times more likely to happen than not. The mathematics of logistic regression is built directly on the logarithm of these odds (log-odds).

Its practical value is this: each coefficient of the model shows how the odds ratio changes when the corresponding variable increases by one unit. For example, in a health model the coefficient of the "smoking" variable tells how many times smoking multiplies the odds of disease. This interpretability makes logistic regression indispensable across industries, because it explains not only "what" is predicted but also "why".

The two methods are often confused because both begin with a linear combination of inputs. But their purposes diverge fundamentally. The difference from linear regression lies in the nature of the output: linear regression predicts a continuous numeric value (house price, temperature, sales quantity), while logistic regression predicts the probability of belonging to a category.

Linear regression versus logistic regression

Property	Linear Regression	Logistic Regression
Purpose	Predict a continuous value (regression)	Predict a class/probability (classification)
Output	An unbounded number	A probability between 0 and 1
Example question	How much is this house worth?	Will this customer churn?
Core transform	None (direct linear output)	Sigmoid function
Error measure	Mean squared error	Log-loss (cross-entropy)

In short, if the thing to be predicted is "how much", linear regression is appropriate; if it is "which one" or "will it happen", logistic regression is. Seeing this distinction is the first step to matching a problem with the right method, and it prevents the most common mistake in model selection — modeling the wrong output type.

Basic logistic regression is for two classes, but it has several common variants. Binary logistic regression is the best known: it decides between two possible outcomes. When there are more than two categories, multinomial (multiclass) logistic regression is used; via the softmax generalization it produces a probability for each class and selects the one with the highest probability. When the categories are ordered (for example "low / medium / high risk"), ordinal logistic regression is preferred.

Another important dimension is regularization. To prevent overfitting, variants with L1 (lasso) and L2 (ridge) regularization are widely used; L1 also simplifies the model by eliminating unnecessary variables. Thanks to these variants, logistic regression can be applied across a broad range, from simple two-class problems to multiclass and high-dimensional ones.

The real strength of logistic regression is visible in the breadth of its application. In banking and finance it is used for credit scoring and fraud detection: the probability that an application will be repaid or that a transaction is fraudulent is estimated; interpretability here is a legal requirement. In healthcare, logistic regression is often at the base of risk models estimating a patient's probability of contracting a disease or responding to a treatment.

In marketing and e-commerce it is preferred for customer churn prediction, campaign response probability, and conversion prediction. In human resources it is used to model an employee's risk of leaving. The common denominator of these examples is that the outcome is a binary decision and the justification of the decision matters for the business. For SMEs in Türkiye too, logistic regression is a practical way to produce fast and explainable predictions from existing customer data without requiring expensive infrastructure.

Although logistic regression is a strong baseline model, it is not suitable for every problem. Its most fundamental limit is that it assumes an approximately linear relationship between the variables and the log-odds of the outcome. On its own it cannot capture the complex, non-linear patterns of the real world; in such cases tree-based models or deep learning may perform better.

These limits do not make logistic regression worthless; on the contrary, they require knowing when to use it. With proper data preparation, appropriate metric choice, and threshold tuning to the business context, logistic regression offers a solid benchmark for evaluating the performance of more complex models and, often, a sufficient solution in itself.

Building a classification model is not enough; how well it works must be measured with the right metrics. Although accuracy is the most common starting metric in logistic regression, on its own it is misleading with imbalanced data: in a problem where 95% of the classes are "negative", a model that labels everything "negative" shows 95% accuracy but is useless.

That is why logistic regression evaluation uses precision and recall together: precision answers "of the ones I called positive, how many are truly positive", and recall answers "of the true positives, how many did I catch". The F1 score that balances the two, and the area under the ROC curve (AUC) that summarizes the model's discriminative power across different thresholds, show the model's real success far more reliably than accuracy. Metric choice should depend on which error is more costly in the problem; for example, in disease detection, not missing a case (high recall) takes priority over a false alarm.

Why is logistic regression called "regression" but does classification?

The name comes from the mathematical structure underlying the method: the model regresses the logit transform of the probability on a linear combination of the inputs; that inner structure is a regression. But because its output is converted into a probability and then a class, in practice it is a classification model.

What does the sigmoid function do?

The sigmoid function converts the model's raw score, which can theoretically range from minus infinity to plus infinity, into a probability between 0 and 1. This S-shaped curve makes the output interpretable as a probability and allows a decision threshold (for example 0.5) to be applied.

What is the difference between logistic regression and linear regression?

Linear regression predicts a continuous numeric value (house price, temperature); logistic regression predicts the probability of belonging to a class. The difference from linear regression is in the output type: one produces an unbounded number, the other a probability between 0 and 1.

When is logistic regression preferred?

It is preferred when the outcome is one of two (or a few) categories, the data is relatively small, and it must be explained why the model made a decision. Credit approval, disease risk, and customer churn prediction are typical examples.

What are the limits of logistic regression?

It assumes an approximately linear relationship between the variables and the logit of the outcome; on its own it cannot capture complex, non-linear patterns. In such cases tree-based models or deep learning may perform better; still, it is a solid baseline model.

Can logistic regression be used with more than two classes?

Yes. For more than two categories, multinomial (multiclass) logistic regression or a softmax generalization is used. The model then produces a probability for each of three or more classes and selects the class with the highest probability.

In short, the answer to what is logistic regression is: an interpretable classification model based on the sigmoid function that produces a probability from inputs and assigns an observation to one of two classes. It converts the raw score into a probability via the odds ratio and the sigmoid; the difference from linear regression is that the output is a class/probability rather than a number. Because it is interpretable, fast, and works with little data, it is the first classification method tried in many industries from banking to healthcare. For the basics see the what is AI, what is an algorithm, and what is big data guides, start with AI consulting for a prediction model tailored to your organization, or see the training programs to upskill your team.