What Is Likelihood?

What is likelihood? Likelihood is a function that, holding an observed dataset fixed and varying a statistical model's parameters, measures how well those parameters explain the data. In short, likelihood is the mathematical answer to the question "under which model setting does this data I hold look most plausible?"

At first glance this resembles probability, and it is even written with the same formula; but the question it asks is reversed. In probability we fix the model and ask about the chance of the data; in likelihood we fix the data and ask about the plausibility of the model. This subtle but decisive probability-vs-likelihood difference underlies all statistical inference and modern machine learning. This guide answers what likelihood is, how it differs from probability, what maximum likelihood and log-likelihood are, and how all of this relates to AI.

Definition

Likelihood

A function that measures how well a statistical model's parameters explain the observed data while that data is held fixed. Unlike probability, the variable is the parameter, not the data; the more likely a parameter value makes the observed data, the higher its likelihood. Likelihood is not a probability distribution and does not have to sum to 1.

Also known as: Likelihood function, likelihood

The key to understanding likelihood is separating it from probability, because the two share the same mathematical expression but ask completely opposite questions. A coin-flip example clarifies this probability-vs-likelihood difference. Suppose we denote the probability of heads by p.

In the world of probability we know p (say p = 0.5) and ask "what is the chance of getting a certain number of heads in 10 tosses?" Here the parameter is fixed and the data varies. In the world of likelihood we do the reverse: we have seen the data (say 7 heads in 10 tosses) and ask "which value of p makes this outcome most plausible?" Here the data is fixed and the parameter varies. The formula is the same binomial expression; but in probability we read it as a function of the data, and in likelihood as a function of the parameter.

The most overlooked consequence of this distinction is that likelihood is not a probability distribution. The likelihoods computed for different values of p do not have to sum to 1, because likelihood defines no distribution over the parameter. This is one of the most common conceptual mistakes in statistics, and internalizing the difference prevents it.

The likelihood function takes the observed data as input and assigns a number to each possible parameter value: how likely that parameter makes the data at hand. A high likelihood value shows that the parameter is consistent with the data; a low value shows it is inconsistent.

Let us make it concrete. Suppose we have a biased coin and observe 7 heads in 10 tosses. We compute the likelihood of this outcome for a heads probability of p = 0.5; then for p = 0.7; then for p = 0.9. What we will see is that the likelihood is highest around p = 0.7 — because 7/10 heads is the outcome most consistent with a coin biased exactly 70% toward heads. If we plot the likelihood function across all values of p, its peak gives us the parameter that best explains the data.

This is exactly what parameter estimation is: finding that peak by choosing the parameter value that maximizes the likelihood function. The principle is so central that it has its own name — maximum likelihood.

Maximum likelihood estimation (MLE) is the parameter estimation method that chooses the parameter value making the observed data most likely. Its intuition is very natural: the data we hold actually happened; so let us take the model setting that best explains it as our estimate.

In the coin-flip example above, the maximum likelihood estimate is simply p = 0.7: if we saw 7 heads in 10 tosses, estimating the heads probability as 0.7 is the value that makes the data most plausible. This result matches intuition and shows why maximum likelihood is so widespread. In statistics, many familiar estimators — the mean, the variance, regression coefficients — are actually maximum likelihood results under certain assumptions.

Maximum likelihood is also the backbone of machine learning. To "train" a model often means to set its parameters so as to maximize the likelihood of the training data. To see this link in a broader frame, see the what is machine learning and what is an algorithm guides.

How to

Parameter estimation with maximum likelihood

The core steps of estimating a parameter from observed data using the maximum likelihood method.

1. 1

   Choose a model

   Pick a probability model for the process that generated the data and its parameters (e.g. binomial, heads probability p).

2. 2

   Write the likelihood function

   Holding the observed data fixed, express the likelihood as a function of the parameter.

3. 3

   Switch to log-likelihood

   Take the logarithm to turn the product into a sum and gain numerical stability.

4. 4

   Find the maximum

   Optimize the log-likelihood over the parameter; the value at the peak is the parameter estimate.

In theory we could maximize the likelihood directly; in practice we almost always use its logarithm, the log-likelihood. There are two strong reasons.

The first is numerical. The joint likelihood of several independent observations is the product of the individual likelihoods. Over hundreds or thousands of observations, this product shrinks to extremely small numbers and causes numerical underflow on a computer. The logarithm removes this problem by turning the product into a sum: the log-likelihood is the sum of the individual log-likelihoods. The second reason concerns optimization. Differentiating and maximizing a sum is far easier than doing so for a product, which makes parameter estimation easier both analytically and numerically.

The critical point is this: because the logarithm is an increasing (monotonic) function, the parameter that maximizes the log-likelihood is the same as the one that maximizes the likelihood. So switching to the logarithm does not change the answer, it only eases the computation. That is why in the statistics and deep learning literature "maximize the likelihood" and "maximize the log-likelihood" mean the same thing in practice.

Comparison of probability, likelihood, and log-likelihood

Concept	Function of what?	Held fixed	Typical use
Probability	The data	Parameter	Predicting the chance of an outcome
Likelihood	The parameter	Data	Parameter estimation (MLE)
Log-likelihood	The parameter	Data	Numerically stable optimization
Negative log-likelihood	The parameter	Data	Loss function (cross-entropy)

Though it looks like an abstract statistics concept, likelihood is at the very center of modern AI. To train a model most often means to set its parameters so as to maximize the likelihood of the training data. The model searches for the internal settings that make the data it observed most likely; this is a maximum likelihood problem.

The most concrete connection is in the loss function. The cross-entropy loss, common in classification and language models, is in fact the negative log-likelihood. Reducing the model's loss and increasing the log-likelihood of the training data are mathematically the same thing. A large language model (LLM) learns, at each step, parameters that assign the highest possible likelihood to the next token; training is a giant log-likelihood maximization over billions of tokens.

This principle appears not only in huge models but also in basic methods. Logistic regression is a classic example that estimates its parameters by maximum likelihood; deep learning scales the same principle to millions of parameters. Generative AI models also try to learn distributions that can produce the data with high likelihood. In short, likelihood is the common language between classical statistics and artificial intelligence.

Likelihood is powerful but not magic; maximum likelihood has limits worth knowing. The best-known problem is overfitting: with little data, maximum likelihood can produce extreme, unreliable estimates. If 3 of 3 tosses come up heads, MLE estimates the heads probability as 1.0 — yet this does not mean the coin will never come up tails, only that the data is scarce.

The second problem is dependence on the model assumption. Likelihood is always defined within a model; if the wrong model is chosen, finding the best parameter of that model still does not give the right answer. Third, interpreting likelihood as if it were a probability is a common mistake — likelihood values do not sum to 1 over the parameter and cannot be read alone as "the probability of that parameter."

These limits have given rise to approaches that complement likelihood. Bayesian statistics softens extreme estimates by adding prior knowledge to the likelihood; regularization provides a similar balancing in machine learning. So the full answer to what is likelihood requires seeing both its power and the places where it falls short on its own.

What is the difference between likelihood and probability?

In probability the parameter (the model's setting) is fixed and you ask about the chance of different data outcomes. In likelihood the data is fixed (you have an observation) and you ask how well different parameter values explain that data. The mathematical expression is the same function; what changes is what is held fixed and what varies. This is why likelihood is not a probability distribution and does not have to sum to 1.

What is maximum likelihood used for?

Maximum likelihood estimation (MLE) finds the parameter value that makes the observed data most likely. It answers "which model setting best explains this data?" If you saw 7 heads and 3 tails in a coin series, estimating the heads probability as 0.7 is an MLE result. It is the most common parameter estimation method in statistics and machine learning.

Why is log-likelihood used?

There are two practical reasons. First, the likelihood of many observations is their product, which shrinks to very small numbers and causes numerical underflow on a computer; the logarithm turns the product into a sum and prevents this. Second, differentiating and optimizing sums is far easier than products. Because the logarithm is an increasing function, maximizing the log-likelihood gives the same parameter as maximizing the likelihood.

Where is likelihood used in AI?

Training modern AI models is largely a likelihood maximization. A language model learns parameters that assign high likelihood to the next word in the training data. The common cross-entropy loss function is in fact the negative log-likelihood; minimizing it is maximizing the likelihood. From logistic regression to deep neural networks, many methods rest on this principle.

Is a likelihood function a probability?

No. Likelihood is a function of the parameter; probability is a function of the data. For fixed data, the likelihood values computed across different parameters do not have to sum (or integrate) to 1, because likelihood does not define a probability distribution over the parameter. Missing this distinction is one of the most common conceptual mistakes in statistics.

Does maximum likelihood always give the right answer?

No. With little data, maximum likelihood is prone to overfitting and can produce extreme estimates; for example, if 3 of 3 tosses come up heads it estimates the heads probability as 1.0. It can also be biased under a wrong model assumption. That is why in practice likelihood is balanced with regularization or by adding prior knowledge through Bayesian approaches.

In short, the answer to what is likelihood is: a function that, holding the observed data fixed and varying a model's parameters, measures how well those parameters explain the data. The probability-vs-likelihood difference — whether the data or the parameter is fixed — is the heart of the concept. Maximum likelihood performs parameter estimation by choosing the value that makes the data most likely; log-likelihood makes this computation numerically stable; and this whole machinery, through the cross-entropy loss, forms the basis of modern AI training. To reinforce the foundation, see the what is machine learning and what is logistic regression guides, and to map out the right AI roadmap for your organization start with AI consulting.