What Is Logistic Regression? How It Works, Types & Assumptions

Logistic regression is a supervised machine learning algorithm that predicts the probability that a data point belongs to a particular category. It relies on the sigmoid function to map a linear output to a probability between 0 and 1, which makes it well suited to binary classification.

Although its name contains “regression,” logistic regression performs classification. It is one of the most widely used algorithms in machine learning and statistics, valued for being fast, interpretable, and a dependable baseline for any classification problem.

Key Takeaways

• Logistic regression predicts the probability that an input belongs to a class, using the sigmoid function to keep the output between 0 and 1.
• It performs classification, even though “regression” appears in its name.
• The three main types are binary, multinomial, and ordinal, set by the nature of the dependent variable.
• It is trained through maximum likelihood estimation, using the log-loss cost function and gradient descent.
• Coefficients are interpreted through odds ratios, which keeps the model transparent and easy to explain.

What Is Logistic Regression?

Logistic regression is a supervised classification algorithm that predicts the probability of a categorical outcome using the sigmoid function.

Logistic regression is a supervised learning algorithm used for classification. It estimates the probability that a given input belongs to a particular category, then uses that probability to assign a class.

Like linear regression, it examines the relationship between one or more independent variables (the predictors) and a dependent variable (the outcome). The two differ in what they predict. Linear regression predicts a continuous number, such as a house price. Logistic regression predicts a categorical result, such as whether an email is spam.

The output is a probability between 0 and 1. A spam model that returns 0.93 for an email is estimating a 93 percent chance the message is spam. That probability can be used as it stands or converted into a class label by applying a threshold.

How Does Logistic Regression Work?

Logistic regression works by passing a linear equation through the sigmoid function, then applying a threshold to assign a class.

Where linear regression fits a straight line to the data, logistic regression fits an S-shaped logistic curve. It reaches that curve in three steps.

1. The Linear Equation

The model starts with a weighted sum of the input features:

z = β₀ + β₁x₁ + β₂x₂ + … + βₙxₙ

Each x is a feature, each β is a coefficient the model learns, and z is the result. On its own, z can take any value from negative to positive infinity, which is why it cannot serve as a probability yet.

2. The Sigmoid Function

The model passes z through the sigmoid function, also called the logistic function:

σ(z) = 1 / (1 + e⁻ᶻ)

This compresses any real number into the range 0 to 1, producing the S-shaped curve. As z grows large and positive, the output approaches 1. As z grows large and negative, it approaches 0. At z = 0, the output is exactly 0.5. The result is the probability that the input belongs to the positive class.

3. The Threshold

To turn the probability into a decision, the model applies a threshold, typically 0.5. If σ(z) is 0.5 or higher, it predicts the positive class (1). If it falls below 0.5, it predicts the negative class (0). The point where the prediction switches from one class to the other is the decision boundary.

The threshold can be adjusted. In fraud detection, where missing a fraudulent case is expensive, a lower threshold catches more positives at the cost of more false alarms.

How Do Odds and the Logit Connect to the Model?

Logistic regression models the log-odds of the outcome as a linear equation, and the sigmoid converts those log-odds back into a probability.

The bridge between the linear equation and the probability is the idea of odds. The odds of an event are the probability it happens divided by the probability it does not:

odds = p / (1 − p)

Odds run from 0 to infinity, which is not balanced around 1, so logistic regression takes their natural log. That produces the log-odds, or logit, which spans the full real number line and is what the linear equation actually models:

ln(p / (1 − p)) = β₀ + β₁x₁ + … + βₙxₙ

Exponentiating both sides returns the odds, and solving for p returns the sigmoid function. This is the mathematical core of the method: logistic regression is linear in the log-odds, and the sigmoid is the transformation that converts those log-odds into a probability bounded between 0 and 1.

What Are the Types of Logistic Regression?

The three types are binary, multinomial, and ordinal, defined by how many categories the outcome has and whether they are ordered.

Logistic regression takes three forms, set by the nature of the outcome variable.

Binary (Binomial) Logistic Regression

The outcome has exactly two categories, such as pass/fail, spam/not spam, or 0/1. This is the most common form and the default most people mean by logistic regression.

Multinomial Logistic Regression

The outcome has three or more categories with no natural order, such as classifying a fruit as an apple, orange, or banana. The model estimates the probability of each category and selects the most likely.

Ordinal Logistic Regression

The outcome has three or more categories that follow a meaningful order, such as rating a service as poor, average, or excellent. It works like multinomial regression but respects the ranking between categories.

Logistic Regression vs Linear Regression

Linear regression estimates a numeric value along a continuous scale, while logistic regression estimates the probability of a category.

Both methods fit a model to predictor variables, and they part ways on what they produce. Linear regression draws a straight line to estimate a quantity. Logistic regression bends that line into an S-curve to estimate a probability, then turns it into a class.

When to Use Which

Use linear regression when the target is a measurement that can move smoothly up or down, like estimating the energy a building will draw next month or the time a delivery will take. The output belongs on a number line.

Use logistic regression when the target is a verdict, like flagging a loan application as approve or decline, or scoring a part on a production line as pass or fail. The output is a category, and the probability behind it tells you how confident the call is.

What Are the Assumptions of Logistic Regression?

Logistic regression assumes a binary outcome, independent observations, little multicollinearity, and a linear link between predictors and the log-odds.

Logistic regression rests on a different set of assumptions from linear regression. It does not require the outcome to be normally distributed, the errors to share equal variance (homoscedasticity), or the predictors to relate linearly to the outcome itself. What it does assume falls into the following.

Binary dependent variable: Standard logistic regression expects a dichotomous outcome, such as yes/no, 0/1, or pass/fail. The multinomial and ordinal variants extend this to more than two categories.

Independent observations: Each record should stand on its own, with no correlation between data points. Repeated measurements from the same subject, for example, break this assumption and call for a different approach.

Linearity between predictors and the log-odds: The outcome is binary, yet each continuous predictor is assumed to have a linear relationship with the log-odds of the outcome, ln(p / (1 − p)). This is the logistic counterpart to linear regression’s linearity assumption.

Little or no multicollinearity: The independent variables should not be strongly correlated with one another. Heavy correlation makes the coefficient estimates unstable and hard to interpret, since the model struggles to separate the effect of one predictor from another.

No extreme outliers: Influential outliers can pull the coefficients out of shape and distort the fit, so identifying and addressing them keeps the model sound.

A large enough sample: Maximum likelihood estimation needs a reasonably large dataset for stable estimates. A common rule of thumb is at least 10 cases of the less frequent outcome for every predictor in the model.

How Is a Logistic Regression Model Trained?

Logistic regression is trained through maximum likelihood estimation, the log-loss cost function, and gradient descent to find the best coefficients.

Training a logistic regression model means finding the coefficients that make the observed data most likely.

The model learns its coefficients through maximum likelihood estimation (MLE), which selects the values that assign the highest probability to the outcomes actually seen in the training data. The likelihood across all records is the product of each predicted probability:

L(β₀, β₁) = ∏ p(xᵢ)^yᵢ · (1 − p(xᵢ))^(1 − yᵢ)

When the true label yᵢ is 1, the first term applies and rewards a predicted probability close to 1. When yᵢ is 0, the second term applies and rewards a predicted probability close to 0. Multiplying across all records assumes the observations are independent. Because this product becomes very small, the calculation works with the log-likelihood, which turns the product into a sum.

The error is measured by the log-loss, also called binary cross-entropy:

Log-loss = −[ y·log(ŷ) + (1 − y)·log(1 − ŷ) ]

Here y is the true label and ŷ is the predicted probability. The function penalizes confident wrong predictions heavily, which pushes the model toward well-calibrated probabilities. Gradient descent then adjusts the coefficients step by step to minimize this loss. In practice, libraries such as scikit-learn handle the fitting in a few lines of code.

How Do You Interpret Logistic Regression Coefficients?

Each coefficient is the change in log-odds per one-unit increase in a predictor, and exponentiating it gives the odds ratio.

Coefficient interpretation is where logistic regression earns its reputation for being explainable. The coefficients carry a different meaning from linear regression coefficients, because the model predicts log-odds.

Key Interpretation Steps

A coefficient, β, is the change in the log-odds of the outcome for a one-unit increase in its predictor. Reading it usefully takes a few steps.

• Check the sign: A positive coefficient increases the likelihood of the outcome as the predictor rises; a negative coefficient decreases it; a coefficient of zero means no effect.
• Exponentiate to get the odds ratio: e^β converts the log-odds coefficient into an odds ratio (OR). An OR above 1 means a higher predictor value raises the odds of the outcome, and an OR below 1 means it lowers them. The derivation is direct: if the odds at x₁ are e^(β₀+β₁x₁), raising x₁ by one gives e^(β₀+β₁x₁) · e^β₁, so the odds are multiplied by e^β₁.
• Convert to a percentage change: The odds ratio reads naturally as a percentage. For a positive coefficient, the odds rise by (e^β − 1) × 100 percent per one-unit increase. For a negative coefficient, they fall by (1 − e^β) × 100 percent. An OR of 1.25 is a 25 percent increase in the odds.
• Hold the other predictors constant: Every interpretation assumes the model’s other predictors stay fixed, so the coefficient reflects the effect of that one variable on its own.

For a categorical predictor, the reading is comparative. If “has existing debt” is coded 1 and “no debt” is 0, then e^β is the odds ratio of approval for someone with debt against someone without it.

Interpreting the Intercept

The intercept, β₀, is the log-odds of the outcome when every predictor equals zero. Exponentiating it, e^β₀, gives the baseline odds at that reference point, before any predictor moves them.

Whether that baseline carries meaning depends on whether “all predictors at zero” describes a real case. If the predictors are centered or scaled, zero represents the average observation, so the intercept describes the odds for a typical case. If a predictor such as age can never realistically be zero, the intercept serves as a mathematical anchor for the model.

Alternative Interpretation Through Probabilities

Odds ratios are precise, though many audiences think in probabilities. The complication is that a one-unit change in a predictor moves the probability by a varying amount, because the sigmoid curve is steep in the middle and flat at the ends.

Near a probability of 0.5, where the curve is steepest, a one-unit change shifts the probability the most. Near 0 or 1, where the curve flattens, the same change barely registers. The effect of a predictor on probability therefore depends on where the other predictors place the observation along the curve.

To express a coefficient in probability terms, convert the log-odds back through the sigmoid. A quick shortcut is the “divide by 4” rule: dividing a coefficient by 4 approximates the largest change in probability a one-unit increase can produce, which occurs at the steepest part of the curve. It gives a fast, plain-language read on impact without solving the full equation.

How Is a Logistic Regression Model Evaluated?

Logistic regression is evaluated with a confusion matrix and metrics such as accuracy, precision, recall, F1 score, and ROC-AUC.

As a classifier, logistic regression is judged on classification metrics.

• Confusion matrix: A table of true positives, true negatives, false positives, and false negatives that underlies the other metrics.
• Accuracy: The share of all predictions that were correct, useful when the classes are balanced.
• Precision and recall: Precision measures how many predicted positives were truly positive, while recall measures how many actual positives the model caught. The two trade off against each other.
• F1 score: The harmonic mean of precision and recall, a single number balancing both, helpful when classes are imbalanced.
• ROC-AUC: The area under the ROC curve, summarizing how well the model separates the classes across every threshold. A value of 1.0 is perfect separation and 0.5 is no better than chance.

Beyond these, statistical tests such as the likelihood ratio test and measures like pseudo-R² help assess how well the overall model fits the data.

What Are the Applications of Logistic Regression?

Logistic regression is applied in fraud detection, spam filtering, disease prediction, credit scoring, and customer churn.

Logistic regression appears across industries wherever the goal is to classify or estimate the probability of an outcome.

• Fraud detection: Estimating the probability that a transaction is fraudulent from features such as amount, location, and timing, where its interpretable coefficients help explain why a case was flagged.
• Spam filtering: Classifying an email as spam or not spam from the words and patterns it contains.
• Disease prediction: Estimating the likelihood that a patient has a condition from clinical predictors, a long-standing use in medical research.
• Credit scoring and loan approval: Predicting the probability that an applicant will default, using inputs such as income, credit history, and savings.
• Customer churn: Estimating which customers are likely to leave, so retention teams can act on the highest-risk accounts first.

Frequently Asked Questions

1. What is logistic regression?

Logistic regression is a method that predicts the probability of a yes-or-no outcome. It takes input features, calculates a probability between 0 and 1 with the sigmoid function, and assigns a class based on a threshold.

2. Is logistic regression a classification or regression algorithm?

It is a classification algorithm. The word “regression” reflects its mathematical roots in linear regression, but it predicts categories rather than continuous values.

3. Why does logistic regression use the sigmoid function?

The sigmoid function maps any number to a value between 0 and 1, which lets the model output a valid probability. A plain linear equation could return values outside that range, which cannot represent a probability.

4. What is the difference between logistic and linear regression?

Linear regression predicts continuous values such as price, while logistic regression predicts categorical outcomes as probabilities. Linear regression fits a straight line, and logistic regression fits an S-shaped curve.

5. What are the three types of logistic regression?

Binary for two categories, multinomial for three or more unordered categories, and ordinal for three or more ordered categories. The type depends on the nature of the outcome variable.