What Is Linear Regression? Formula, Assumptions, and Examples

Linear regression is a statistical method that models the relationship between a dependent variable and one or more independent variables by fitting a straight line to the data. It uses that line to predict a continuous outcome, which makes it one of the foundations of both predictive analytics and machine learning.

It is among the oldest and most widely used techniques in statistics, valued because it is simple to fit, easy to interpret, and a dependable starting point before more complex models are tried.

Key Takeaways

• Linear regression models the relationship between variables by fitting a straight line that predicts a continuous outcome.
• Simple linear regression uses one predictor; multiple linear regression uses two or more.
• The line is found with Ordinary Least Squares, which minimizes the sum of the squared residuals.
• Model quality is read through R², which measures explained variance, and p-values, which test significance.
• It rests on five assumptions: linearity, independence, homoscedasticity, normality of errors, and no multicollinearity.

What Is Linear Regression?

Linear regression is a statistical method that models the relationship between a dependent variable and independent variables to predict a continuous value.

Linear regression examines how a dependent variable, the outcome you want to predict, relates to one or more independent variables, the predictors that explain it. It assumes that relationship can be described by a straight line.

Once the line is fitted, it serves two purposes. It explains the strength and direction of the relationship between the variables, and it predicts the outcome for new inputs. Because the predicted value is a continuous number, linear regression suits questions of “how much” or “how many,” such as estimating revenue, temperature, or demand.

The method comes in two main forms depending on how many predictors are involved, covered further below.

What Is Linear Regression in Machine Learning?

In machine learning, linear regression is a supervised algorithm that learns the relationship between input features and a continuous target to make predictions.

Linear regression began as a statistical technique, and machine learning uses it as a supervised learning algorithm. The two views describe the same method with a difference in emphasis.

Classical statistics uses linear regression mainly to explain relationships, testing which variables matter and how confident the estimates are. Machine learning uses it mainly to predict, training the model on labeled data so it can estimate the target for new, unseen inputs.

In machine learning terms, the independent variables are the input features, the dependent variable is the target, and the coefficients are the weights the model learns during training. For large datasets, those weights are often found through gradient descent rather than a direct formula, and the trained model is judged on how accurately it predicts on data it has not seen. Despite its simplicity, linear regression remains a common baseline, since it is fast to train and easy to interpret before more complex models are attempted.

What Are the Key Concepts of Linear Regression?

The key concepts are the dependent and independent variables, the slope, the intercept, the line of best fit, and residuals.

Dependent and independent variables: The dependent variable (y) is the outcome being predicted. The independent variables (x) are the predictors used to explain it.

Slope and intercept: The slope shows how much y changes for each one-unit increase in x. The intercept is the value of y when x equals zero, the point where the line crosses the vertical axis.

Line of best fit: Many straight lines could pass through a scatter of data points. The line of best fit is the single line that sits closest to all of them at once, and finding it is the heart of the method.

Residuals: A residual is the gap between an actual data point and the value the line predicts for it. Residuals measure the error in the fit, and the best-fit line is the one that makes them as small as possible overall.

What Are the Types of Linear Regression?

The two main types are simple linear regression, with one predictor, and multiple linear regression, with two or more.

Linear regression is grouped by how many independent variables it uses.

Simple Linear Regression

Simple linear regression uses one independent variable to predict the outcome, such as estimating sales from advertising spend alone. It is the most basic form and maps directly to the straight-line equation y = mx + b.

Multiple Linear Regression

Multiple linear regression uses two or more predictors together, such as estimating sales from advertising spend, price, and season at once. It is the more common form in practice, since most real outcomes have several drivers, and it follows the equation y = β₀ + β₁x₁ + … + βₙxₙ.

Polynomial Regression

Polynomial regression is a related extension that fits a curved line by raising a predictor to a power, for instance using x² to model a U-shaped relationship. It is still considered linear regression, because the equation stays linear in its coefficients even though the curve it traces is not straight.

What Is the Linear Regression Formula?

Simple linear regression uses the straight-line equation y = mx + b, which extends to y = β₀ + β₁x₁ + … + βₙxₙ for multiple predictors.

Simple linear regression rests on the familiar equation of a straight line:

y = mx + b

In this form, y is the outcome being predicted, x is the predictor, m is the slope (how much y moves per one-unit change in x), and b is the intercept (the value of y when x is zero).

In statistics and machine learning, the same equation is usually written with β to denote the coefficients:

y = β₀ + β₁x + ε

Here β₀ is the intercept, β₁ is the slope, and ε is the error term, the part of y the line cannot explain. When a model has several predictors, it extends to multiple linear regression:

y = β₀ + β₁x₁ + β₂x₂ + … + βₙxₙ

Each βᵢ is the coefficient for its predictor, representing the change in y for a one-unit increase in that variable while the others are held constant.

How Does Linear Regression Find the Best-Fit Line?

Linear regression finds the best-fit line using Ordinary Least Squares, which minimizes the sum of the squared residuals.

The goal is the line that sits as close as possible to every data point. The standard way to find it is Ordinary Least Squares (OLS).

OLS works in three moves. It measures the residual, the vertical distance from each point to the line. It squares each residual, so that positive and negative gaps do not cancel out and larger errors weigh more heavily. It then adjusts the slope and intercept until the sum of those squared residuals reaches its smallest possible value. That sum is the cost the model minimizes, often expressed as the mean squared error (MSE):

MSE = (1/n) Σ (yᵢ − ŷᵢ)²

where yᵢ is the actual value and ŷᵢ is the predicted value.

For simple linear regression, OLS has a closed-form solution. The slope and intercept can be calculated directly:

β₁ = Σ(xᵢ − x̄)(yᵢ − ȳ) / Σ(xᵢ − x̄)²

β₀ = ȳ − β₁x̄

where x̄ and ȳ are the means of the predictor and the outcome. The slope formula is the covariance of x and y divided by the variance of x, and the intercept places the line so it passes through the average point.

For large datasets with many predictors, an iterative method called gradient descent is often used instead. It starts with rough coefficients and adjusts them step by step in the direction that lowers the error, arriving at the same minimum without solving the equations directly.

How Do You Interpret a Linear Regression Model?

The slope and intercept describe the relationship, while R² and p-values measure how well the model fits and whether the relationship is significant.

Interpretation happens at two levels: reading the coefficients, then judging the model as a whole.

Reading the Coefficients

Each coefficient is read directly, which is one of the method’s strengths. A slope of 5 on an advertising-spend predictor means that every additional unit of spend is associated with a 5-unit rise in predicted sales, holding the other predictors fixed. A positive slope means the outcome rises with the predictor; a negative slope means it falls. The intercept gives the predicted outcome when all predictors are zero.

Evaluating the Model

Two metrics carry most of the interpretation of overall fit.

R-squared (R²) is the share of the variance in the outcome that the model explains, on a scale from 0 to 1, often read as a percentage. A higher R² generally signals a closer fit, though a very high value can also hint at overfitting. 

Adjusted R² refines this for multiple regression, since plain R² never falls when predictors are added; adjusted R² penalizes predictors that do not earn their place.

P-values test statistical significance. For each coefficient, the p-value indicates whether the relationship is likely real or could be down to chance. A low p-value, typically below 0.05, suggests the predictor contributes meaningfully. Error metrics such as MAE, MSE, and RMSE round out the picture by measuring the typical size of the prediction error.

What Are the Assumptions of Linear Regression?

Linear regression rests on five assumptions: linearity, independence, homoscedasticity, normality of errors, and no multicollinearity.

Linear regression produces valid, unbiased results only when several assumptions about the data hold. A common way to remember the core four is the mnemonic LINE: Linear, Independent, Normal, Equal variance.

• Linearity: The relationship between the predictors and the outcome is genuinely a straight line. A scatter plot or a residual plot reveals whether this holds; if the pattern curves, the model is misspecified.
• Independence: The observations, and their errors, are independent of one another. This breaks most often in time-ordered data, where one period’s value carries into the next, a condition called autocorrelation.
• Normality of errors: The residuals are roughly normally distributed around zero. This matters most for the reliability of p-values and confidence intervals, and a Q-Q plot or histogram of residuals is the usual check.
• Homoscedasticity: The residuals have constant variance across all levels of the predictors. When the spread of errors fans out as the predicted value grows, the data is heteroscedastic, which undermines the standard errors.
• No multicollinearity: In multiple regression, the predictors are not strongly correlated with one another. Heavy correlation makes the individual coefficients unstable and hard to interpret, and it is checked with the variance inflation factor (VIF).

When these conditions are met, the Gauss-Markov theorem guarantees that OLS produces the Best Linear Unbiased Estimator (BLUE), meaning the most precise estimates available among all linear, unbiased methods. If an assumption is violated, options include transforming a variable, adding a missing predictor, or switching to a method such as generalized least squares or regularized regression.

What Are the Applications of Linear Regression?

Linear regression is applied in sales forecasting, financial analysis, risk assessment, medical research, and operations planning.

Linear regression appears across fields wherever the goal is to predict a continuous value or quantify a relationship.

• Sales and demand forecasting: Estimating future sales from drivers such as price, advertising spend, and seasonality.
• Financial analysis: Modeling how factors like interest rates or market indices relate to returns, including the basis of techniques such as the capital asset pricing model.
• Risk assessment: Quantifying how risk factors relate to an outcome, for instance how claim size relates to policyholder attributes in insurance.
• Medical and scientific research: Measuring how variables such as dosage, age, or exposure relate to a continuous health measure.
• Operations and resource planning: Predicting quantities such as energy consumption or staffing needs from historical drivers.

Frequently Asked Questions

1. What is linear regression?

Linear regression is a method that draws the straight line best fitting a set of data points, then uses that line to predict a number. For example, it can estimate sales from advertising spend by learning the relationship between the two.

2. What is the difference between simple and multiple linear regression?

Simple linear regression uses one independent variable to predict the outcome. Multiple linear regression uses two or more predictors together, which usually models real situations more accurately.

3. What is the least squares method?

Ordinary Least Squares is the standard way to fit the line. It finds the slope and intercept that minimize the sum of the squared residuals, the squared gaps between the actual data points and the line.

4. What does R-squared mean in linear regression?

R-squared is the proportion of the variance in the outcome that the model explains, from 0 to 1. A higher value means the predictors account for more of the variation, though a very high value can signal overfitting.

5. What are the assumptions of linear regression?

Linearity, independence of observations, homoscedasticity (constant error variance), normality of the residuals, and no multicollinearity among predictors. The first four are captured by the LINE mnemonic.