Multiple Regression: Formula, Analysis & Assumptions

Content Curator

Multiple linear regression is used to estimate the link between two or more independent variables and one dependent variable. It can be used to know the extent to which two or more independent variables and one dependent variable are related and the dependent variable's value at a given value of the independent variables.

Multiple regression is a variant of linear (OLS) regression in which just one explanatory variable is used. It's widely utilized in econometrics and financial analysis.

Read Also: Math Formulas

Table of Content

Formula of Multiple Linear Regression
Stepwise Multiple Regression
Difference Between Simple Linear Regression and Multiple Regression
Multiple Regression Analysis
Multiple Regression Assumptions
Multicollinearity
Multivariate Multiple Regression
Things to Remember
Sample Questions

Key Takeaways- Multiple linear regression, Multiple regression, Formula, Simple linear regression, Multicollinearity, Multiple Regression Analysis

Formula of Multiple Linear Regression

[Click Here for Sample Questions]

The formula for a multiple linear regression is:

y_i =β₀ +β₁ xi₁ +β₂ xi₂ +...+β_p xi_p +ϵ

Here, for i = n observations

y_i = dependent variable

x_i = explanatory variables

β₀ = y-intercept (constant term)

β_p = slope coefficients for each explanatory variable

ϵ = the model’s error term (also known as the residuals)

Stepwise Multiple Regression

[Click Here for Sample Questions]

Stepwise regression is a step-by-step technique that starts with a single predictor variable and builds a regression model from there, adding and deleting predictor variables one at a time. Stepwise multiple regression is a method for determining a regression equation that starts with a single independent variable and gradually increases the number of independent variables.

Because we start with no independent variables and add one independent variable to the regression equation at each iteration, the stepwise multiple regression approach is also known as the forward selection method. Another strategy is the backward elimination method, which starts with a large number of variables and eliminates one independent variable at a time.

Advantages of Stepwise Multiple Regression

There are two major advantages to applying a multiple regression model to analyse data.

The first is the capacity to determine one or more predictor variables' relative influence on the criteria value. The real estate agent may discover that the size of the homes and the number of bedrooms have a substantial association with the price of a home, whereas proximity to schools has no correlation, or even a negative correlation if the community is largely a retirement community.
The ability to detect outliers, or abnormalities, is the second advantage. For example, a human resources manager evaluating data on management pay would discover that the number of hours worked, the size of the department, and its budget all showed a substantial association to compensation, whereas seniority did not. Alternatively, it's possible that all of the mentioned predictor values were linked to each of the wages studied, with the exception of one manager who was overpaid in comparison to the rest.

Difference Between Simple Linear Regression and Multiple Regression

[Click Here for Sample Questions]

Simple Linear Regression	Multiple Regression
There is only one x and one y variable in simple linear regression.	There are two x and one y variables in multiple regression.
Simple linear regression is used to anticipate rent based just on square footage.	Multiple regression is used to forecast rent based on square footage and the age of the building.
One regression coefficient.	One regression coefficient for each independent variable.

Multiple Regression Analysis

[Click Here for Sample Questions]

A statistical strategy for predicting the value of a dependent variable based on the values of two or more independent variables is known as multiple regression analysis.Because the outcome or value of the anticipated value is reliant on the behavior of other variables, it is referred to as a dependent variable. The value of the independent variables is usually determined by looking at the sample or population.

Check Important Differences Between in Maths

Multiple Regression Assumptions

[Click Here for Sample Questions]

Assumption of linearity is important.
Multiple regression models should be linear in nature.
In multiple regression, the assumption of homoscedasticity is required.
Between the independent variables, there is a low degree of correlation.
The variance of the independent variable is constant at all levels.
In multiple regression, the assumption of normality is required. It means that variables in multiple regression must have a normal distribution.
In multiple regression, the model should be specified in a methodical manner. It suggests that the model should contain just important variables and be accurate.

Multicollinearity

[Click Here for Sample Questions]

When the inter-correlation of independent variables is large, it is referred to as multicollinearity. It usually happens when independent variables in a regression model are correlated.

Multicollinearity Testimony:

The degree of correlation between independent variables is rather high.
The magnitude or symbols of regression coefficients are incomprehensible.
The addition or deletion of an independent variable is caused by the greatest sensitivity of magnitude or sign of regression coefficients.
Regression coefficients on significant independent variables that are non-significant.

Multivariate Multiple Regression

[Click Here for Sample Questions]

The linear relationship between more than one independent variable and more than one dependent variable (DV) is modeled using multivariate multiple regression (MMR) (DV). Because there are numerous independent variables, MMR is called multiple. Because there are more than one dependent variable, MMR is multivariate.

Things to remember

[Click Here for Sample Questions]

Multiple linear regression is used to identify the mathematical relationship between numerous random variables.
You'll almost certainly need to utilize specialized statistical tools or capabilities inside applications like Excel to do a multiple regression.
Multiple regression is a statistical method for predicting the outcome of a response variable by integrating a number of explanatory variables.
In the regression equation, only independent variables with non-zero regression coefficients are considered.
Finding the regression equation with just significant regression coefficients is simple with stepwise multiple regression.

Sample Questions

Ques. How could I know if there are significant differences between two means adjusted for other variables in a multiple linear regression model? (3 marks)

Ans. The simplest approach from where you are currently is to add another predictor to the multiple regression model that codes the two groups as 0 and 1 (use 0 for control/reference and 1 for treated/experimental). The regression model will then offer a coefficient for this predictor, which is the mean difference in response between the groups after all other predictors in the model have been taken into account. The programme will provide a standard error of this difference, as well as a t-value, p-value, and confidence interval - everything that is needed to assess the importance of this mean difference.

Ques. There are two series of index numbers P for price index and S for stock of the commodity. The mean and standard deviation of P are 100 and 8 and of S are 103 and 4 respectively. The correlation coefficient between the two series is 0.4. With these data obtain the regression lines of P on S and S on P. (5 marks)

Ans. Consider the symbols X for price P and Y for stock S.

The mean and SD for P are then calculated as X-Bar = 100 and σx= 8. S's mean and SD are Y-Bar =103 and σy=4, respectively. r(X,Y)=0.4 is the correlation coefficient between the two series.

Let X on Y be the regression line.

Data

Ques. The two regression lines are 3X+2Y=26 and 6X+3Y=31. Find the correlation coefficient. (5 marks)

Ans. Let the regression equation of Y on X be

3X+2Y = 26

Data

Ques- Find the means of X and Y variables and the coefficient of correlation between them from the following two regression equations: (5 marks)

2Y–X–50 = 0

3Y–2X–10 = 0.

Ans.We are given

2Y–X–50 = 0 ... (1)

3Y–2X–10 = 0 ... (2)

Solving equation (1) and (2)

We get Y = 90

Putting the value of Y in equation (1)

We get X = 130

Hence, X= 130Y= 90

Calculating correlation coefficient

Assume that equation (1) is the Y on X regression equation.

2Y = X+50

Data

Ques. Find linear regression equation for the following two sets of data: (5 marks)

Ans. Construct the following table:

x	y	x²	xy
2	3	4	6
3	6	9	18
5	5	25	25
8	12	64	96
X=18	Y=26	X²=102	XY=145

Now we'll use the formula to determine the values of a and b:

Data

Ques. What are the Types of Linear Regression? (3 marks)

Ans. There are six types of Linear Regression:

Simple Linear Regression
Multiple Linear Regression
Logistic Regression
Ordinal Regression
Multinomial Regression
Discriminant Analysis

Ques. What are the Advantages of Multiple Regression? (4 marks)

Ans. There are two significant advantages to utilizing a multiple regression model to analyze data.

MLR is capable of determining the relative effect of several independent variables on criteria values. Real estate brokers may discover that the size of a room and the number of bedrooms have a substantial relationship with the cost of a home, but proximity to schools has little or even a negative relationship.
The capacity to discover outliers or anomalies is the second advantage of multiple regression. The human resource manager, for example, may discover the number of hours worked, the size of the department, and its budget when looking at data connected to management compensation. All of them have a significant link to pay, but seniority does not. In other words, the independent factors were connected to each of the wages under consideration, with the exception of the manager who was overpaid compared to others.

Mathematics Related Links:
Linear Regression Formula	Differential equations	Mode
Roots of Polynomials	Statistics	Nature of Roots of Quadratic Equation
Linear Equation in Two Variables	Number System	Algebraic Operations on Complex Numbers
Sequence and Series	Polynomials in One Variable	Formation of a Differential Equation
Parabola Formula	Events in Probability	Mode, Mean
Ordinary Differential Equations	NCERT Solution for Class 11 Mathematics	Maths Study Material

Multiple Regression: Formula, Analysis & Assumptions

Formula of Multiple Linear Regression

Stepwise Multiple Regression

Advantages of Stepwise Multiple Regression

Difference Between Simple Linear Regression and Multiple Regression

Multiple Regression Analysis

Multiple Regression Assumptions

Multicollinearity

Multivariate Multiple Regression

Things to remember

Sample Questions

CBSE CLASS XII Related Questions

1.
Find:

The shortest distance between the lines: \[ \vec{r}=(4+\lambda)\hat{i}+(2\lambda-1)\hat{j}-3\lambda\hat{k} \] and \[ \vec{r}=(1+2\mu)\hat{i}+(4\mu-1)\hat{j}+(2-5\mu)\hat{k} \]

2.
A carpenter needs to design a wooden box in the shape of a cuboid such that the sum of its length and breadth is 3 cm more than its height. Twice of its length, thrice of its breadth and its height add up to 10 cm. Its breadth added to 7 times its height is 1 cm less than 3 times its length.

3.
Differentiate \( \tan^{-1}\left( \frac{\sqrt{1 + x^2} + \sqrt{1 - x^2}}{\sqrt{1 + x^2} - \sqrt{1 - x^2}} \right) \) with respect to \( \cos^{-1}(x^2) \).

4.
Find:
Let \(A=[a_{ij}]\) be a \(2\times2\) matrix whose elements are given by \[ a_{ij}=\frac{(2i-j)^2}{3} \] Find the transpose matrix \(A'\).

6.
At a birthday party, children are being served orange juice in conical cups, as shown in the figure.

Each cup is 15 cm deep and has a radius 5 cm. The juice is being poured into this cup at a rate of 0·1 cm3/s.
On the basis of the above information, answer the following questions :

Comments

SUBSCRIBE TO OUR NEWS LETTER

Multiple Regression: Formula, Analysis & Assumptions

Formula of Multiple Linear Regression

Stepwise Multiple Regression

Advantages of Stepwise Multiple Regression

Difference Between Simple Linear Regression and Multiple Regression

Multiple Regression Analysis

Multiple Regression Assumptions

Multicollinearity

Multivariate Multiple Regression

Things to remember

Sample Questions

CBSE CLASS XII Related Questions

1. Find: The shortest distance between the lines: \[ \vec{r}=(4+\lambda)\hat{i}+(2\lambda-1)\hat{j}-3\lambda\hat{k} \] and \[ \vec{r}=(1+2\mu)\hat{i}+(4\mu-1)\hat{j}+(2-5\mu)\hat{k} \]

2.A carpenter needs to design a wooden box in the shape of a cuboid such that the sum of its length and breadth is 3 cm more than its height. Twice of its length, thrice of its breadth and its height add up to 10 cm. Its breadth added to 7 times its height is 1 cm less than 3 times its length.

3.Differentiate \( \tan^{-1}\left( \frac{\sqrt{1 + x^2} + \sqrt{1 - x^2}}{\sqrt{1 + x^2} - \sqrt{1 - x^2}} \right) \) with respect to \( \cos^{-1}(x^2) \).

4.Find:Let \(A=[a_{ij}]\) be a \(2\times2\) matrix whose elements are given by \[ a_{ij}=\frac{(2i-j)^2}{3} \] Find the transpose matrix \(A'\).

6.At a birthday party, children are being served orange juice in conical cups, as shown in the figure. Each cup is 15 cm deep and has a radius 5 cm. The juice is being poured into this cup at a rate of 0·1 cm3/s.On the basis of the above information, answer the following questions :

Comments

SUBSCRIBE TO OUR NEWS LETTER

1.
Find:

The shortest distance between the lines: \[ \vec{r}=(4+\lambda)\hat{i}+(2\lambda-1)\hat{j}-3\lambda\hat{k} \] and \[ \vec{r}=(1+2\mu)\hat{i}+(4\mu-1)\hat{j}+(2-5\mu)\hat{k} \]

2.
A carpenter needs to design a wooden box in the shape of a cuboid such that the sum of its length and breadth is 3 cm more than its height. Twice of its length, thrice of its breadth and its height add up to 10 cm. Its breadth added to 7 times its height is 1 cm less than 3 times its length.

3.
Differentiate \( \tan^{-1}\left( \frac{\sqrt{1 + x^2} + \sqrt{1 - x^2}}{\sqrt{1 + x^2} - \sqrt{1 - x^2}} \right) \) with respect to \( \cos^{-1}(x^2) \).

4.
Find:
Let \(A=[a_{ij}]\) be a \(2\times2\) matrix whose elements are given by \[ a_{ij}=\frac{(2i-j)^2}{3} \] Find the transpose matrix \(A'\).

6.
At a birthday party, children are being served orange juice in conical cups, as shown in the figure.

Each cup is 15 cm deep and has a radius 5 cm. The juice is being poured into this cup at a rate of 0·1 cm3/s.
On the basis of the above information, answer the following questions :