Question

How do you determine the relationship between 2 variables? And how do you use simple linear regression to describe and test whether this relationship is significant?

Answer 1

Determine the relationship between 2 variables

The Pearson correlation coefficient (named for Karl Pearson) can be used to summarize the strength of the linear relationship between two variables.

A correlation could be positive, meaning both variables move in the same direction, or negative, meaning that when one variable’s value increases, the other variables’ values decrease. Correlation can also be neural or zero, meaning that the variables are unrelated.

• Positive Correlation: both variables change in the same direction.
• Neutral Correlation: No relationship in the change of the variables.
• Negative Correlation: variables change in opposite directions.

Simple linear regression to describe and test whether this relationship is significant

We estimate the coeffcients a and b in the below regression line :

Y = a + bX

where, Y = Dependent variable

X = Independent variable

a = Intercept

b = Slope

We find the relationship between Y and X which yields values of Y with the least error.

We have N paired data point (x_i, y_i ) that we want to approximate their relationship with a linear regression

We estimate the value of a and b using the formulas :

Once  a and b are known, the fitted regression line can be written as :

$\hat{Y} = a + bX$

Hypothesis Tests in Simple Linear Regression

The t tests are used to conduct hypothesis tests on the regression coefficients obtained in simple linear regression.

Null hypothesis : b₀ = 0 ( Slope is not significantly different from 0 )

Alternative hypothesis : b₀ not equal to 0 ( Slope is significantly different from 0 )

Test statistic is defined as :

$t = \frac{b - b_0}{s.e(b)}$

$s.e(b) = \sqrt{\frac{\sum_{i=1}^{n} e_i^2/(n-2)}{\sum_{i=1}^{n}(x_i-\bar{x})^2}}$

$e_i = (y-\hat{y})$

y = Observed value

$\hat{y}$ = Predicted value

Now, P-value is calculated and if P-value < Level of significance , then , we reject null hypothesis and conclude that Slope is significantly different from 0 and the regression model is significant otherwise if P-value > Level of significance , then , we do not reject null hypothesis and conclude that Slope is not significantly different from 0 and the regression model is not significant.