Question

Using diagram(s) explain why a simple linear regression with a constant term will generally provide a better fit than a simple linear regression which excludes a constant.

Answer 1

It is simply because - if there is no constant, then at the point of origin (0,0) X and Y become equal which is unlikely and which is not always correct. And the slope would not be correct if the regression line unnaturally passes through the origin. The constant, also called Y-intercept means that it is the value of response variable, Y when the explanatory variable, X is 0. But sometimes, we also get negative constant which is not possible to interpret.. If the Y-variable is height or weight or any other variable for which negative values are not possible, it is not possible to interpret saying that when X=0, the Y-variable: Height is - 3 feet because - 3 feet is not possible as the height cannot be negative. This does not mean that the model is incorrect. The model with a constant (constant may be naturally 0, sometimes) is better than that without a constant because we are interested in knowing how the changes in explanatory variable(X) influences the changes in response variable(Y), i.e., slope is important that says by how much (magnitude) and in what direction (+ or -) will be the change.

For Example, let us take a data with X and Y values as follows:

The regression equation is: =3.16393X - 7.40984

The slope is +3.16393 which indicates that as there is an increase of 1 unit in X, then Y increases by 3.16393 units.

Now, let us see this model without constant. So, the regression equation is: =3.16393X

For X= 3, =9.49179

For X= 5, =15.81965

For X= 6, =18.98358

For X =7, =22.14751

For X =8, =25.31144

For X =9, =28.47537

For X =10, =31.6393

We can observe that the regression lines in both the above models are different. The second regression line (without constant) is steeper than the first one (with constant). And in the second regression line, all points are lying on the straight line which in unnatural indicating it's not a good fit.

So, a simple linear regression with a constant term will generally provide a better fit than a simple linear regression which excludes a constant.