Hi,
Regress
y on
x
Regress x on y
1. Use GC, do a scatter
plot 1. Use GC,
do a scatter plot
2. Note that gradient =
a
2. Note that gradient = c
Vertical intercept =
b Vertical
intercept = d
3. Residual Error
ei
3. Residual Error ei
= Observed Value of
each yi =
Observed value of each xi
- each predicted y
value -
each predicted x
value
= yi - (a +
bxi) =
xi - (c + dyi)
4. Using the least
squares
4. Using the least squares
method to minimise
the
method to mininise the
sum of the squared
residual sum of the
squared residual
errors, differentiation
and
errors, differentiation and
substitution, the formula
for susbstitution,
the formula for
a and b are
obtained.
c and d are obtained.
5. When the sum of
squared 5. When the sum
of squared
residual errors are not
zero, residual
errors are not zero,
a =/= - cb and b =/=
1/d c
=/= -(a/b) and d =/= 1/b
ie r =/= 1 or r =/= -
1
ie r =/= 1 or r =/=
-1
6. When the sum of
squared 6. When the sum
of squared
residual errors
are zero,
residual errors are zero,
a = - cb and b =
1/d c
= - (a/b) and d = 1/b
ie r = 1 or r = -
1
ie r = 1 or r =
-1
This means that all
the
This means that all the
observed y values
are
observed x values are
equal to the
predicted equal
to the predicted
y
values
x
values
So to TS,
(a) why a is not equal to c
or - (a/b) and b is not equal to d or 1/b because the
the
sum of squared residual errors for regress y on x and regress
x on y are
different and not equal to zero. So, the least
squares regression lines for both
cases are different ( ie r are not equal to 1 or -1 ).
(b) a is equal to c or -
(a/b) and b is equal to d or 1/b when the sum of squared
residual errors for regress y on x and regress x on y are both
equal to zero.
So,
the least squares regression lines for both cases are the same (ie
r are
equal to 1 or - 1). So, only in this case, we
can from y = a + bx and re-
arrange and express x in terms of y into x = - (a/b) + (1/b)y = c +
dy.
PS : For Step 4, TS
you may refer to your textbook Appendix 6.
This will require you to
use partial differentiation with respect to a, then b, c
and d. Since we want to
minimise the sum of squared residual errors, we
can set the 4 equations
equal to zero (known as normal equations). Solving
these equations, we get the
formula for a, b, c and d. To prove that it is a
minimum, we can find the
second derivatives for a, b, c and d.