Home > Chapter 12 - Simple Linear Regressions

# Chapter 12 - Simple Linear Regressions

1

Chapter 12
Simple Linear Regression

• Simple Linear Regression Model
• Least Squares Method
• Coefficient of Determination
• Model Assumptions
• Testing for Significance
• Using the Estimated Regression Equation

for Estimation and Prediction

• Computer Solution
• Residual Analysis: Validating Model Assumptions

2

• The equation that describes how y is related to x and an error term is called the regression model.
• The simple linear regression model is:

y = b0 + b1x +e

• b0 and b1 are called parameters of the model.
• e  is a random variable called the error term.

Simple Linear Regression Model

3

• The simple linear regression equation is:

E(y) = 0 + 1x

• Graph of the regression equation is a straight line.
• b0 is the y intercept of the regression line.
• b1 is the slope of the regression line.
• E(y) is the expected value of y for a given x value.

Simple Linear Regression Equation

4

Simple Linear Regression Equation

• Positive Linear Relationship

E(y

x

Slope b1

is positive

Regression line

Intercept

b0

5

Simple Linear Regression Equation

• Negative Linear Relationship

E(y

x

Slope b1

is negative

Regression line

Intercept

b0

6

Simple Linear Regression Equation

• No Relationship

E(y

x

Slope b1

is 0

Regression line

Intercept

b0

7

• The estimated simple linear regression equation is:
• The graph is called the estimated regression line.

• b0 is the y intercept of the line.
• b1 is the slope of the line.
•     is the estimated value of y for a given x value.

Estimated Simple Linear Regression Equation

8

Estimation Process

Regression Model

y = b0 + b1x +e

Regression Equation

E(y) = b0 + b1x

Unknown Parameters

b0, b1

Sample Data:

x        y

x1      y1

.       .

.       .

xn     yn

Estimated

Regression Equation

Sample Statistics

b0, b1

b0 and b1

provide estimates of

b0 and b1

9

• Least Squares Criterion

where:

yi = observed value of the dependent variable

for the ith observation

yi = estimated value of the dependent variable

for the ith observation

Least Squares Method

^

10

• Slope for the Estimated Regression Equation

The Least Squares Method

11

• y-Intercept for the Estimated Regression Equation

where:

xi = value of independent variable for ith observation

yi = value of dependent variable for ith observation

x = mean value for independent variable

y = mean value for dependent variable

n = total number of observations

The Least Squares Method

12

Example:  Reed Auto Sales

• Simple Linear Regression

Reed Auto periodically has a special week-long sale.  As part of the advertising campaign Reed runs one or more television commercials during the weekend preceding the sale.  Data from a sample of 5 previous sales are shown on the next slide.

13

Example:  Reed Auto Sales

• Simple Linear Regression

Number of TV Ads     Number of Cars Sold

1 14

3 24

2 18

1 17

3 27

14

• Slope for the Estimated Regression Equation

b1  =  220 - (10)(100)/5  =  _____

24 - (10)2/5

• y-Intercept for the Estimated Regression Equation

b0  =  20 - 5(2)  =  _____

• Estimated Regression Equation

y = 10 + 5x

Example:  Reed Auto Sales

15

Example:  Reed Auto Sales

• Scatter Diagram

^

16

• Relationship Among SST, SSR, SSE

SST = SSR + SSE

where:

SST = total sum of squares

SSR = sum of squares due to regression

SSE = sum of squares due to error

The Coefficient of Determination

^

17

• The coefficient of determination is:

r2 = SSR/SST

where:

SST = total sum of squares

SSR = sum of squares due to regression

The Coefficient of Determination

18

• Coefficient of Determination

r2 = SSR/SST = 100/114 =

The regression relationship is very strong because 88% of the variation in number of cars sold can be explained by the linear relationship between the number of TV ads and the number of cars sold.

Example:  Reed Auto Sales

19

The Correlation Coefficient

• Sample Correlation Coefficient

where:

b1 = the slope of the estimated regression

equation

20

• Sample Correlation Coefficient

The sign of b1 in the equation        is ��+��.

rxy =    +.9366

Example:  Reed Auto Sales

21

Model Assumptions

• Assumptions About the Error Term
1. The error   is a random variable with mean of zero.
2. The variance of , denoted by 2, is the same for all values of the independent variable.
3. The values of are independent.
4. The error   is a normally distributed random variable.

22

Testing for Significance

• To test for a significant regression relationship, we must conduct a hypothesis test to determine whether the value of b1 is zero.
• Two tests are commonly used
• t  Test
• F  Test
• Both tests require an estimate of s 2, the variance of e in the regression model.

23

• An Estimate of s 2

The mean square error (MSE) provides the estimate

of s 2, and the notation s2 is also used.

s2 = MSE = SSE/(n-2)

where:

Testing for Significance

24

Testing for Significance

• An Estimate of s
• To estimate s  we take the square root of s 2.
• The resulting s is called the standard error of the estimate.

25

• Hypotheses

H0: 1 = 0

Ha: 1 = 0

• Test Statistic

where

Testing for Significance:  t  Test

26

• Rejection Rule

Reject H0 if t < -tor t > t

where:   t is based on a t  distribution

with n - 2 degrees of freedom

Testing for Significance:  t  Test

27

• t  Test
• Hypotheses

H0: 1 = 0

Ha: 1 = 0

• Rejection Rule

For = .05 and d.f. = 3,  t.025 = _____

Reject H0 if t > t.025 = _____

Example:  Reed Auto Sales

28

• t  Test
• Test Statistics

t = _____/_____ = 4.63

• Conclusions

t = 4.63 > 3.182, so reject H0

Example:  Reed Auto Sales

29

Confidence Interval for

1

• We can use a 95% confidence interval for 1 to test the hypotheses just used in the t test.
• H0 is rejected if the hypothesized value of  1 is not included in the confidence interval for  1.

30

• The form of a confidence interval for 1 is:

where      b1    is the point estimate

is the margin of error

is the t value providing an area

of

a

/2 in the upper tail of a

t  distribution with n - 2 degrees

of freedom

Confidence Interval for

1

31

• Rejection Rule

Reject H0 if 0 is not included in

the confidence interval for

1.

• 95% Confidence Interval for 1

= 5 +/- 3.182(1.08) = 5 +/- 3.44

or    ____ to ____

• Conclusion

0 is not included in the confidence interval.

Reject H0

Example:  Reed Auto Sales

32

• Hypotheses

H0:

1 = 0

Ha:

1 = 0

• Test Statistic

F = MSR/MSE

Testing for Significance: F  Test

33

• Rejection Rule

Reject H0 if F > F

where: F

is based on an F distribution

with 1 d.f. in the numerator and

n - 2 d.f. in the denominator

Testing for Significance: F  Test

34

• F Test
• Hypotheses

H0: 1 = 0

Ha:

1 = 0

• Rejection Rule

For = .05 and d.f. = 1, 3:  F.05 = ______

Reject H0 if F > F.05 = ______.

Example:  Reed Auto Sales

35

• F Test
• Test Statistic

F = MSR/MSE = ____  / ______ = 21.43

• Conclusion

F = 21.43 > 10.13, so we reject H0

Example:  Reed Auto Sales

36

Interpretation of Significance Tests

• Rejecting H0: b1 = 0 and concluding that the relationship between x and y is significant does not enable us to conclude that a cause-and-effect relationship is present between x and y.
• Just because we are able to reject H0: b1 = 0 and demonstrate statistical significance does not enable us to conclude that there is a linear relationship between x and y.

37

• Confidence Interval Estimate of E(yp)
• Prediction Interval Estimate of yp

yp + t

/2 sind

where: confidence coefficient is 1 -

and

t

/2 is based on a t distribution

with n - 2 degrees of freedom

Using the Estimated Regression Equation
for Estimation and Prediction

38

• Point Estimation

If 3 TV ads are run prior to a sale, we expect the mean number of cars sold to be:

y = 10 + 5(3) = ______ cars

Example:  Reed Auto Sales

39

• Confidence Interval for E(yp)

95% confidence interval estimate of the mean number of cars sold when 3 TV ads are run is:

25 + 4.61  =  ______ to _______ cars

Example:  Reed Auto Sales

40

• Prediction Interval for yp

95% prediction interval estimate of the number of cars sold in one particular week when 3 TV ads are run is:

25 + 8.28  =  _____ to ______ cars

Example:  Reed Auto Sales

41

• Residual for Observation  i

yi

yi

• Standardized Residual for Observation  i

where:

and

Residual Analysis

^

42

Example:  Reed Auto Sales

• Residuals

43

Example:  Reed Auto Sales

• Residual Plot

44

Residual Analysis

• Residual Plot

x

Good Pattern

Residual

45

Residual Analysis

• Residual Plot

x

Nonconstant Variance

Residual

46

Residual Analysis

• Residual Plot

x

Residual

47

End of Chapter 12

Search more related documents:Chapter 12 - Simple Linear Regressions 