Chapter 12 - Simple Linear Regressions

 
 

1  

Chapter 12 
Simple Linear Regression 

• Simple Linear Regression Model
• Least Squares Method 
• Coefficient of Determination
• Model Assumptions
• Testing for Significance
• Using the Estimated Regression Equation

       for Estimation and Prediction

• Computer Solution
• Residual Analysis: Validating Model Assumptions

 
 

2  

• The equation that describes how y is related to x and an error term is called the regression model.
• The simple linear regression model is:

 

y = b₀ + b₁x +e 
 

• b₀ and b₁ are called parameters of the model.
• e  is a random variable called the error term.

 

Simple Linear Regression Model

 
 

3  

• The simple linear regression equation is:

 

E(y) = ₀ + ₁x 
 

• Graph of the regression equation is a straight line.
• b₀ is the y intercept of the regression line.
• b₁ is the slope of the regression line.
• E(y) is the expected value of y for a given x value.

 

Simple Linear Regression Equation

 
 

4  

Simple Linear Regression Equation 

• Positive Linear Relationship

 

E(y) 

x 

Slope b₁

is positive 

Regression line 

Intercept

             b₀

 
 

5  

Simple Linear Regression Equation 

• Negative Linear Relationship

 

E(y) 

x 

Slope b₁

is negative 

Regression line 

Intercept

             b₀

 
 

6  

Simple Linear Regression Equation 

• No Relationship

 

E(y) 

x 

Slope b₁

is 0 

Regression line 

Intercept

             b₀

 
 

7  

• The estimated simple linear regression equation is:

• The graph is called the estimated regression line. 
   
   
  
• b₀ is the y intercept of the line.
• b₁ is the slope of the line.
•     is the estimated value of y for a given x value.

 
 

Estimated Simple Linear Regression Equation

 
 

8  

Estimation Process 
 

Regression Model

y = b₀ + b₁x +e

Regression Equation

E(y) = b₀ + b₁x

Unknown Parameters

b₀, b₁ 

Sample Data:

x        y 

x₁      y₁

.       .

.       .

x_n     y_n 

Estimated

Regression Equation

 

Sample Statistics

b₀, b₁ 

b₀ and b₁

provide estimates of

b₀ and b₁

 
 

9  

• Least Squares Criterion

 
 
 

    where:

          y_i = observed value of the dependent variable

                 for the ith observation

          y_i = estimated value of the dependent variable

                 for the ith observation 
 

Least Squares Method 

^

 
 

10  

• Slope for the Estimated Regression Equation

 
 

The Least Squares Method

 
 

11  

• y-Intercept for the Estimated Regression Equation

 
 
 

    where:

    x_i = value of independent variable for ith observation

    y_i = value of dependent variable for ith observation

   x = mean value for independent variable

      y = mean value for dependent variable

      n = total number of observations 

_ 

_ 

The Least Squares Method

 
 

12  

Example:  Reed Auto Sales 

• Simple Linear Regression

                 Reed Auto periodically has a special week-long sale.  As part of the advertising campaign Reed runs one or more television commercials during the weekend preceding the sale.  Data from a sample of 5 previous sales are shown on the next slide.

 
 

13  

Example:  Reed Auto Sales 

• Simple Linear Regression

 

          Number of TV Ads     Number of Cars Sold

          1 14

          3 24

          2 18

          1 17

          3 27

 
 

14  

• Slope for the Estimated Regression Equation

 

                 b₁  =  220 - (10)(100)/5  =  _____

                         24 - (10)²/5 

• y-Intercept for the Estimated Regression Equation

 

                       b₀  =  20 - 5(2)  =  _____ 

• Estimated Regression Equation

 

y = 10 + 5x 

^ 

Example:  Reed Auto Sales

 
 

15  

Example:  Reed Auto Sales 

• Scatter Diagram

 

^

 
 

16  

• Relationship Among SST, SSR, SSE

 

SST = SSR + SSE 
 
 
 
 
 
 
 
 

    where:

               SST = total sum of squares

               SSR = sum of squares due to regression

               SSE = sum of squares due to error 

The Coefficient of Determination 

^ 

^

 
 

17  

• The coefficient of determination is:

 

r² = SSR/SST 

    where:

               SST = total sum of squares

               SSR = sum of squares due to regression 

The Coefficient of Determination

 
 

18  

• Coefficient of Determination

 

                r² = SSR/SST = 100/114 =

          The regression relationship is very strong because 88% of the variation in number of cars sold can be explained by the linear relationship between the number of TV ads and the number of cars sold. 

Example:  Reed Auto Sales

 
 

19  

The Correlation Coefficient 

• Sample Correlation Coefficient

 
 
 
 

    where:

              b₁ = the slope of the estimated regression

                equation

 
 

20  

• Sample Correlation Coefficient

 
 

    The sign of b₁ in the equation        is ��+��. 
 

                       r_xy =    +.9366  
 
 

Example:  Reed Auto Sales

 
 

21  

Model Assumptions 

• Assumptions About the Error Term 
9. The error   is a random variable with mean of zero.
2. The variance of  , denoted by  ², is the same for all values of the independent variable.
3. The values of  are independent.
4. The error   is a normally distributed random variable.

 
 

22  

Testing for Significance 

• To test for a significant regression relationship, we must conduct a hypothesis test to determine whether the value of b₁ is zero.
• Two tests are commonly used
• t  Test
• F  Test
• Both tests require an estimate of s ², the variance of e in the regression model.

 
 

23  

• An Estimate of s ²

The mean square error (MSE) provides the estimate

of s ², and the notation s² is also used.  

                     s² = MSE = SSE/(n-2) 

where: 

Testing for Significance

 
 

24  

Testing for Significance 

• An Estimate of s
• To estimate s  we take the square root of s ².
• The resulting s is called the standard error of the estimate.

 
 

25  

• Hypotheses

 

                          H₀: ₁ = 0

                          H_a: ₁ = 0 

• Test Statistic

 
 
 

    where 
 

Testing for Significance:  t  Test

 
 

26  

• Rejection Rule

 

Reject H₀ if t < -t_or t > t_ 
 

    where:   t_ is based on a t  distribution

            with n - 2 degrees of freedom 

Testing for Significance:  t  Test

 
 

27  

• t  Test
• Hypotheses

                            H₀: ₁ = 0

                          H_a: ₁ = 0 

• Rejection Rule

                For  = .05 and d.f. = 3,  t_.025 = _____

                        Reject H₀ if t > t_.025 = _____ 

Example:  Reed Auto Sales

 
 

28  

• t  Test
• Test Statistics

                      t = _____/_____ = 4.63

• Conclusions

          t = 4.63 > 3.182, so reject H₀  

Example:  Reed Auto Sales

 
 

29  

Confidence Interval for



₁ 

• We can use a 95% confidence interval for ₁ to test the hypotheses just used in the t test.
• H₀ is rejected if the hypothesized value of  ₁ is not included in the confidence interval for  ₁.

 

 
 

30  

• The form of a confidence interval for ₁ is:

 
 

    

where      b₁    is the point estimate

                      

is the margin of error

                      

is the t value providing an area

                      

of

a

/2 in the upper tail of a

       t  distribution with n - 2 degrees

                      

of freedom 
 

Confidence Interval for



₁

 
 

31  

• Rejection Rule

          

Reject H₀ if 0 is not included in

          

the confidence interval for 



₁.

• 95% Confidence Interval for ₁

                

        = 5 +/- 3.182(1.08) = 5 +/- 3.44 

                      

or    ____ to ____ 

• Conclusion

    

0 is not included in the confidence interval. 

    Reject H₀   

Example:  Reed Auto Sales

 
 

32  

• Hypotheses

 

          

               H₀:



₁ = 0

          

              H_a:



₁ = 0 

• Test Statistic

 

F = MSR/MSE 

Testing for Significance: F  Test

 
 

33  

• Rejection Rule

 

Reject H₀ if F > F

_ 

    

where: F

_

is based on an F distribution

                

with 1 d.f. in the numerator and

                

n - 2 d.f. in the denominator 

Testing for Significance: F  Test

 
 

34  

• F Test
• Hypotheses

                            H₀: ₁ = 0

          

                H_a:



₁ = 0

• Rejection Rule

                For  = .05 and d.f. = 1, 3:  F_.05 = ______

                       Reject H₀ if F > F_.05 = ______. 

Example:  Reed Auto Sales

 
 

35  

• F Test
• Test Statistic

F = MSR/MSE = ____  / ______ = 21.43

• Conclusion

                F = 21.43 > 10.13, so we reject H₀. 

Example:  Reed Auto Sales

 
 

36  

Some Cautions about the 
Interpretation of Significance Tests 

• Rejecting H₀: b₁ = 0 and concluding that the relationship between x and y is significant does not enable us to conclude that a cause-and-effect relationship is present between x and y.
• Just because we are able to reject H₀: b₁ = 0 and demonstrate statistical significance does not enable us to conclude that there is a linear relationship between x and y.

 
 

37  

• Confidence Interval Estimate of E(y_p)

• Prediction Interval Estimate of y_p 
   
  

 

y_p + t

_

_/2 s_ind 
 

    

where: confidence coefficient is 1 -



and

                

t

_

_/2 is based on a t distribution

                

with n - 2 degrees of freedom 
 

Using the Estimated Regression Equation 
for Estimation and Prediction

 
 

38  

• Point Estimation

    

If 3 TV ads are run prior to a sale, we expect the mean number of cars sold to be: 

y = 10 + 5(3) = ______ cars 
 

^ 

Example:  Reed Auto Sales

 
 

39  

• Confidence Interval for E(y_p)

    

95% confidence interval estimate of the mean number of cars sold when 3 TV ads are run is: 

25 + 4.61  =  ______ to _______ cars 

Example:  Reed Auto Sales

 
 

40  

• Prediction Interval for y_p

    

95% prediction interval estimate of the number of cars sold in one particular week when 3 TV ads are run is: 

         25 + 8.28  =  _____ to ______ cars 

Example:  Reed Auto Sales

 
 

41  

• Residual for Observation  i

 

y_i

–

 y_i 

• Standardized Residual for Observation  i

 
 

    

where: 

    

and   

Residual Analysis 

^ 

^ 

^ 

^

 
 

42  

Example:  Reed Auto Sales 

• Residuals

 
 

43  

Example:  Reed Auto Sales 

• Residual Plot

 
 

44  

Residual Analysis 

• Residual Plot

 

x 

0 

Good Pattern 

Residual

 
 

45  

Residual Analysis 

• Residual Plot

 

x 

0 

Nonconstant Variance 

Residual

 
 

46  

Residual Analysis 

• Residual Plot

 

x 

0 

Model Form Not Adequate 

Residual

 
 

47  

End of Chapter 12