Chapter 6.6, Problem 25E

Exercises 19 and 20 involve a design matrix X with two or more columns and a least-squares solution $\hat{\beta }$ of y = Xβ. Consider the following numbers.

(i) ${\Vert X\hat{\beta }\Vert }^2$—the sum of the squares of the “regression term.” Denote this number by SS(R).

(ii) ${\Vert y-X\hat{\beta }\Vert }^2$—the sum of the squares for error term. Denote this number by SS(E).

(iii) ${\Vert y\Vert }^2$—the “total” sum of the squares of the y-values. Denote this number by SS(T).

Every statistics text that discusses regression and the linear model y = Xβ + ε introduces these numbers, though terminology and notation vary somewhat. To simplify matters, assume that the mean of the y-values is zero. In this case, SS(T) is proportional to what is called the variance of the set of y-values.

19. Justify the equation SS(T) = SS(R) + SS(E). [Hint: Use a theorem, and explain why the hypotheses of the theorem are satisfied.] This equation is extremely important in statistics, both in regression theory and in the analysis of variance.