Home / DOE / Additional Information / Understanding Least Square Residuals
Understanding Least Square Residuals¶
A quantitative regression produces a set of per-run numbers alongside the coefficients. They are what the residual plots draw and what the residual report lists, and each one answers a different question about a single run.
What the regression produces per run¶
| Quantity | What it measures |
|---|---|
| Actual Y | the response you entered |
| Predicted Y | what the model says the response should have been at those settings |
| Residual | Actual Y minus Predicted Y, in the response's own units |
| Studentized Residual | the residual divided by its own estimated standard error, so it is comparable across runs |
| R-Studentized Residual | the same, but with the run left out of the standard error estimate, so a bad point cannot inflate its own denominator |
| Leverage | how unusual the run's factor settings are, independent of its response |
| Cook's D | how much every coefficient moves when the run is removed |
The distinction that matters most in practice is between the two studentized forms. A single badly wrong point raises the residual standard error it is divided by, which can make it look ordinary. The R-studentized version excludes the point from that estimate, so an outlier stands out.
Leverage and Cook's D are about a different question from the residuals. A run can sit exactly on the fitted surface and still have high leverage, because leverage is about where the run is in factor space, not how well it was predicted. Cook's D combines the two: it is large when a run both is unusual in its settings and does not fit.
How each one is computed¶
Leverage is the diagonal of the hat matrix, the matrix that turns the observed responses into the fitted ones. For run \(i\),
where \(\mathbf{x}_i^{\prime}\) is run \(i\)'s row of the design matrix. It measures how unusual run \(i\) is in its factor settings alone, with no reference to the response: the response does not appear in the expression at all. Leverage runs from \(0\) to \(1\), and the leverages of all runs add up to \(p\), the number of terms in the model counting the constant, so the average leverage across a design is \(p/n\).
For a quantitative output this value is produced by the regression library. The definition above is the standard one.
The residual of run \(i\) is \(e_i = y_i - \hat{y}_i\). Residuals do not all have the same variance: a run with high leverage pulls the fitted surface toward itself, so its residual is smaller than a run with low leverage would give on the same data. The studentized residual divides each residual by its own estimated standard deviation:
\(s\) is the model's own standard error, the square root of the residual mean square, computed from all \(n\) runs. Because the divisor carries the \(\sqrt{1 - h_i}\) factor, studentized residuals are comparable across runs in a way that raw residuals are not.
Run \(i\) is one of the runs \(s\) is computed from, so a run that fits badly raises the number it is then divided by.
This value is produced by the regression library. The definition above is the standard one.
The R-studentized residual has the same form as the studentized residual, with one difference: the variance estimate leaves run \(i\) out.
\(s_{(i)}\) is the residual standard error of the model fitted without run \(i\), so a run cannot inflate its own denominator. It does not require refitting the model \(n\) times, because it can be written from the full fit:
where \(p\) is the number of terms counting the constant. The two forms give different numbers for the same run. They always agree in sign, and \(|t_i|\) is larger than \(|r_i|\) exactly when \(r_i^2 > 1\).
This value is produced by the regression library. The definition above is the standard one.
Cook's D measures how far the whole vector of fitted values moves when one run is dropped from the fit. Writing \(\hat{\mathbf{y}}\) for the fitted values from the full model and \(\hat{\mathbf{y}}_{(i)}\) for the fitted values from the model refitted without run \(i\),
The same quantity can be written from the full fit alone, which is why a value is available for every run without refitting anything:
That second form shows what the number combines. The first factor is built from the studentized residual, which is about how well the run was fitted. The second is a function of leverage alone, which is about where the run sits in factor space. Both factors have to be away from zero for \(D_i\) to be large: a run lying exactly on the fitted surface has \(r_i = 0\) and therefore \(D_i = 0\) however high its leverage is, and a run at the centre of the design has small \(h_i / (1 - h_i)\) however badly it fits.
This value is produced by the regression library. The definitions above are the standard ones.
Full details: Regression Diagnostics.
Where they appear¶
Residual Plots pairs one quantity from an X list with one from a Y list. The two lists are not the same list, so many pairings cannot be asked for at all:
| Axis | What it offers, for a quantitative output |
|---|---|
| X | one row per factor, each shown by that factor's own name, plus block when the design is blocked, then Run #, Actual Y and Predicted Y |
| Y | Actual Y, Predicted Y, Leverage, Residual, Studentized Residual, R-Studentized Residual and Cook's D |
So Leverage, both studentized forms and Cook's D are Y only, and the factor names and Run # are X only. Only Actual Y and Predicted Y appear on both.
Run # is the value in the design sheet's Run column, not the row's position in the regression matrix. On a randomized design those are different numbers.
The same command can also write the values out as numbers, on a sheet named Residual Report - output name whose own heading reads Quantum XLResiduals Report, with no space between the two words, but only if you tick Create textual report in the dialog. That box is off unless you tick it, so the report is not produced by default. See Residual Plots.
Observed vs Predicted is the residual view without the arithmetic: it plots Actual Y against Predicted Y with a 45 degree reference line, so distance from the line is the residual. See Observed vs Predicted.
The thresholds the DOE Advisor uses¶
With Display DOE Advisor on regression sheet turned on, the advisor checks the standardized residuals and writes its findings onto the regression sheet in words. See The DOE Advisor for the rest of what it writes. Two of its thresholds are stated in the text it writes:
| Finding | The advisor's own wording |
|---|---|
| Large Standardized Residuals | Approximately 95% of the residuals should fall between +/-2; values outside the range +/-2 are considered large and should be investigated as potential outliers. The large standardized residual points have been marked red in the design sheet. |
| Extreme Standardized Residuals | Approximately 99% of the residuals should fall between +/-3; values outside the range +/-3 are considered large and should be investigated as potential outliers. The extreme standardized residuals have been marked red in the design sheet. |
A third threshold is not stated on the sheet. The advisor also runs a Shapiro-Wilk test on the residuals at a fixed 0.05, and below that writes The Shapiro-Wilk test rejected that the residuals are normal at the .05 level. Care should be exercised when using this model.
Those runs really are marked red in the design sheet, so red cells in a design sheet after a regression are the advisor's work and not something you did. No chart command marks cells. Two conditions put a run on that list, and the second is not about residuals at all:
- a standardized residual outside plus or minus 2, or outside plus or minus 3
- leverage at or above 3 times the number of active terms divided by the number of rows, which marks a run whose settings are unusual even if it fits the model well
It is the run's output value cell that is filled red, not the whole row.
Pure error, and where the residuals go next¶
The residual sum of squares splits into pure error and lack of fit whenever the design has repeated design points, and that split is the formal version of the question the residual plots ask by eye.
What the residuals contain beyond pure error is lack of fit:
The test compares the lack of fit mean square against the pure error mean square. The lack of fit sum of squares is reported as \(0\) if the arithmetic makes it negative, and every cell of the test reads NA in any of three cases: there is no pure error, the pure error mean square is at or below \(10^{-11}\), or \(df_{\text{lof}}\) comes out at zero or below. The last of those happens when the model uses up every degree of freedom that is not a replicate, so a saturated model shows no test even though it has pure error.
Full details: Sums of Squares.