Home / DOE / Additional Information / Type I and Type II Error
Type I and Type II Error¶
Two numbers in the DOE reports are governed by these two ideas: the alpha you give the power calculation, and the power it gives back. This page defines the terms and lists every place in Quantum XL where one of them is set or shown.
The two errors¶
A Type I error is concluding a factor has an effect when it does not. Its probability is alpha.
A Type II error is concluding a factor has no effect when it does. Its probability is usually written beta, and power is 1 - beta, the probability of detecting an effect that is really there.
Where alpha appears¶
Quantum XL never asks you for an alpha. Every alpha in the product is fixed in the software. There is no alpha control on any DOE dialog and none on the Options dialog.
| Where | The value, and whether you can change it |
|---|---|
| The power calculation | 0.05, fixed. The dialog has no alpha control |
| Coefficient confidence intervals on the regression sheet | 95 percent, so alpha 0.05, fixed |
| The prediction interval on the regression sheet | 99 percent, computed as Y-hat plus or minus three times S-hat, fixed |
| Pareto bar colours | a bar is red at p of 0.05 or below and blue at 0.10 or below, both fixed |
| The DOE Advisor | applies 0.05 to a blocking term's p value, 0.05 to interaction and quadratic terms, 0.05 to the model's Significance of F, 0.05 to a normality test on the residuals, and 0.10 to deciding a term could be removed. All fixed |
| The p-value columns on the regression sheet | none. The p value is printed and no threshold is applied to it |
So the p-value columns are the one place where the comparison is yours to make. Everywhere else the threshold is already chosen, and the Recommended row in the power grid works the same way: it is the first replicate count reaching a power of 0.90, and 0.90 is fixed too.
Where power appears¶
Power is computed in one place, the power and sample size calculation, reached from the replicates page of a design builder or of the wizard. The button is labelled Help Me Choose Power/Sample size for Quantitative Outputs in the design builders and Help Me Choose... in the wizard. Both open the same calculation.
It answers one question: for this design, this many replicates, an effect of this size and this standard deviation, what is the probability of detecting the effect. Increasing the replicates increases the power, which is why the calculation sits on the replicates page.
Power is computed from the noncentral \(F\) distribution. The critical value is the ordinary \(F\) value that cuts off \(\alpha\) in the upper tail, and the power is the probability that a noncentral \(F\) with the same degrees of freedom exceeds it:
Everything specific to the design is in the numerator degrees of freedom \(df_1\) and the noncentrality parameter \(\lambda\).
Full details, including both formulas and their limits: Power and Sample Size Calculation.
Where a verdict is written, and where only a number is¶
Lack of fit, the G statistic, and the goodness-of-fit chi-squares all report a p value and no verdict. So does every term in the ANOVA table.
Two places do write a verdict, and both use their own fixed thresholds:
- The Pareto chart colours a bar red at p of 0.05 or below and blue at 0.10 or below.
- The DOE Advisor, when it is turned on, writes findings onto the regression sheet in words, and those findings do declare terms significant. It says a blocking variable is significant below 0.05, names significant interaction and quadratic terms below 0.05, and suggests a term could be removed above 0.10.
So the statement to hold on to is narrower than it looks: the ANOVA table and the coefficient table report numbers only, and the Pareto and the Advisor are where Quantum XL commits to a verdict.