Home / DOE / Analyze / Evaluate Design / Math Details
Math Details¶
Every figure on the Evaluate Design worksheet, and the expression it comes from. What the tool is for is on Evaluate Design, and the controls are on Options.
The three criteria themselves, D, A and I, are defined once on Create Optimal Design Math Details. This page gives the scalings and the per-run and per-column quantities the worksheet reports.
Notation¶
| Term | Description |
|---|---|
| \(N\) | the number of runs in the design |
| \(p\) | the number of columns in the model matrix, the constant counted |
| \(X\) | the model matrix, \(N\) rows by \(p\) columns |
| \(X^{\prime}X\) | the information matrix, \(p\) by \(p\) |
| \(\mathbf{x}_i^{\prime}\) | run \(i\)'s row of \(X\) |
| \(\mathcal{R}\) | the design region, the coded box the factors span |
| \(r\) | a replicate count, 1 through 5 |
| \(\alpha\) | the significance level |
Every quantitative factor is put on a common scale before the regression runs. One expression does it, whichever coding is in force, using a centre \(m_c\) and a scale \(s_c\) that the coding chooses:
A categorical factor is never coded this way. Its coded value is \(1\), and the level it sits at selects which coefficient is used instead.
An interaction's coded value is the product of the coded values of the factors in it.
No response data enters any figure¶
Every quantity below is computed from \(X\) alone. The leverage and the variance inflation factors are obtained by running the ordinary least squares fit, and a vector of zeros is passed as the response purely to satisfy that routine: neither quantity depends on the response.
A singular design¶
\(X^{\prime}X\) is singular when the chosen runs cannot support the chosen model, either because \(N < p\) or because two model columns coincide over the runs. The determinant-based measures, the alias matrix and the power figures are then unavailable, and the sheet's Engine status line reads:
Design information matrix is singular; determinant-based measures are unavailable.
Leverage and the variance inflation factors are still attempted, because the fit they come from is rank revealing and handles an aliased column.
The nine design measures¶
They appear on the sheet in this order.
Log determinant of \(X'X\)¶
The quantity the two determinant-based efficiencies are built from, reported directly so a design whose determinant is far from 1 can still be compared.
D-efficiency (%)¶
The \(p\)-th root of the determinant, scaled by the run count. The reference is an orthogonal design of the same run size, whose information matrix is \(N I\); such a design scores exactly 100.
G-efficiency (%)¶
The same \(p/N\) reference against the largest scaled prediction variance anywhere in the design region, so it is the measure of the worst case rather than the average. Reported as unavailable when the maximum is zero, which would divide by zero.
A-efficiency (%)¶
An orthogonal design of the same run size has \(X^{\prime}X = N I\), so its trace is \(p/N\) and it too scores exactly 100.
A value¶
The A criterion itself, the sum of the coefficient variances up to the error variance, reported alongside the efficiency it is scaled into.
I (average prediction variance)¶
where \(\mathbf{f}(\mathbf{x})\) is the model row at the point \(\mathbf{x}\). \(M\) is the region moment matrix, the model row's outer product averaged over the region, and it is built from the model term structure, which describes each model column as a product of factor settings. The trace form is the integral of the prediction variance over the region, so no numerical integration is performed at report time.
Maximum prediction variance over the design region¶
The quantity G-efficiency divides by, reported directly.
Max VIF and Average VIF¶
The largest and the mean of the per-column variance inflation factors below, taken over the columns for which a figure was reported. The constant is not among them.
Why three of the nine are missing on a blocked design¶
\(M\) and the maximum both need the model term structure, which says how a point of the region becomes a model row. A block is not a factor setting, so that structure cannot be built for a blocked design and there is nothing to average or search over. G-efficiency, I and the maximum prediction variance are then left off the sheet. See Evaluate Design.
When a measure overflows¶
An uncoded model matrix on a large scale can overflow the exponential in D-efficiency, or drive the A value to a figure that cannot be scaled. The measure is then reported as unavailable rather than as a number that is not one.
Variance Inflation Factor (VIF) by model column¶
For model column \(j\), with \(R_j^2\) the coefficient of determination from regressing that column on the other model columns:
The figures are produced by the regression library's generalized variance inflation routine. Because each model column is treated as a term of its own, one column each, the generalized factor equals the classic factor above.
The constant has no variance inflation factor. A variance inflation factor says how much one predictor's variance is inflated by its overlap with the other predictors, and the constant is not a predictor. The engine reports no figure for it on every design there is, so it is left out of the table rather than given a row that could only ever read Not Avail.
An aliased column has none either. A column that is a linear combination of the others has no variance of its own to inflate, and reads Not Avail.
Power by model column¶
Power is computed from the noncentral \(F\) distribution. The critical value is the ordinary \(F\) value that cuts off \(\alpha\) in the upper tail, and the power is the probability that a noncentral \(F\) with the same degrees of freedom exceeds it:
Everything specific to the design is in the numerator degrees of freedom \(df_1\) and the noncentrality parameter \(\lambda\).
For this report the test is on one model coefficient at a time, so \(df_1 = 1\) throughout, and the figures are given for \(r = 1\) through \(5\) replicates of the whole design.
Replicating the design \(r\) times stacks \(X\) on itself \(r\) times, so \(X^{\prime}X\) becomes \(r\,(X^{\prime}X)\). Two things follow: the residual degrees of freedom become
and the noncentrality for column \(j\) is multiplied by \(r\):
where \(\beta\) is the Anticipated coefficient and \(\sigma\) the Standard deviation typed on the dialog. The two are folded into a single ratio, so what moves the power is the size of the coefficient relative to the response's standard deviation and not either number alone. The sign of \(\beta\) never enters, only its size.
The critical value is the ordinary \(F\) value at the typed significance level, and the power is the upper tail of the noncentral \(F\) beyond it:
A saturated design reports no power at that replicate count. When \(rN - p \le 0\) there are no error degrees of freedom, so there is no \(F\) test to run, and the cell reads Not Avail. Doubling the design can restore the test at a higher replicate count while the cell for one replicate stays empty.
An overflowing noncentrality reports no power. On an extreme uncoded scale \(\lambda_j\) can overflow, and the figure is then reported as unavailable.
Alias matrix¶
Which terms count as alias terms. The set is every two-way interaction plus every quadratic the engine considers eligible, which means a continuous factor taking more than two distinct values, with the terms already in the model removed. Nothing of order three or higher is ever an alias term, so a three-factor interaction does not appear in this matrix. A design whose model already holds every two-way interaction and offers no eligible quadratic therefore has no alias terms at all, and the block carries a sentence saying so rather than an empty table.
Writing \(X_1\) for the model matrix and \(X_2\) for the matrix of the alias terms evaluated over the same runs:
Entry \((j, k)\) is the amount by which alias term \(k\) biases the estimate of model column \(j\). A column of the model that is clear of every alias term has a row of zeros.
This form is the one that can carry partial aliasing, an entry that is neither 0 nor \(\pm 1\), which is the normal case in a custom or optimal design and which a defining relation cannot write down.
It is not the same thing as the alias table on the design sheet. That table lists terms sharing
one column, printed as A=BD=CE, and it is built by comparing columns entry by entry, so it can only
record aliasing that is complete. Every design type has an alias matrix; only the factorial designs
and the five Taguchi arrays have a defining relation to print a table from. See
Aliasing and Resolution.
Correlations among the model columns¶
The signed Pearson correlation between each pair of model columns, the constant excluded. Each column is centred on its own mean and scaled to unit length, and the matrix is the product of the scaled columns:
The sign is reported as it is computed. No absolute values are taken and no colour is applied on the worksheet.
A constant column has no correlation. If any model column holds the same value in every run its centred column is zero, it has no length to scale by, and the whole matrix is unavailable rather than part filled.
Leverage by run¶
Leverage is the diagonal of the hat matrix, the matrix that turns the observed responses into the fitted ones. For run \(i\),
where \(\mathbf{x}_i^{\prime}\) is run \(i\)'s row of the design matrix. It measures how unusual run \(i\) is in its factor settings alone, with no reference to the response: the response does not appear in the expression at all. Leverage runs from \(0\) to \(1\), and the leverages of all runs add up to \(p\), the number of terms in the model counting the constant, so the average leverage across a design is \(p/n\).
For a quantitative output this value is produced by the regression library. The definition above is the standard one.
The runs are numbered from 1 in sheet order.
Shared Math Details used here¶
This report uses shared formulas defined once in Shared Math Details.
| Shared concept | Used here for | Reference |
|---|---|---|
| Design coding | the coded values that fill \(X\) | Design Coding |
| Prediction equation | the model whose terms are the columns of \(X\) | Prediction Equation |
| Regression diagnostics | leverage | Regression Diagnostics |
| Power and sample size | the noncentral \(F\) form the power figures take | Power and Sample Size |
| Aliasing and resolution | the alias table the design sheet prints, and how it differs from the matrix | Aliasing and Resolution |
See Also¶
- Evaluate Design
- Options
- How-To
- Create Optimal Design Math Details, where D, A, G and I are defined
References¶
- Atkinson, A. C., Donev, A. N., and Tobias, R. D. (2007). Optimum Experimental Designs, with SAS. Oxford University Press.
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd edition. Lawrence Erlbaum Associates.
- Montgomery, D. C. (2013). Design and Analysis of Experiments, 8th edition. Wiley.
- Myers, R. H., Montgomery, D. C., and Anderson-Cook, C. M. (2016). Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 4th edition. Wiley.