Home / DOE / Analyze / Evaluate Design

Evaluate Design¶

This command measures a design sheet against its own model and writes the diagnostics to a new worksheet. It reads the design only: no response values are used, and none are needed, so a design can be evaluated before a single run is performed.

QXL DOE > Analyze Design > Design Diagnostics > Evaluate Design

When to use it¶

Use it on a design you already have, whether it came from one of the design generators, from Create Optimal Design, from Create Custom Design or from Modify Design. The report answers questions about the design's structure: how well the model terms are separated, which runs carry the most weight, what each term is aliased with, and how likely the experiment is to detect a term of a stated size.

The same measures are shown for a design the optimal-design wizard has just built, on its own Results page. Evaluate Design is how you get them for a design that did not come from that wizard, and for one that has since been modified.

What you get¶

A new worksheet named Evaluate Design. The design sheet is not changed.

The sheet carries a summary in its upper left corner and then six blocks, one under another, in this order:

Block	What it holds
Design measures	nine single numbers describing the whole design
Variance Inflation Factor (VIF) by model column	one figure per model column, the constant excluded
Power by model column	the three values the power was computed from, then one row per model column and one column per replicate count, 1 through 5
Alias matrix	model columns down the side, alias terms across the top
Correlations among the model columns	the signed correlation between each pair of model columns
Leverage by run	one figure per run, numbered from 1 in sheet order

Each block is drawn in the same style as the regression report: a title, a bold column header row where the block is a table, an outline round the block at that block's own width, and light grey on alternate rows so a row can be followed across. Gridlines are hidden.

Every number on the sheet is one figure the evaluation engine returned. Nothing is recomputed on the way to the worksheet.

The summary¶

Line	What it says
Design sheet	the name of the sheet that was evaluated
Runs	how many runs it has
Model columns	how many columns its model matrix has, the constant counted
Blocks	how many blocks the design runs in. Present only on a blocked design
Engine status	why the engine did not fully succeed. Present only when it did not

Three cases where the sheet is shorter¶

A blocked design. Three of the nine design measures are taken over the design region, the coded box the factors span: G-efficiency, I (average prediction variance) and the maximum prediction variance. Computing them means turning a point of that region into a row of the model matrix, and a block is not a factor setting, so there is no value of "block" to give such a point. The engine reports those three as unavailable and the sheet leaves them off altogether: no row, no note and no sentence in their place. A blocked design's Design measures block is six rows instead of nine.

Blocked forms of these criteria do exist in the literature, through a mixed model with random block effects. This engine fits ordinary least squares on \(X'X\) and has none of that machinery, so what is true is that this engine cannot compute them.

One reading hazard worth knowing, which the sheet does not state: D-efficiency and A-efficiency scale against an orthogonal design using the model matrix column count as the parameter count, and on a blocked design that count includes the block columns. A blocked design's D-efficiency is therefore not comparable with the same design's unblocked D-efficiency. That is the standard convention rather than a defect.

The power analysis turned off. Clearing Include power analysis on the dialog leaves the whole power block off the sheet, and the three power values go with it. Nothing takes their place, and the sheet has five blocks instead of six.

A saturated design. A design with no error degrees of freedom left at a given replicate count has no \(F\) test to run, so no power is reported at that count. The cell reads Not Avail. The sheet never says why a power figure is unavailable, deliberately: there is no footnote and no legend.

When a measure is unavailable¶

A variance inflation figure the engine did not report reads Not Avail. With the constant off the table, the reasons left are an aliased column, which has no variance to inflate, and a variance inflation computation the engine could not finish.

When the whole block cannot be produced, a sentence stands in its place instead of a table:

Power is not reported for this design. The engine reports no power for a singular design.

The alias matrix is not available because this design is singular, so the aliasing cannot be worked out.

The alias matrix is not shown because the model already holds every two-way interaction and every eligible quadratic, so there are no alias terms left.

The correlation map is not available because at least one model column holds the same value in every run of this design, so it has no correlation with anything.

The correlation map needs at least two model columns besides the intercept, and this design has 1.

What is not reported¶

No whole-effect test for a categorical factor with more than two levels. Power is reported per model column. A categorical factor with three or more levels occupies several model columns, and each gets its own power figure; there is no combined figure for the factor as a whole.

No prediction-variance profile, prediction-variance surface, relative coefficient standard errors or Fraction of Design Space curve. The feature reports what the evaluation engine produces, and none of those is among it.

The condition number is not reported, for the reason given on Create Optimal Design Math Details.