Home / DOE / Analyze / The DOE Advisor

The DOE Advisor¶

The DOE Advisor is a block of plain-language findings that Run Regression writes onto the regression sheet, below the regression itself. It reads the fit you just ran and reports what it noticed: terms that could come out, points that behave unlike the rest, a design that cannot separate the effects it was asked about. It also marks the runs it is unhappy with red on the design sheet.

It is on by default. The setting is Display DOE Advisor on regression sheet, on the Regression page of DOE Options, and it ships set to Yes.

Every output gets its own Advisor. A design with three outputs produces three Advisor blocks, side by side in the same left-to-right order as the outputs, all of them below the regressions.

Two rows above each output's regression, the sheet carries a coloured banner that is a link to that output's Advisor block. It says one of two things:

Banner	Fill	When
There are some warnings for this model. Click here to see DOE Advisor.	red, with white italic text	the Warnings section has something in it
There are no warnings for this model. Click here to see DOE Advisor.	green, plain text	it does not

Clicking the banner jumps to the Advisor block. The banner is written whenever the Advisor is on, whether or not there is anything to report.

What the block contains¶

The Advisor block always opens with DOE Advisor for output name, and then prints these sections in this order:

Section	What it is	When it appears
Next Steps	what to do next, as bullets	on a coded regression, though the heading can stand with nothing under it
Type Design: Modeling, Screening or Custom	how the Advisor classified the design, and one sentence about that class	always
Warnings	things it thinks are wrong, one sub-heading each	on a coded regression. With nothing to report it reads There are no warnings for this model.
Notes	things worth knowing that it does not call wrong	only when it has something
Potential Validation Points	factor settings you could run to test the model	only when it has something

An uncoded regression gets a shorter report, described at the end of this page.

Next Steps¶

The bullets appear in the order below, and only the ones that apply are printed.

Bullet, as the Advisor writes it	When it appears
Investigate the warnings below.	the Warnings section has something in it
Reduce the model by removing insignificant terms. Most experimenters will remove terms with p-values greater than 0.1. Rerun the regression after you've removed the terms.	the model still holds an active main factor or interaction whose p values are all above 0.1. Whether another term in the model needs that one to stay makes no difference to this bullet
The model has insufficient degrees of freedom to calculate p-values. Consider removing some terms with smaller coefficients and re-running the regression. With the terms removed, p-values will be available.	the regression produced no usable p values at all. This bullet takes the place of the one above
If you've fully reduced the model, consider validating the model using the validation points below.	always, on a modeling or custom design
Your model has significant interactions; consider creating an interaction plot to visualize these interactions.	a two-factor interaction has a p value at or below 0.05. Quantitative outputs only
Your model has significant quadratics; consider creating a surface or contour plot to visualize these quadratics.	a quadratic or custom term has a p value at or below 0.05. Quantitative outputs only
If you need the output to be maximized, minimized, or set to a target value, consider using the optimizer to find the optimal inputs.	always, on a modeling or custom design
Consider creating a Marginal Means plot or a Pareto of Regression Coefficients to obtain a rank order of the inputs. Consider following up with a modeling DOE with the top inputs from this screening DOE.	screening designs only, and it is the only Next Steps bullet a screening design gets

So a screening design's Next Steps is the Pareto bullet, plus the warnings bullet if there are warnings. A modeling or custom design always gets at least the validation and optimizer bullets.

Type Design¶

The Advisor sorts every design into one of three classes, and the class is not decorative: it decides which of the checks below run at all.

Class	Designs in it	The sentence the Advisor writes
Modeling	Box-Behnken, Central Composite, 2-Level Factorial, 3-Level Factorial, N-Level Factorial	Modeling designs are typically used to create a model and discover significant effects.
Screening	Plackett-Burman, Taguchi	Screening designs are typically used to discover significant effects (interactions are not available).
Custom	Custom Designs, which is also what Setup Historical Analysis opens	Custom designs are typically used to analyze historical data.

A screening design is exempt from the aliasing, Sig F, quadratics, normality and validation point checks, and from every Next Steps bullet except the Pareto one. A custom design is exempt from the aliasing and Sig F checks, and gets the same Next Steps as a modeling design.

Warnings¶

If nothing was found the section reads There are no warnings for this model. Otherwise each finding gets its own bold sub-heading, printed in this order.

Sub-heading	What triggers it
Failed to Converge	the logistic fit did not converge. Binary and Nominal outputs only
Variance Inflation Factor (VIF)	the largest VIF in the model reaches 2. Quantitative outputs only
Outliers	a run has a standardized residual at or beyond plus or minus 3. Quantitative outputs only
Missing data	rows were dropped from the regression
Aliasing	a modeling design's resolution, or its blocked resolution, is II, III or IV
High Sig F	Significance of F is at or above 0.05. Quantitative outputs on modeling designs only
Block	a blocking term has a p value at or below 0.05
Non-Normal Residuals	a Shapiro-Wilk test on the residuals returns a p value at or below 0.05. Quantitative outputs only
Quadratic Terms Removed	the regression could not be solved with the quadratic terms in it

The text under each heading:

Failed to Converge. This model failed to converge. In its current form, this model is suspect. Consider removing some interactions and recalculating the regression.

Variance Inflation Factor (VIF). Two thresholds, two messages, decided by the largest VIF in the model:

at or above 2 and below 10: This model has some VIF elements between 2 and 10 indicating you have lack of orthogonality.
at or above 10: This model suffers from an extreme lack of orthogonality as at least one factor has a VIF over 10. With VIFs over 10, you may find the coefficients are incorrect and potentially even have the wrong sign.

Outliers. Extreme Standardized Residuals have been found. followed by Approximately 99% of the residuals should fall between +/-3; values outside the range +/-3 are considered large and should be investigated as potential outliers. The extreme standardized residuals have been marked red in the design sheet.

Missing data. Following Excel rows were not used in the regression: and then the rows. In a stacked design that is a list of Excel row numbers. In a table design each entry reads row n replicate r, because a run occupies one row and one replicate column. Long lists are cut short with and more...: a table design lists four entries, a stacked design seventeen.

Aliasing. One of two lines, and the line itself is the link to the alias table on the design sheet:

This design is resolution n; click here to go to the aliasing table.
This design has blocked resolution n; click here to go to the aliasing table., which comes from the blocking confound rather than from the design's own resolution.

See Screening versus Modeling for why neither line ever appears on a Taguchi or Plackett-Burman design, even a saturated one that has a real alias table.

High Sig F. (click here to see) This model suffers from a low F value and a Sig F that is greater than .05 and may not be suitable for prediction. The line is a link to the Sig F cell in the model summary.

Block. One or more of the blocking variables is significant.

Non-Normal Residuals. The Shapiro-Wilk test rejected that the residuals are normal at the .05 level. Care should be exercised when using this model. The test needs at least three residuals to run.

Quadratic Terms Removed. Regression cannot be calculated with quadratic terms. Quadratic terms have been removed from the model.

Notes¶

The Notes section holds the two findings the Advisor does not treat as warnings. Both are about individual runs, and both mark the run red on the design sheet.

It opens with a sentence naming what was found, High Leverage and Large Standardized Residuals have been found., or just the one of them that applies, and then describes each:

High Leverage points are not by definition outliers; however, due to their location in the X direction they exert more influence on the regression coefficients than other points. The High Leverage Points have been marked red in the design sheet.
Approximately 95% of the residuals should fall between +/-2; values outside the range +/-2 are considered large and should be investigated as potential outliers. The large standardized residual points have been marked red in the design sheet.

The runs it marks red¶

This is the one thing the Advisor changes outside the regression sheet. It fills the run's output value cell red on the design sheet, and attaches a comment saying why. Not the whole row, just that one cell, and only for quantitative outputs.

Condition	Grade	Comment on the cell
standardized residual at or beyond plus or minus 2	Notes	*Large Standardized Residuals point (Standardized Residual = value).*
standardized residual at or beyond plus or minus 3	Warnings	*Extreme Standardized Residual point (Standardized Residual = value).*
leverage at or above 3 times the number of active terms divided by the number of rows	Notes	High Leverage

A run can meet more than one condition, and then the comment carries a line for each. A run with an ordinary residual can still be marked red on leverage alone, which is why a red cell is not evidence of a large residual by itself. See Understanding Least Square Residuals and Residual Plots.

Each regression clears the previous one's marks first, repainting the output area back to its alternating yellow shading and clearing its comments before marking anything. That happens only while the Advisor is on, so turning the Advisor off leaves the last run's red cells and comments sitting on the design sheet. To clear them, run the regression again with the Advisor on.

Potential Validation Points¶

The last section lists settings you could run to check the model against reality. One line per main factor that is still in the model, written as the factor name, a colon, and the settings separated by semicolons.

A continuous factor gets the midpoints between the settings it was actually run at, in uncoded units. Two levels give one midpoint; three levels give two.
A categorical factor gets its levels listed rather than midpoints.

The section is skipped entirely on a screening design, and on an uncoded regression.

An uncoded regression gets a shorter report¶

Ask for uncoded coefficients and the Advisor prints three things and stops: the DOE Advisor for output name title, the Type Design section, and a section headed Uncoded units reading This regression table is in uncoded units. Typically uncoded equations are created before Monte Carlo Simulations or in other cases when an uncoded equation is required.

There is no Next Steps section, no Warnings section, no Notes and no validation points, and no run is marked red. The VIF check, the residual and leverage checks, the aliasing, Sig F, normality, missing data and quadratics checks are all skipped rather than passed. An uncoded run still clears the red marks left by an earlier coded run, so an empty design sheet after an uncoded regression does not mean the earlier run found nothing.

See Uncoded Coefficients for what else changes in uncoded units.

Turning it off¶

QXL DOE New > Options, the Regression page, Display DOE Advisor on regression sheet set to No. Press Save settings, because nothing on that dialog takes effect until you do. The next regression sheet is then written without the Advisor block and without the banner above each regression, and nothing on the design sheet is repainted, which leaves any red cells from an earlier run where they are.