Home / DOE / Analyze / The DOE Advisor / The DOE Advisor How-To

The DOE Advisor How-To¶

This walkthrough runs a regression on a known set of sixteen runs and then reads the Advisor block it writes, section by section. Every finding below is one this fit really produces, so you can check your block against it rather than against a description of what a block might contain.

Every chart in this family is drawn from a fitted model, so the walkthrough starts by building one. The same experiment is used on all ten chart walkthroughs, so once you have the regression sheet you can work through any of the others without setting the data up again.

The data is on this page rather than in a file to download. Press Copy for Excel, then paste it into a blank worksheet.

The data¶

A three factor experiment on a bonding process, run twice through. Temp is in degrees C, Press in psi and Time in seconds, and the response Strength is the peel strength of the finished bond. Each of the eight factor combinations was run twice, giving 16 runs.

Temp	Press	Time	Strength
180	40	10	49.5
220	40	10	65.4
180	60	10	61.8
220	60	10	66.8
180	40	20	52.3
220	40	20	70.7
180	60	20	60.7
220	60	20	70.0
180	40	10	50.6
220	40	10	66.4
180	60	10	61.2
220	60	10	67.2
180	40	20	51.4
220	40	20	67.1
180	60	20	61.9
220	60	20	71.6

Each factor takes exactly two values, so this is a full factorial of the three at two levels each, replicated once. That matters for more than one chart: a factor with exactly two observed values is the only kind that can carry a Cube Plot axis, and a design with centre points would have failed that test.

Steps¶

Put the data in Excel

Press Copy for Excel above the table. In Excel, open a blank worksheet, click cell A1, and press Ctrl+V. You should have headers in row 1 and the 16 runs in rows 2 through 17.
Make a design sheet to hold it

From the Excel ribbon: QXL DOE New > Create Design > Special > Create Custom Design.

On the first page set the number of factors to 3 and the number of runs to 16. On the second page set the number of outputs to 1. That gives you an empty design sheet of the right shape, which is how data collected outside Quantum XL gets analysed.
Fill in the design sheet

Copy the four columns of data into the three factor columns and the one output column. Name the factors Temp, Press and Time, and the output Strength, so the names on your charts match the ones quoted below.
Answer the two questions a custom design asks

A custom design does not record how it was built, so the first time you run a regression, run Optimize, or open Modify Design on it, Quantum XL asks for what the sheet cannot say, and stores your answers so it never asks again.

On the first page leave Type: at Quantitative for all three factors. On the second, the interaction editor, move AB, AC and BC from Interactions available into Interactions in model: three main effects plus those three interactions is the model every walkthrough below assumes. The remaining degrees of freedom fall as you add each one, because the sixteen runs are fixed and every term spends one.

Afterwards the same editor is reachable at any time from QXL DOE New > Modify Design > Inputs > Add/Remove Interactions.
Run the regression

QXL DOE New > Analyze Design > Run Regression. A worksheet called Regression is added after the design sheet, and it is the sheet the walkthroughs below read.
Read the block below the regression

It is written automatically, so there is no command to run and nothing to switch on if you have left the default alone. The banner sits above the regression and the block below it.

Check the fit first¶

Check the coefficient table on the Regression sheet against this first. If these numbers match, every number quoted further down will match too; if they do not, the design was not entered the way this walkthrough assumes and nothing below will line up.

Term	Coefficient	Standard error	t	p
Constant	62.1625	0.2912	213.477	0.000000
Temp (A)	5.9875	0.2912	20.562	0.000000
Press (B)	2.9875	0.2912	10.260	0.000003
Time (C)	1.0500	0.2912	3.606	0.005696
AB	-2.2375	0.2912	-7.684	0.000030
AC	0.6500	0.2912	2.232	0.052506
BC	-0.1500	0.2912	-0.515	0.618877

The fitted equation is

Strength = 62.1625 + 5.9875 A + 2.9875 B + 1.0500 C - 2.2375 AB + 0.6500 AC - 0.1500 BC

in coded units, where each factor runs from -1 at its low value to +1 at its high value.

Every standard error is the same 0.2912. That is not a coincidence and it is a useful sign that the design was entered correctly: on a balanced full factorial every coded column is orthogonal to every other and carries the same amount of information, so every coefficient is estimated equally precisely.

What this fit produces, section by section¶

Type Design: Custom¶

The steps above build the sheet with Create Custom Design, so the Advisor classifies it as Custom and prints Custom designs are typically used to analyze historical data. Build the same sixteen runs as a 2-Level Factorial with two replicates instead and the same data is classified Modeling, with a different sentence under it. The classification comes from how the sheet was built, not from what is in it.

Next Steps: four bullets, and why each one is there¶

Bullet	Why this fit gets it
Reduce the model by removing insignificant terms.	BC has a p value of 0.618877, above 0.1
Your model has significant interactions; consider creating an interaction plot to visualize these interactions.	AB has a p value of 0.000030, at or below 0.05
If you've fully reduced the model, consider validating the model using the validation points below.	always, on a modeling or custom design
If you need the output to be maximized, minimized, or set to a target value, consider using the optimizer to find the optimal inputs.	always, on a modeling or custom design

The quadratics bullet is absent because the model holds no quadratic or custom term. The warnings bullet is absent if the Warnings section is empty, which is the next thing to check.

Warnings: eight of the nine conditions are closed before you look¶

This is the section worth working through, because knowing why a warning is not there is how you learn what it would take to produce one.

Warning	On this fit
Variance Inflation Factor	cannot fire. Every VIF is 1, and the threshold is 2. A balanced full factorial has orthogonal columns
Outliers	cannot fire. It needs a standardized residual at or beyond plus or minus 3, and the largest here is -2.4612
Missing data	cannot fire. All sixteen rows are complete
Aliasing	skipped. It applies to modeling designs, and this sheet is classified Custom
High Sig F	skipped, for the same reason, and it could not fire anyway: the significance of F is 0.0000001 against a threshold of 0.05
Block	cannot fire. There is no blocking term
Quadratic Terms Removed	cannot fire. There are no quadratics to remove
Failed to Converge	not applicable. It belongs to the two logistic engines; least squares does not iterate
Non-Normal Residuals	this is the only one left open. It fires when a Shapiro-Wilk test on the residuals returns a p value at or below 0.05, and the sheet reports the result

So the banner is green unless the normality test fails. A green banner reads There are no warnings for this model. Click here to see DOE Advisor. and a red one reads There are some warnings for this model. The banner is the fastest thing on the sheet to read and it is worth reading first.

Notes: one finding, on one run¶

The Notes section reports Large Standardized Residuals have been found, and the run it means is run 14.

Run	Settings	Strength	Fitted	Standardized residual
14	220, 40, 20	67.1	69.25	-2.4612

No High Leverage finding appears with it, and that is checkable rather than lucky: every run of this design has a leverage of 0.4375, seven active terms over sixteen runs, and the threshold is three times that same ratio, 1.3125. On a balanced design no run can ever reach it, so the leverage half of the Notes section stays silent whatever the responses are.

Run 14 is in Notes and not in Warnings, because its residual is past 2 and not past 3. That is the whole difference between the two sections for a residual: at or beyond plus or minus 2 it is Large and goes to Notes; at or beyond plus or minus 3 it is Extreme and goes to Warnings.

The run it marks red, on the design sheet¶

Look at the design sheet, not the regression sheet. Run 14's Strength cell is filled red with a comment on it reading Large Standardized Residuals point (Standardized Residual = -2.4612). Just that one cell, not the whole row.

Run 14's pair is what makes it worth investigating. Run 6 was made at the same three settings and returned 70.7 where run 14 returned 67.1. The two disagree by 3.6 where the residual standard deviation of the whole fit is 1.16, so the model sits between them and neither is close to it. The Advisor cannot tell you which of the two is the odd one, or whether either is: a mistake in one run, a real effect the model does not carry, and ordinary variation all look like this.

Potential Validation Points: three lines, one per factor¶

Each main factor still in the model gets the midpoints between the settings it was actually run at, in uncoded units. All three factors were run at exactly two levels, so each gives one midpoint:

Factor	Validation setting
Temp	200
Press	50
Time	15

Those are settings the experiment never used, which is the point: the model predicts 62.1625 at that combination, and running it is how you find out whether the model is right somewhere it has not been tested. Add three centre points to the design and this section changes, because the factors then have three observed levels each and two midpoints apiece.

Now make the block change¶

Remove BC and run the regression again. The Reduce-the-model bullet disappears from Next Steps, because no term is left with all its p values above 0.1: the weakest is now AC at 0.042844.

Run 14 is still the only Note, its standardized residual moving from -2.4612 to -2.2564, and the validation points do not change. Leverage moves too, from 0.4375 to 0.375 on every run, six active terms over sixteen instead of seven, with the threshold falling from 1.3125 to 1.125. Still nowhere near it, which is the point.

Run Uncoded Coefficients instead and the block is three sections long: the title, Type Design, and a section headed Uncoded units. No Next Steps, no Warnings, no Notes, no validation points, and no run marked red. What is left out and why is on The DOE Advisor.

Watch out for one thing there. An uncoded run clears the red marks an earlier coded run left, so a clean design sheet after an uncoded regression does not mean the earlier run found nothing.

What to do with a finding¶

A warning is a finding, not a verdict. Read what it says about the fit, then look at the number it came from on the regression sheet rather than acting on the sentence alone.

A run marked red is not automatically wrong, and a red cell is not even evidence of a large residual by itself: leverage alone can mark one. Read the comment on the cell, which says which condition it met.

Things to try next¶

Turn it off and on. Display DOE Advisor on regression sheet, on the Regression page of QXL DOE New > Options. Nothing else about the regression changes, and the red marks from the last run stay on the design sheet until a regression runs with the Advisor back on.
Break the VIF check on purpose. Build a custom design whose runs are not balanced and the VIFs rise above 1; reach 2 and the warning appears.
Make an Extreme residual. Change run 14's Strength to something further out and run again: past a standardized residual of 3 the finding moves from Notes to Warnings and the banner turns red.