Home / Tutorials / DOE Tutorial

DOE Tutorial¶

One experiment, start to finish, in five steps. Build a design, run it, fit a model, check the model, and use it to choose settings. Every number below is one you can check against your own sheet.

This is the same experiment the rest of the DOE section uses, so once you have the sheet you can work through any of the individual tool pages without setting the data up again.

The experiment¶

A bonding process with three factors and one response. The question is which settings give the strongest bond.

Factor	Low	High	Units
Temp	180	220	degrees C
Press	40	60	psi
Time	10	20	seconds

The response, Strength, is the peel strength of the finished bond.

Step 1: Build the design¶

Choose the design type¶

QXL DOE New > Create Design. Choose 2 Level Factorial and press Next.

The grid shows factors along the top and the fraction down the side. Pick 3 factors and Full, which is 8 runs, and press Next.

Eight runs is the smallest design that separates everything for three factors: the three main effects, the three two-factor interactions and the three-factor interaction are all estimable and none is aliased with another. A fraction would cost fewer runs and give some of that up, which Two-Level Factorial Designs covers.

Name the factors and the output¶

On Enter Factor Names and Coding, type the three factors and the settings from the table above, leaving Categorical clear on all three. Press Next.

On Define outputs (responses), set Select number of outputs: to 1 and name it Strength. Its Type: is already Quantitative, which is what makes ordinary least squares the engine that runs later.

On the last page leave the replicates at 1, the blocking list at No blocking, the centre points at 0 and Fold design: clear. Press Finish.

You now have eight runs in standard order, the first factor changing fastest, with 180, 40, 10 on run 1 and 220, 60, 20 on run 8.

Double the design¶

One replicate cannot separate noise from misfit. With eight runs and no repeats there are no two runs at the same settings, so nothing measures the process's own variation and the regression's lack-of-fit test has nothing to work with.

QXL DOE New > Modify Design > Change Replicates. Set Select number of replicates: to 2 and finish.

Sixteen runs where there were eight, every setting combination now run twice. See Modify Design.

Enter the responses¶

Press Copy for Excel, then fill the Strength column from this block, matching each row by its settings rather than by its position: the sheet is the authority on which run is which.

Temp	Press	Time	Strength
180	40	10	49.5
220	40	10	65.4
180	60	10	61.8
220	60	10	66.8
180	40	20	52.3
220	40	20	70.7
180	60	20	60.7
220	60	20	70.0
180	40	10	50.6
220	40	10	66.4
180	60	10	61.2
220	60	10	67.2
180	40	20	51.4
220	40	20	67.1
180	60	20	61.9
220	60	20	71.6

Check the model holds the interactions¶

The model below is the three main effects plus AB, AC and BC. Add any that are missing with QXL DOE New > Modify Design > Inputs > Add/Remove Interactions.

Step 2: Fit the model and read it¶

QXL DOE New > Analyze Design > Run Regression. There is no dialog: the command reads the active sheet and fits the model the sheet declares. A worksheet called Regression is added.

The coefficient table¶

Term	Coefficient	Standard error	t	p
Constant	62.1625	0.2912	213.477	0.000000
Temp (A)	5.9875	0.2912	20.562	0.000000
Press (B)	2.9875	0.2912	10.260	0.000003
Time (C)	1.0500	0.2912	3.606	0.005696
AB	-2.2375	0.2912	-7.684	0.000030
AC	0.6500	0.2912	2.232	0.052506
BC	-0.1500	0.2912	-0.515	0.618877

If your table matches this, everything else on the sheet will match too. If it does not, the design was not entered the way this tutorial assumes and nothing below will line up.

Every standard error is the same 0.2912, and that is a useful sign rather than a coincidence: on a balanced full factorial every coded column carries the same amount of information, so every coefficient is estimated equally precisely.

What a coded coefficient means¶

A coefficient is the change in Strength for a one unit change in that factor in CODED units, where each factor runs from -1 at its low setting to +1 at its high one. So the full effect of moving a factor from low to high is twice its coefficient: Temp is worth 11.9750 of Strength across its range, Press 5.9750 and Time 2.1000.

That is what makes them comparable. Degrees, psi and seconds cannot be ranked against each other; coded units can. See Design Coding.

The fit statistics¶

	Value
R-squared	0.9854
Adjusted R-squared	0.9756
Residual standard deviation	1.1648
F	100.893

The analysis of variance, and why the replicate paid for itself¶

Source	df	Sum of squares	Mean square	F	p
Regression	6	821.2675	136.8779	100.893	0.000000
Residual error	9	12.2100	1.3567
Lack of fit	1	1.9600	1.9600	1.530	0.251224
Pure error	8	10.2500	1.2813
Total	15	833.4775

Pure error is the variation between runs made at identical settings, which no model could explain: eight pairs, one degree of freedom each. Lack of fit is what is left, and the F ratio of the two asks whether the leftover is bigger than the noise. At p = 0.2512 it is not, so there is no evidence the model is missing a term this design could have found.

Doubling the design in step 1 is what bought that test. With eight runs the lack-of-fit block is blank.

The DOE Advisor¶

Below the regression, a block written automatically. On this fit it says:

Type Design: Modeling, with one sentence about that class
Next Steps, four bullets, including Reduce the model by removing insignificant terms because BC's p value is 0.618877, and an interaction-plot suggestion because AB's is 0.000030
Notes, reporting a large standardized residual on run 14, which measured 67.1 where the model predicts 69.25
Potential Validation Points, one per factor: Temp 200, Press 50, Time 15

Run 14's cell is filled red on the design sheet with a comment saying why. Its pair is what makes it interesting: run 6 was made at the same settings and returned 70.7. The two disagree by 3.6 where the residual standard deviation is 1.16, so the model sits between them.

A red run is a finding, not a verdict. See The DOE Advisor.

Step 3: Reduce the model¶

Remove BC with Modify Design > Inputs > Add/Remove Interactions and run the regression again.

	Full model	BC removed
R-squared	0.9854	0.9849
Adjusted R-squared	0.9756	0.9774
Residual standard deviation	1.1648	1.1212
F	100.893	130.614
p value on AC	0.052506	0.042844

R-squared went down and adjusted R-squared went up. R-squared can never rise when a term is removed, which is exactly why it cannot be used to choose between models of different sizes. Adjusted R-squared rose because the degree of freedom BC was spending bought less than it cost.

AC crossed 0.05 without its coefficient changing. It is still 0.6500. What changed is the residual standard deviation the test divides by: 1.1648 became 1.1212 when BC's share of the residual went back to the error term.

The reduced model is:

Strength = 62.1625 + 5.9875 A + 2.9875 B + 1.0500 C - 2.2375 AB + 0.6500 AC

in coded units. Stop here. Every remaining term has a p value at or below 0.05.

Step 4: Look at it¶

Five charts, from QXL DOE New > Charts. Each has its own page with a walkthrough; this step is about what the set of them tells you together.

Main Effects Plot¶

Three panels, one per factor, two points each. Every panel's rise is exactly twice its coefficient: Temp 11.9750, Press 5.9750, Time 2.1000.

All three share one value axis, so Temp visibly climbs while Time is nearly flat. That comparison is the point of drawing them together. See Main Effects Plots.

Pareto of Regression Coefficients¶

One bar per term, tallest first, the constant never drawn. It ranks what matters by size: Temp 5.9875, Press 2.9875, AB 2.2375, Time 1.0500, AC 0.6500.

The bars are absolute values, so two of them lost a sign: AB is 2.2375 on the chart and -2.2375 in the table. The chart ranks by size; the table carries the direction. See Pareto of Regression Coefficients.

Interaction Plots¶

This is the chart the AB term exists for. Temp's effect is not one number: it depends on Press.

Press	Strength gained by moving Temp from 180 to 220
40	16.4500
60	7.5000

The main effects plot said Temp was worth 11.9750, which is the average of those two. A main effect is an average over a real disagreement, and with an interaction in the model the average is not the whole story. See Interaction Plots.

Surface and Contour Plots¶

The fitted response across two factors at once, with the third held at a set point. The AB interaction shows as a twist in the surface: it is not a tilted plane. See Surface Plot and Contour Plot.

Residual Plots¶

Run these before quoting any p value. The standard errors, the t statistics and the p values all assume the residuals are independent, of constant spread, and roughly normal, and none of that is checked for you. See Residual Plots.

Step 5: Choose settings¶

Predict first¶

The reduced model predicts 62.1625 at Temp 200, Press 50, Time 15, the centre of the design, which is the constant: at the centre every coded factor is 0.

Those three settings are the Advisor's validation points, and no run was made there. Running one is how you find out whether the model is right somewhere it has not been tested. A model that predicts well inside its own data and badly between it is the failure a designed experiment is supposed to catch.

Optimize¶

QXL DOE New > Optimize searches the fitted model for the settings that best meet one goal.

With the goal Maximize Strength and nothing else set, expect Temp 220, Press 60, Time 20, and a predicted Strength of 70.45.

You can check that by hand on this model. It holds main effects and two-factor interactions only, so it is multilinear: no peak in the middle of the range, and its largest value over the box is always at one of the eight corners. Evaluate all eight and 220, 60, 20 wins.

Optimize takes ONE goal¶

Everything else you care about becomes a constraint on another output, so a single-output design cannot show the control at all. Optimize How-To adds a second response, Warp, to the same sixteen runs and works the trade through: the unconstrained answer gives Warp 0.781, capping Warp at 0.65 costs about 2.4 of Strength, and it is spent almost entirely on Time because Time is the worst buy of the three.

That trade table is the useful output of an optimisation, more than any single row of it.

What the answer is and is not¶

It is a prediction, not a measurement. It comes from the fitted model, so read R-squared before trusting it, and confirm it with an actual run.

It can be a combination no run used. That is the point of optimising a model rather than picking the best run, and it is also why confirming it matters.

Where to go next¶

If you want	Go to
A different design type	Create Designs, twelve of them
A run count no classical design offers	Create Optimal Design (D, A, I)
An output that is pass or fail	Binary Logistic Regression
The equation in your factors' own units	Uncoded Coefficients
The mathematics	DOE Math
Help choosing a design	Design Wizard