Home / Statistical Tools / MSA / Nested / Math Details
Math Details¶
The mathematics behind a nested measurement system analysis. A nested study uses the expected mean squares method or restricted maximum likelihood; the XbarR method is not available, so it is not covered here.
The model and the components¶
A measurement system analysis fits a variance components model. Every measured value is written as the sum of the study's overall mean, one contribution per model term, and a residual:
\(\mu\) is the grand mean, \(\tau_t\) is the contribution of model term \(t\) (a part effect, an operator effect, an interaction, or an additional factor in an Extended study), and \(\varepsilon\) is the residual. Each random term is taken to vary with its own variance component, and the residual with \(\sigma^2_{\text{repeat}}\):
Estimating those \(\sigma^2_t\) is the whole job of the analysis. Which estimation method is used decides how they are computed, but not what they mean, and not how they are then grouped and reported.
In a nested study the model terms are the operator main effect and the part effect nested within operator, written as part within operator. There is no part by operator interaction term, because no part appears under more than one operator. The residual is repeatability.
The nesting is what changes the part term's levels: the part effect has one level per distinct physical part across all operators, not one level per part label.
Every estimated component is placed into exactly one of three buckets. The rule has three clauses and no exceptions:
| Bucket | Which components go in it |
|---|---|
| Repeatability | the residual, and nothing else |
| Part to part | every term selected under Part-to-part variation |
| Reproducibility | every other term in the model |
So reproducibility is defined by what is left over rather than by a list of its own. An interaction, an additional factor in an Extended study, and a term whose estimate came from a fixed effect all roll into reproducibility unless the user moved them to part to part.
That last point is what the Part-to-part variation checkbox list on the Model tab does, and the dialog says so in the same words: by default the variation from additional factors and interactions is attributed to reproducibility, and selecting a term attributes its variation to part to part instead.
The three buckets are summed, and two further aggregates are formed from those sums:
\(\sigma^2_{\text{gage}}\) is reported as Total Gage R&R and \(\sigma^2_{\text{total}}\) as Total Variation. Every percentage on the report is a ratio against one of these two, so they are computed once and never recomputed further down the sheet.
The report prints one row per bucket total and one row per member component, in this fixed order: Total Gage R&R, Repeatability, Reproducibility total, each reproducibility member, Part to part total, each part to part member, Total Variation. Members appear in ascending model term order.
Full details: Variance Components.
What is reported for each component¶
Each reported component is also reported as a standard deviation, which is simply its square root:
Because the floor at zero is applied to the variance before the square root is taken, this is always defined.
The study variation of a component is its standard deviation multiplied by \(k\), the study variation multiplier:
\(k\) is set by Study variation multiplier (k): on the Extended options group and defaults to 6, which spans a range of plus and minus three standard deviations. A value of zero or below selects the default rather than being used.
Percent contribution compares variances:
Because the parts are variances and variances add, these percentages sum to 100 across the components.
Percent study variation compares standard deviations:
The multiplier \(k\) cancels, so this is the same number whether it is formed from standard deviations or from study variations.
On the report this quantity is not a column. It appears as four named rows beneath the results table, one for each aggregate, under the abbreviations the gage literature uses:
| Row | Which component's percent study variation |
|---|---|
%GRR |
Total Gage R&R |
%EV |
Repeatability, the equipment variation |
%AV |
Reproducibility, the appraiser variation |
%PV |
Part to part, the part variation |
A row whose component the study did not produce reports NA. The member components of
reproducibility and of part to part have no row of their own here, so their percent study variation is
not reported at all even though the aggregate ones are.
These two percentage columns answer different questions and do not agree, and neither is wrong. Percent contribution is a ratio of variances; percent study variation is a ratio of standard deviations, which is the square root of the same ratio. A component contributing 9 percent of the variance contributes 30 percent of the standard deviation. Percent study variation does not sum to 100 across the components, because standard deviations do not add.
Percent of tolerance compares a component's study variation against the tolerance width:
Unlike the two percentage columns above, this one does move with \(k\). It is not reported when no specification limit was supplied.
The number of distinct categories estimates how many separate groups of parts the measurement system can tell apart:
The ratio is truncated toward zero and then floored at 1, so the reported count is never zero and never negative. A ratio large enough to exceed the integer range is clamped to the largest reportable integer.
Two properties of this number are worth stating because they are easy to assume otherwise:
- There is exactly one of it. It is always computed from the estimated part to part and total gage standard deviations, even when a historical standard deviation was supplied. No historical variant of the count exists.
- It is a truncation, so a reported count of 4 covers a range of underlying ratios. Where a confidence interval on the count is reported, that interval is on the underlying continuous ratio, not on the truncated integer.
Full details, including the tolerance width, the precision to tolerance ratio and the percent of historical process column: Gage Evaluation.
How the components are estimated¶
Expected mean squares¶
The expected mean squares method builds the analysis of variance table, works out what each mean square is expected to be in terms of the variance components, and then solves that system for the components.
Its name is what it does. Every mean square in the table estimates some combination of the variance components; writing down those combinations gives a set of equations, and the components are what solves them.
On balanced data this is the classical analysis of variance estimate, which is what the option label means by "Equivalent to ANOVA when balanced".
For each term \(T\) in the model, the mean square \(MS_T\) has an expected value that is a weighted sum of the variance components plus the residual variance:
\(B\) runs over the random terms of the model and \(c_{TB}\) is the coefficient of term \(B\)'s component in term \(T\)'s expected mean square. Collecting these for every term gives a triangular system of equations in the unknown components.
A fixed term's column is left empty. A fixed effect has no variance component, so no coefficient is computed for it and it takes no part in the solve.
When the data are balanced the coefficients have a closed form. The coefficient of \(B\) in \(T\)'s expected mean square is the total number of observations divided by the product of the level counts of the factors in \(B\):
\(N\) is the number of observations used and \(L_f\) the number of levels of factor \(f\). The coefficient is non-zero only when term \(B\) contains term \(T\).
Read plainly: a component's coefficient is the number of observations that share each of its levels. In a crossed study with \(p\) parts, \(o\) operators and \(r\) replicates, the part component appears in the part mean square with coefficient \(o\,r\), because each part was measured \(o\,r\) times.
The closed form is used only when the data are genuinely balanced, and that is checked rather than assumed.
On balanced data the back-substitution reduces to the familiar textbook form. A term's component is the difference between its own mean square and its matched denominator's, divided by its own coefficient:
The matched denominator is the term whose expected mean square is \(T\)'s with \(T\)'s own component taken out, so the subtraction removes everything except the component being estimated. For a main effect in a simple crossed study the denominator is the interaction, or the residual where there is no interaction.
This is worth stating because it makes the whole method checkable by hand on a balanced study: the numbers in the analysis of variance table are enough to reproduce every component.
The subtraction in that formula is why this method can produce a negative variance estimate. If the mean square being tested comes out smaller than its own denominator, the difference is negative, and dividing by a positive coefficient keeps it negative.
Nothing about the arithmetic is wrong when that happens. A variance component is a non-negative quantity being estimated by a difference of two noisy numbers, and a difference can land on the wrong side of zero when the true component is small.
The report handles it by reporting zero, keeping the unfloored estimate beside it, flagging the row, and printing this footnote:
A negative variance component estimate was set to zero. Consider the REML method, which estimates variance components under the constraint that they cannot be negative.
Full details, including the unbalanced coefficients, the solve order and the F tests: Expected Mean Squares.
Restricted maximum likelihood¶
Restricted maximum likelihood, abbreviated REML, estimates every variance component in the model at once, from the data as a whole, rather than solving for them from an analysis of variance table. It is the method the Automatic setting chooses when the data are not balanced.
The components are estimated under the constraint that a variance cannot be negative. This is the substantive difference between REML and the expected mean squares method, and it shows up directly on the report:
- Under expected mean squares a component's moment estimate can come out negative, and zero is reported in its place with a footnote saying so.
- Under REML nothing negative is ever computed. A reported zero means the constrained estimate reached the zero boundary, and the footnote says that instead.
The product itself points a reader from the first case to the second: the expected mean squares footnote suggests considering the REML method precisely because it constrains the components to be non-negative.
Full details, including what is maximised and what is reported about the fit: Restricted Maximum Likelihood.
Probabilities of misclassification¶
The misclassification block asks what the measurement system does to accept and reject decisions. A part has a true value and a measured value, and they differ by measurement error, so a good part can be rejected and a bad part accepted.
Both quantities are treated as normal. The true value has the study mean and the part to part standard deviation; the measured value differs from the true value by measurement error with the total gage standard deviation:
\(X\) is the true value and \(Y\) the measured value. "In specification" means lying between whichever specification limits were supplied.
Every probability below is a probability over that pair, so it is a statement about the population of parts and the gage together, not about any part that was actually measured.
The two joint probabilities each cover one way of getting the decision wrong, as a fraction of all parts:
Because they are fractions of all parts, they can be added to each other and compared directly against each other.
The two conditional probabilities ask the same two questions of a restricted population: of the parts that really are good, what fraction get rejected, and of the parts that really are bad, what fraction get accepted.
Each is its joint probability divided by the probability of the condition, so the two families carry the same information on different denominators. The conditional numbers are typically much larger than the joint ones, and the difference is entirely the denominator: a conditional probability discards every part that was not in the group it conditions on.
A conditional probability needs a population to condition on. When the specification limits sit far enough from the study mean, one of those two populations effectively does not exist, and the corresponding conditional probability is withheld rather than reported as a number computed from almost nothing.
The rule: once the distance from the study mean to the nearest specification limit exceeds 8 part to part standard deviations, the conditional probability for the empty population is not reported, and the report names which case it is:
| Situation | Withheld | Still reported |
|---|---|---|
| every part is good | the probability a bad part is accepted | the other conditional, both joint probabilities, and the probability the part is good |
| every part is bad | the probability a good part is rejected | the other conditional, both joint probabilities, and the probability the part is good |
The rule is symmetric between the two, and 8 standard deviations is a long way out: the fraction of parts beyond it is about \(6.2 \times 10^{-16}\), roughly one part in \(1.6 \times 10^{15}\).
Full details, including the historical standard deviation, the refusals and the sweep: Probabilities of Misclassification.
Bias and linearity¶
Bias and linearity are the only part of the analysis that needs to know the true value of a part. Everything else in a measurement system analysis compares measurements against each other, so it can say how much the system varies without knowing whether it is right. These two say whether it is right.
That is why both are reported only when a Reference (Optional) column was supplied. Without one the whole block is absent, and its absence is not a problem to be reported.
Bias is the difference between what the system measures and what the part actually is:
\(\bar{y}\) is the mean of the measurements in the group being reported and \(x_{\text{ref}}\) the reference value. A positive bias means the system reads high.
Linearity asks whether the bias is the same everywhere in the measuring range, or whether it changes as the parts get bigger. It is a straight-line fit of bias against reference value:
- \(\beta_1\), the slope, is how much the bias changes per unit of reference value. A slope of zero means the bias is the same across the range, whatever its size.
- \(\beta_0\), the intercept, is the bias the line gives at a reference value of zero.
One row is reported per operator plus one pooled row, so an operator whose bias drifts across the range can be told from one whose bias is merely offset.
The fit uses one point per individual measurement, not one point per part average. A study with ten parts measured three times by each of three operators contributes ninety points to the pooled fit, not ten.
Each linearity row reports the fit and then two quantities derived from the slope:
| Reported | Meaning |
|---|---|
| intercept and slope | each with its standard error, t statistic and p value |
| R squared | the fraction of the variation in bias the line accounts for |
| regression standard error | the spread of the points about the line |
| residual standard deviation | what sets the half width of the confidence band drawn on the chart |
| degrees of freedom | the point count minus two |
| process variation used | the spread of the process the two quantities below are scaled against |
Percent linearity reduces exactly to the absolute slope, because the process variation it is divided by is the same one it was multiplied by. That is worth knowing for two reasons: percent linearity does not depend on the process variation at all, and linearity does. The process variation that was used is reported alongside them for exactly that reason.
Full details, including the four grouping levels, the bias test and the acceptance verdict: Bias and Linearity.