Home / Statistical Tools / Control Charts / Levey-Jennings / Math Details
Math Details¶
This page gives the exact formulas Quantum XL uses to build a Levey-Jennings chart. Each equation lists what it computes and where it appears on the chart. Most of the formulas are shared with other control charts, and each of those blocks is followed by a link to the page that owns it. Two things on this page belong to this chart alone: the use of the overall standard deviation in the limits, and the R-4s rule.
Notation¶
| Term | Description |
|---|---|
| \(i\) | point index, in plotted order. One point is one observation. |
| \(x_i\) | the \(i\)th observation, and the value plotted at point \(i\) |
| \(N\) | the number of non-missing observations in the baseline set. On this page \(N\) always means an observation count; the rule thresholds below are written out as numbers rather than given a symbol. |
| \(\bar{x}\) | the mean of the observations in the baseline set, and the center line |
| \(\hat{\sigma}_{\text{overall}}\) | the overall (long-term) standard deviation of the individual observations |
| \(k\) | sigma multiplier (number of standard deviations), default \(3\) |
| \(k^{*}\) | the independent sigma multiple used by the 1-2s rule, fixed at \(2\) |
| \(\mathcal{B}\) | the baseline set of points the estimates are computed from |
| \(\text{CL}\), \(\text{UCL}\), \(\text{LCL}\) | center line, upper and lower control limits |
| \(A_i^{\pm}\), \(B_i^{\pm}\) | the outer and inner zone lines at point \(i\) |
| \(c_4(N)\) | the expected value of the sample standard deviation of \(N\) standard normal observations |
Plotted statistic¶
The chart plots the measurement itself, in worksheet order, one point per row. Nothing is averaged, differenced, or accumulated. Every usable row is plotted whatever the baseline selection is, because the baseline governs the limits and never what is drawn.
Center line¶
The center line is the mean of the observations in the baseline set, which is every usable observation unless Estimate Limits From restricts it. A supplied historical mean replaces \(\bar{x}\) after it has been computed, so the baseline still decides which observations fed the estimate that was discarded.
The standard deviation behind the limits¶
This is the one formula that separates a Levey-Jennings chart from every other individuals-form chart in the product. The limits are built from the overall standard deviation, the ordinary sample standard deviation of the individual observations about their mean:
\(N\) is the number of non-missing observations inside the baseline window and \(\bar{x}\) their mean. When Use unbiased constants (overall sigma) is checked, the estimate is divided by \(c_4(N)\):
Full details: Process Sigma Estimation.
Three points about how that block applies here:
- The estimator is fixed in the calculation engine. There is no Estimate Sigma group in the dialog and no way to choose another estimator. An I-MR chart, an Individuals chart, an Xbar-R chart and an Xbar-S chart all build their limits from a short-term estimate, formed either from the spread inside each subgroup or from how much consecutive points differ. This chart uses none of those. It uses the spread of all the data about one mean, so anything that moves the process over the life of the chart is inside the estimate rather than outside it, and the band widens instead of the points falling outside it.
- The unbiased form is never used. The \(c_4(N)\) correction shown in that block is reachable on other charts through the Use unbiased constants (overall sigma) checkbox. This dialog does not show that checkbox, and the engine rejects an unbiased request for this chart type, so \(\hat{\sigma}_{\text{overall}}\) here is always the uncorrected form.
- There is no within-subgroup sigma anywhere in the output. The summary table reports the mean, the overall sigma and the coefficient of variation, and no within row, because no within estimate is formed.
A supplied historical standard deviation replaces the estimate after it has been computed. It must be greater than zero.
Control limits¶
Every limit on a Quantum XL control chart has the same form, a center line plus or minus \(k\) standard deviations of the plotted statistic:
Full details: Control Limits and Zones.
On this chart the plotted statistic is a single observation, and one standard deviation serves the whole chart, so \(\sigma_i\) does not vary from point to point and the general form collapses to two constants:
The limits are therefore straight lines rather than the stepped ones a chart of subgroup statistics produces. Neither limit is floored or capped: an individual measurement can legitimately take any value, so both are drawn wherever the arithmetic puts them. With split control limits in effect the pair above is recomputed inside each phase, giving one straight band per phase.
Under Manual limits the three values you typed replace \(\text{UCL}\), \(\text{CL}\) and \(\text{LCL}\) directly and \(\hat{\sigma}_{\text{overall}}\) takes no part in the drawing, although it is still estimated and still reported in the summary table.
Sigma zones¶
The zone lines, and the three Westgard rules that read them, are placed by cutting the drawn band into thirds on each side. The sigma multiplier does not enter the calculation.
On the variable charts the band that is actually drawn is cut into thirds on each side, and the sigma multiplier \(k\) does not enter the calculation:
Each side is measured from its own limit, so an asymmetric band gives asymmetric zone lines, and a manual band is divided the same way as a calculated one. On the range, standard deviation, and moving range panels the zone lines are built from the unclamped lower limit and then floored at zero, so when the drawn lower limit has been clamped to zero the lower zone lines are not thirds of the band that appears on the chart.
Full details, including what the labels "1 sigma" and "2 sigma" really mean at a multiplier other than \(3\): Control Limits and Zones.
At the default \(k = 3\) a Levey-Jennings band is \(\pm 3\hat{\sigma}_{\text{overall}}\), so its thirds fall on a true one and two standard deviations and the 2-2s, R-4s and 4-1s rule names describe what those rules actually test. At any other multiplier the lines are still thirds of the band you asked for, which changes what those three rules count. The band is symmetric about the center line whenever the limits are calculated, so the zone lines are symmetric too; a manual band that is not symmetric is divided into thirds on each side separately.
The six Westgard rules¶
Levey-Jennings replaces the eight Shewhart tests with the six Westgard multi-rule tests, and all six are on when the dialog opens. The chart has one panel, so all six run on it. Five of the six are tests the shared rule engine already provides, applied at Westgard thresholds; R-4s is offered on this chart alone.
| Westgard rule | Condition | Parameters | Shared definition |
|---|---|---|---|
| 1-2s | one point beyond \(\text{CL} \pm k^{*}\) sigma, a warning | \(k^{*} = 2\) | Rule 1, independent mode |
| 1-3s | one point beyond a drawn control limit | none | Rule 1 |
| 2-2s | \(2\) of \(2\) consecutive points beyond the same outer zone line | \(2\) of \(2\) | Rule 5 |
| R-4s | a consecutive pair straddling the plus and minus outer zone lines | none | R-4s, below |
| 4-1s | \(4\) of \(4\) consecutive points beyond the same inner zone line | \(4\) of \(4\) | Rule 6 |
| 10x | \(10\) consecutive points on one side of the center line | \(10\) | Rule 2 |
Of the eight Shewhart rules only 1, 2, 5, and 6 have Westgard counterparts, so rules 3, 4, 7, and 8 never run on this chart, whatever their stored settings say. Rule 9 does not arise here at all: it is offered on the G chart only and is not one of the eight. Each of the five shared conditions is written out in full on Out-of-Control Rules; the rest of this section covers the two that need saying here.
1-2s and 1-3s: the same test at two widths¶
Both are the beyond-a-limit test. 1-3s uses the limits as drawn. 1-2s uses the engine's independent mode, where one sigma of the plotted statistic is recovered from the drawn band and the test is applied at \(k^{*} = 2\) of those units:
Point \(i\) is flagged when the plotted value falls outside the limits drawn at that point:
Both comparisons are strict, so a point sitting exactly on a control limit is not flagged. The test uses the limits as drawn, which means it follows stepped limits, manual limits, and transformed data with no special handling. The rule has no threshold of its own: its width is the chart's sigma multiplier \(k\), chosen with the control limits.
The engine also supports an independent multiple \(k^{*}\) for this rule. When one is supplied, one sigma of the plotted statistic is recovered separately for each side from the drawn limits,
and the test is applied at \(k^{*}\) of those units instead:
Recovering each side on its own means an asymmetric band gives an asymmetric per-side sigma. The only chart that uses an independent multiple is Levey-Jennings, whose 1-2s warning is this same test at \(k^{*} = 2\). On every other chart Rule 1 tests the drawn limits.
Full details: Out-of-Control Rules.
Because one sigma is recovered from the band as \((\text{UCL} - \text{CL})/k\), the two tests differ only in their width: on a calculated chart 1-3s fires at \(\bar{x} \pm k\,\hat{\sigma}_{\text{overall}}\) and 1-2s at \(\bar{x} \pm 2\,\hat{\sigma}_{\text{overall}}\). At the default \(k = 3\) that is the classic pairing of a warning at two sigma and a rejection at three. Set \(k\) to \(2\) and the two tests coincide.
R-4s¶
R-4s is the one test in the product that no Shewhart rule number corresponds to. In its original clinical form the range rule compares two control materials measured in the same run and fires when the two results differ by more than four sigma with one high and one low. A Levey-Jennings chart in Quantum XL carries a single stream of measurements rather than two materials, so the rule is adapted to consecutive pairs of that one series:
For each consecutive pair of points, R-4s fires when one point of the pair is beyond the upper outer zone line and the other is beyond the lower one, so that the pair straddles the center line with a span greater than four sigma:
Both points of a firing pair are flagged. Two points beyond the same outer zone line do not fire this rule; that pattern is 2-2s. R-4s has no parameters.
Full details: Out-of-Control Rules.
The pair therefore has to straddle the center line. Two points beyond the same outer zone line is the 2-2s pattern, not this one. Because the test reads the outer zone lines, it moves with the sigma multiplier in the same way 2-2s and 4-1s do.
Blank points¶
A blank row keeps its position on the chart and is plotted as a gap, with no center line, no limits and no zones at that position. The gap breaks every run rather than bridging it, and the pair containing a blank point is skipped by R-4s:
A blank row keeps its position on the chart: it is plotted as a gap, and the points after it do not shift left. Each rule handles that gap in its own way.
| Rule | Effect of a blank point |
|---|---|
| 1 | the point is skipped and is never flagged |
| 2 | both same-side counts reset, so the gap breaks the run |
| 3 | both counts reset when either the current or the previous point is blank |
| 4 | the alternation count resets when any of the three points forming the two steps is blank |
| 5 and 6 | every window containing the blank point is discarded, so the gap blocks all \(M\) windows that overlap it |
| 7, 8, and 9 | the run count resets, so the gap breaks the run |
| R-4s | the pair containing the blank point is skipped |
A gap therefore breaks a run rather than bridging it.
Two other situations remove a point's limits, and they are not the same as a gap:
- A subgroup with no usable observation has no plotted value either, so it is a gap and the table above applies.
- An incomplete set of manual limits is rejected for the whole panel, and the panel is replaced by an error message instead of a chart, so no rule is evaluated or reported for it.
Turning the control limit type off is different again, and is covered separately below.
Full details: Out-of-Control Rules.
Baseline, splits, and the minimum data check¶
None of these changes the formulas above; they change which observations the sums run over.
Estimate Limits From restricts \(\bar{x}\) and \(\hat{\sigma}_{\text{overall}}\) to the first \(N\) usable points and then extends the resulting center line and limits across the whole chart. The minimum is 5 points, the window advances past blanks and excluded outliers rather than spending a slot on them, and the plotted values, the minimum, the maximum, and the counts are always computed from all of the data. Formulas: Baseline and Limit Estimation.
Split Control Limits divides the chart into phases and re-runs the whole estimation inside each one, so each phase gets its own mean, its own overall standard deviation, its own straight band and its own zone lines, and every Westgard run counter and window restarts at the boundary. A baseline count is reused inside each phase, capped at that phase's length. Formulas: Splits and Phases.
The minimum data check, and the one waiver. Ordinarily the analysis stops rather than draw a chart when a phase holds fewer than five usable observations, or when \(\hat{\sigma}_{\text{overall}}\) comes out as zero because the data has no spread. On Levey-Jennings only, that check is skipped when the limits do not depend on the data at all: when both the historical mean and the historical overall standard deviation are supplied, or when manual limits are supplied. In those cases nothing is being estimated, so there is nothing for a short or flat data set to break, and the chart is drawn.
Reported statistics¶
Besides the panel's center line and limits, the summary table carries three rows this chart writes and the subgrouped charts do not: the mean, the overall standard deviation, and the coefficient of variation as a percentage,
which is reported as blank when the mean is zero or when either quantity is not a finite number. A split chart repeats all three inside each phase.
Shared Math Details used here¶
This chart uses shared formulas defined once in Shared Math Details.
| Shared concept | Used here for | Reference |
|---|---|---|
| Process sigma estimation | the overall standard deviation the limits are built from | Process Sigma Estimation |
| Control limits and zones | the general limit form, the thirds-of-the-band zone lines, and the manual and disabled limit types | Control Limits and Zones |
| Out-of-control rules | the five shared Westgard conditions, the R-4s condition, and the treatment of blank points | Out-of-Control Rules |
| Baseline and limit estimation | what Estimate Limits From restricts, and what it does not | Baseline and Limit Estimation |
| Splits and phases | per-phase estimation and the run counters restarting | Splits and Phases |
Levey-Jennings contributes no equation the shared pages do not already define. What it contributes is a choice: of the estimators on the sigma page it takes the overall one, and of the rules on the out-of-control page it takes the Westgard six.
See Also¶
References¶
- Levey, S., and Jennings, E. R. (1950). The use of control charts in the clinical laboratory. American Journal of Clinical Pathology, 20, 1059-1066.
- Westgard, J. O., Barry, P. L., Hunt, M. R., and Groth, T. (1981). A multi-rule Shewhart chart for quality control in clinical chemistry. Clinical Chemistry, 27(3), 493-501.
- Montgomery, D. C. (2013). Introduction to Statistical Quality Control (7th ed.). Wiley.
- Shewhart, W. A. (1931). Economic Control of Quality of Manufactured Product. Van Nostrand.