Home / Statistical Tools / MSA / Attribute MSA: Crosstabulations Method

Attribute MSA: Crosstabulations Method¶

An attribute study analyses a measurement system whose output is a judgement rather than a number: pass or fail, conforming or not, grade A, B or C. There is no variance to decompose, because there is no measurement scale to have variance on. What there is instead is agreement, and this study measures it four ways.

Find it at QXL Stat Tools > MSA / Gage R&R > Attribute MSA - Crosstabulations Method.

When to use it¶

Use it when appraisers assign each part to a category rather than measuring it. A visual inspection against a boundary sample, a go or no-go gage, a pass or fail functional test, and a severity grading are all attribute measurement systems.

If your appraisers write down a number on a scale, run Crossed, Nested or Extended instead. Those separate the variation in the numbers; this one cannot, because categories do not have variation in that sense.

The four questions it answers¶

Everything in the report hangs off four comparisons, and the report names them in these words:

Section	The question it asks	What it needs
Within Appraiser	Does one appraiser agree with themselves from trial to trial?	At least 2 trials
Each Appraiser vs Standard	Does each appraiser agree with the right answer?	A standard column
Between Appraisers	Do the appraisers agree with one another?	At least 2 appraisers
All Appraisers vs Standard	Do they all, together, agree with the right answer?	A standard column and at least 2 appraisers

A section whose requirement is not met is left out of the report entirely, with no heading and no explanation. Absence is the answer: a study with one trial per appraiser has nothing to say about whether an appraiser repeats, so it says nothing.

What the study needs¶

Four columns, in the layout one row per rating:

Column	Required	What it holds
Rating	Yes	What the appraiser decided. One column, or several, and each column becomes its own study
Part	Yes	Which part was being judged
Appraiser	Yes	Who judged it
Standard	No	The known right answer for that part, the same label the ratings use

Every appraiser must rate every part the same number of times. An unbalanced study is refused with a sentence saying which appraiser rated which part how many times.

Without a standard column you still get Within Appraiser and Between Appraisers: whether the appraisers are consistent. You do not get anything about whether they are right, because nothing in the data says what right is.

The three data types¶

The Data Type you choose on the Options tab decides which statistics can be computed:

Binomial (two levels)

: Pass and fail, conforming and nonconforming: exactly two categories. Offered only when the data holds exactly two levels.

Nominal (two or more unordered levels)

: Categories with no natural order: defect types, colours, dispositions. Always offered, and the choice the study falls back to.

Ordinal (three or more ordered levels)

: Categories that rank: grade A, B, C, or severity 1 through 5. Adds Kendall's Coefficient of Concordance and Kendall's Correlation Coefficient, which use the ordering. Ordinal needs at least three distinct values in the data, and is refused below that.

Only Ordinal changes what is computed

Binomial and Nominal are the same study. The choice between them changes the Data Type line in the report's summary and nothing else. In particular, the Assessment Disagreement table is not something Binomial unlocks: it appears whenever the data holds exactly two levels and there is a standard, whichever of the two you picked. Choosing Binomial on data that turns out to hold three or more levels does not fail either; the study runs as nominal and says so in a note.

Ordinal is the choice that does change the arithmetic, because it is the only one that says the levels have an order.

The page sequence¶

The dialog has three tabs, Data, Options and Gage Info. Every control is described on Options.

On Data, tick the Rating column or columns, and name the Part, Appraiser and, if you have one, Standard columns.
On Options, choose the data type, set the level order if the study is ordinal, choose the conforming level if you have a two-value standard, and set alpha.
On Gage Info, optionally record the gage and study details.
Choose Finish.

The report appears before the dialog does¶

Running the command computes a study from sensible defaults and writes the worksheet first. The dialog then opens on top of the finished report. Every change you commit closes the dialog, rewrites the worksheet and reopens the dialog with your settings still in place, so the sheet always matches the boxes in front of you.

Starting from a template¶

Create MSA Template builds a data collection sheet for an attribute study, with the parts numbered, the appraiser and trial columns laid out, and a Data type: cell already on it. Run the analysis with that sheet active and the study is read straight off it.

On a template run the Data tab is hidden, because the template has already said which column is which, and the dialog opens on Options. Everything else behaves the same way.

What you get¶

One worksheet, named Attribute MSA, carrying every study the run produced, stacked down the page with five blank rows between them. Up to 100 studies run; past that the sheet is replaced by the message Too many results. Please reduce the number of groups.

The sheet is protected when it is written. The seven Gage Information value cells are left unlocked so you can still fill them in, and a note at the bottom left says so in these words: This worksheet is protected to prevent accidental changes. To unprotect it, select the Review tab and then Unprotect Sheet.

How one study is laid out¶

Columns A and B carry the input echo, the Gage Information block and then, for each study, a Study Summary block. Column C is left blank. The charts and every table sit from column D rightwards.

Study Summary records what the study was, so the sheet can be read a year later: Parts, Appraisers, Trials per Appraiser, Data Type, Levels, Known Standard, Conforming Level, Alpha and Confidence Level. A Source row naming the template is added when the study came from one. Underneath sit any notes about what the run had to do with your data, for example a row left out for a blank Part cell, or two spellings of one level treated as the same level.

The charts¶

Up to two, each six columns wide and eighteen rows tall, drawn level with the study's title:

Assessment Agreement Within Appraiser
Assessment Agreement Appraiser vs Standard

Each plots one point per appraiser on a category axis, with Percent Agreement up the Y axis from 0% to 100%. Three markers per appraiser show the agreement percentage as a filled circle and its two confidence bounds as crosses, joined by a red vertical line, so the length of the red line is the width of the interval. There are no connecting lines between appraisers, because appraisers are not in any order. A chart whose section was not computed is simply not drawn, and a lone chart takes the first chart position rather than leaving a gap.

The tables¶

The four sections come first, in the order in the table above, and each carries the tables its data supports:

Assessment Agreement, with # Inspected, # Matched, Percent and a confidence interval. A part counts as matched only if every relevant rating on it agreed, so this is a strict, all-or-nothing figure. A note under each table says exactly what its # Matched counted.
Assessment Disagreement, in the Each Appraiser vs Standard section and only when the data holds exactly two levels, splitting the errors by direction and counting the parts the appraiser rated inconsistently.
Fleiss' Kappa Statistics, one row per level plus Overall, with Kappa, SE Kappa, Z and P(vs > 0).
Cohen's Kappa Statistics, in the same shape, where the comparison is between exactly two things.
Kendall's Coefficient of Concordance and Kendall's Correlation Coefficient, for ordinal data only.

Then the tables that stand outside the four sections:

Agreement Between Appraiser Pairs, a kappa for every pair of appraisers.
Pairwise % Agreement, a gentler figure than assessment agreement: it counts agreeing pairs of ratings rather than demanding that a whole part be unanimous. Reported By Part, By Appraiser and Overall.
Effectiveness, the fraction of individual decisions that matched the standard, with a confidence interval, per appraiser and overall.
Agreement Counts, correct and incorrect counts per appraiser and level.
Misclassifications, a grid of what was rated as what.
Conformance, the probability of a false alarm and of a miss, per appraiser.
Assessment Disagreement with Standard by Part, showing which parts caused the trouble.

Any figure that could not be computed reads N/A rather than being left blank or filled with a placeholder.

When a study cannot be computed¶

A refused study prints its title, then the heading This study could not be computed, then a sentence naming the condition, then any notes about the data. No summary, no charts and no tables. Other studies in the run are unaffected. Options lists every sentence.