Home / Shared Math Details / MSA / Restricted Maximum Likelihood

Restricted Maximum Likelihood¶

This page defines what the restricted maximum likelihood method estimates and what it reports. It is the method the Automatic setting chooses when the data are not balanced, and it is the only method that reports confidence intervals on unbalanced data.

The components it produces are then handled exactly as Variance Components describes.

Notation¶

Term	Description
\(\boldsymbol{\theta}\)	the collection of variance components being estimated
\(\mathbf{V}\)	the variance matrix of the observations implied by \(\boldsymbol{\theta}\)
\(\mathbf{X}\)	the fixed effects design
\(\mathbf{y}\)	the observations
\(\mathbf{P}\)	the projection that removes the fixed effects
\(n\)	the number of observations used
\(q\)	the number of fixed effects in the model
\(\ell_R\)	the restricted log-likelihood

What the method does¶

Restricted maximum likelihood, abbreviated REML, estimates every variance component in the model at once, from the data as a whole, rather than solving for them from an analysis of variance table. It is the method the Automatic setting chooses when the data are not balanced.

What is maximised¶

The method chooses the variance components that maximise the restricted likelihood. The engine works with that likelihood on the \(-2\log\) scale, so it makes the following criterion as small as possible, and it is this criterion that the report publishes:

\[ -2\ell_R = (n - q)\ln(2\pi) \;+\; \ln\lvert \mathbf{V} \rvert \;+\; \ln\lvert \mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{X} \rvert \;+\; \mathbf{y}^{\top}\mathbf{P}\mathbf{y} \]

The pieces are:

Symbol	Meaning
\(n\)	the number of observations used
\(q\)	the number of fixed effects in the model
\(\mathbf{V}\)	the variance matrix of the observations implied by the current variance components
\(\mathbf{X}\)	the fixed effects design
\(\mathbf{y}\)	the observations
\(\mathbf{P}\)	\(\mathbf{V}^{-1} - \mathbf{V}^{-1}\mathbf{X}(\mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{X})^{-1}\mathbf{X}^{\top}\mathbf{V}^{-1}\), which removes the fixed effects

The final term \(\mathbf{y}^{\top}\mathbf{P}\mathbf{y}\) is the sum of squares that remains after the fixed effects have been fitted at the current variance components.

Two of those four terms are what the word "restricted" refers to. Ordinary maximum likelihood uses the same expression with \(n\) in place of \(n - q\) and without the \(\ln\lvert \mathbf{X}^{\top}\mathbf{V}^{-1}\mathbf{X} \rvert\) term at all. Estimating the fixed effects and the variance components together that way leaves the variance estimates systematically too small, because they take no account of the degrees of freedom spent on the fixed effects. The extra determinant term and the reduced count are what account for it, and in the balanced case they are what make REML agree with the analysis of variance estimates.

Because the criterion is on the \(-2\log\) scale, a smaller reported value is a better fit.

Which terms enter the fit¶

The terms that enter the fit are exactly the terms of the declared model, and nothing is added or dropped by the method itself:

one term per declared factor main effect
each interaction left checked on the Model tab
each nesting relationship declared on the Model tab, which changes what a term's levels are rather than adding a term
the residual

A factor declared Fixed enters as a fixed effect, so it contributes to \(\mathbf{X}\) and has no variance component estimated for it. At least one term must be random. A model in which every factor is declared fixed leaves REML with no variance to estimate, and the engine refuses it with a named reason rather than returning zeros; the expected mean squares method reports the same study.

The non-negativity constraint¶

The components are estimated under the constraint that a variance cannot be negative. This is the substantive difference between REML and the expected mean squares method, and it shows up directly on the report:

Under expected mean squares a component's moment estimate can come out negative, and zero is reported in its place with a footnote saying so.
Under REML nothing negative is ever computed. A reported zero means the constrained estimate reached the zero boundary, and the footnote says that instead.

The product itself points a reader from the first case to the second: the expected mean squares footnote suggests considering the REML method precisely because it constrains the components to be non-negative.

What is reported about the fit¶

Four numbers about the fit are reported, and they are reported only under this method:

Reported	What it is
the criterion, as \(-2\ell_R\)	the restricted log-likelihood at the fitted values, doubled and negated, so smaller is a better fit
the final scaled gradient	how close to flat the criterion was at the reported estimates; a small value means the fit settled
whether the solver converged	reported as a yes or no, and a fit that did not converge is refused rather than reported
the number of iterations used	a count, reported for the record

Under the other two methods these four are not reported at all.

The model REML cannot fit¶

REML cannot fit a model containing a term that involves more than three factors once nesting is counted. Nesting counts here because a nested term carries its parents with it, so a term that looks like two factors in the dialog can involve four once its nesting chain is followed.

When the declared model crosses that limit, the analysis uses the expected mean squares method instead, whether REML was chosen directly or reached through Automatic. The Model tab warns about this before the analysis is run, in these words:

A term in this model involves more than three factors once nesting is counted, so the REML method cannot fit it; if REML is selected or chosen automatically, the analysis will use expected mean squares instead.

The Estimation method: row of the report states the method that was actually used, so the substitution is visible afterwards as well as being warned about beforehand.

Which intervals it reports¶

REML reports confidence intervals on balanced and unbalanced data alike, which is the main way it differs from expected mean squares on the interval columns. Two gaps remain, and both are absences a reader will notice on the report:

An aggregate row (Total Gage R&R, Reproducibility total, Part to part total, Total Variation) reports no interval when the separately computed aggregate block is unavailable, or when one of its member components has no interval of its own.
Percent contribution and percent study variation carry no bounds under this method. Both would require a confidence interval on a ratio of two sums of components, which the method does not supply. The variance, standard deviation, study variation and percent of tolerance bounds are all reported as usual.

Where a bound is not reported the report prints a sentence naming the reason rather than leaving the cell blank.