Skip to content

Home / DOE / Analyze / Nominal Logistic Regression / Math Details

Math Details

The mathematics behind the fit. What the engine is for and when Run Regression chooses it are on Nominal Logistic Regression, and what governs a run is on Options.

How it models several outcomes

With \(r\) outcomes, one is taken as the baseline and the model describes each of the other \(r - 1\) against it. So there are \(r - 1\) sets of coefficients, each a comparison with the baseline rather than a description of an outcome in isolation.

For outcome \(j\) against baseline outcome \(r\):

\[ \ln\!\left(\frac{\pi_j}{\pi_r}\right) = \sum_k c_k b_{jk} \]

A coefficient is therefore the change in the log odds of landing in outcome \(j\) rather than the baseline, per unit of that term, with the other terms held fixed. Every coefficient in this report is a comparison with the baseline, which is the single most important thing to keep in mind when reading it. A coefficient does not say that outcome \(j\) becomes more likely in absolute terms, only that it becomes more likely relative to the baseline.

Transforming back, for a non-baseline outcome:

\[ \pi_j = \frac{e^{\sum_k c_k b_{jk}}}{1 + \sum_{l=1}^{r-1} e^{\sum_k c_k b_{lk}}} \]

and for the baseline itself:

\[ \pi_r = \frac{1}{1 + \sum_{l=1}^{r-1} e^{\sum_k c_k b_{lk}}} \]

The \(1\) in the denominator is the baseline's own exponent, which is 1 by construction. The probabilities across all \(r\) outcomes sum to 1.

With a design matrix of \(p\) columns the model has \((r - 1) \times p\) free parameters, so the parameter count grows with the number of outcomes. This is why a nominal response needs more runs than a quantitative one to support the same set of terms.

How the coefficients are found

By Newton's method, maximising the log likelihood, with no closed form.

Three limits govern the iteration, and all three are fixed in the engine. There is no control for any of them, on any dialog or on the Options page:

  • 20 iterations.
  • A convergence criterion of 1e-6, which sets how small a change counts as settled.
  • Up to 7 half steps. Newton's method sometimes proposes a step that overshoots and makes the fit worse. When that happens the step is halved and retried, and this limits how many times.

See Also