Home / DOE / Analyze / Nominal Logistic Regression / Results

Regression Results for Nominal Logistic Regression¶

Run Regression fits every nominal output with Nominal Logistic Regression and writes the results to a regression sheet.

The report has one table per outcome, minus one¶

An output with \(r\) distinct values produces \(r - 1\) coefficient tables. One value is taken as the baseline and gets no table; each of the others gets a table describing it against that baseline.

So an output with the values Monday to Friday gives five distinct values, one baseline and four tables.

Every coefficient in every table is a comparison with the baseline. A coefficient in the Tuesday table is about Tuesday versus the baseline day, not about Tuesday in general. Two tables cannot be compared with each other directly, because they share a reference rather than describing each other.

The coefficient table¶

Coeff¶

The coefficient in the units of the model, normally coded, on the log odds scale. Specifically it is the change in the log odds of this outcome rather than the baseline, per unit of the term, holding the other terms fixed.

SE¶

How much the coefficient would move if the experiment were repeated.

Z¶

The coefficient divided by its standard error:

\[ Z = \frac{b_{jk}}{SE(b_{jk})} \]

P¶

The 2-tail p value from comparing Z against the normal distribution, coloured red at 0.05 or below, blue above 0.05 and up to and including 0.1, and black above 0.1.

Odds Ratio, 95% CI Low, 95% CI Up¶

The exponentiated coefficient, with exponentiated confidence bounds:

\[ \text{OR} = e^{b_{jk}} \qquad \text{CI} = e^{\text{CI}(b_{jk})} \]

For a nominal model the odds ratio compares this outcome against the baseline, so it answers "how do the odds of Tuesday rather than the baseline multiply per unit of this factor". An odds ratio of 1 means no effect on that comparison.

A confidence bound on a coefficient is the coefficient plus or minus a multiplier times its standard error. The nominal engine uses a fixed normal multiplier rather than a \(t\) value. It multiplies by the literal constant \(1.95996398454005\), the 95 percent two-sided standard normal value:

\[ b_j \pm 1.95996398454005 \times SE(b_j) \]

That number does not change with the number of runs or the number of terms. A bound recomputed with a \(t\) multiplier will therefore not reproduce the product's number.

The coefficient bounds themselves are not written to a sheet. What the sheet reports is the odds ratio bounds, which are these bounds exponentiated:

\[ \left(e^{\,b_j - 1.95996398454005\, SE(b_j)},\; e^{\,b_j + 1.95996398454005\, SE(b_j)}\right) \]

Because the exponential is not linear, the odds ratio bounds are not symmetric about the odds ratio, even though the coefficient bounds are symmetric about the coefficient.

Full details: Regression Diagnostics.

Odds are not probabilities

The gap is wide. Where 90 percent of one group and 20 percent of another show an outcome, one is 4.5 times as likely, but the odds ratio is 36. Both describe the same data.

For probabilities, use the prediction block rather than the odds ratios.

In Model¶

A checkbox per term. Clearing it and re-running the regression drops that term. A term is dropped from all the tables at once, since there is one model with one term list, not \(r - 1\) independent models.

Model fit statistics¶

These describe the whole model across all outcomes, so there is one set of them and not one per table.

LogLikelihood¶

The log of the likelihood the engine reports, which is the last entry of its log-likelihood history, the quantity the iteration maximised. Always negative, and meaningful only in comparison.

RSquaredU¶

McFadden's pseudo R squared, comparing the fitted model with a constant-only model:

\[ R^2_U = 1 - \frac{\ln L_{\text{full}}}{\ln L_{\text{constant only}}} \]

Larger is stronger. It is not the proportion of variation explained, and its values should not be compared with a least squares R squared.

AIC and BIC¶

Both trade fit against complexity, and lower is better for both:

\[ \text{AIC} = -2\ln L + 2k \qquad \text{BIC} = -2\ln L + k \ln n \]

with \(k\) parameters and \(n\) observations. For a nominal model \(k\) grows with the number of outcomes, since each non-baseline outcome has its own set of coefficients, so these criteria penalise a many-outcome model heavily.

This is the plain AIC, not the corrected AICc. Both are written to the sheet with no threshold applied.

G Stat, G df, G P-Value¶

G tests whether the terms together explain anything:

\[ G = -2\left(\ln L_{\text{constant only}} - \ln L_{\text{full}}\right) \]

against a chi-square distribution on G df degrees of freedom. A G P-Value below 0.05 says not all the slopes are zero, so the model predicts better than the constant alone.

Goodness of Fit Test¶

One block heading, Goodness of Fit Test, over two rows labelled Pearson and Deviance, with the columns Chi-Sq, DF and P.

Pearson¶

The nominal engine computes no residual of any kind. It groups the runs into distinct covariate patterns and builds a table of observed and expected counts, one row per pattern and one column per outcome. The Pearson statistic is the ordinary chi-square total over that table, each cell's squared difference between its observed and expected count divided by its expected count. A small p value is the bad outcome here: below 0.05 indicates significant lack of fit.

G small is good, goodness of fit small is bad. Keeping those straight is the main difficulty in reading a logistic report.

The Pearson statistic compares observed counts with the counts the fitted model expects, and it is referred to a chi-square distribution. The nominal engine computes the statistic and chooses the degrees of freedom itself, and asks the regression library for the upper tail probability of the chi-square distribution at that statistic on those degrees of freedom.

It is not built from residuals at all. The nominal engine first groups the runs into \(J\) distinct covariate patterns, which it does only at this point and not for the fit itself, and builds a table of \(J\) rows by \(K\) columns holding the observed count \(O_{ik}\) of each outcome at each pattern and the expected count \(E_{ik}\), which is the pattern's total weight times the fitted probability of that outcome. The statistic is the ordinary chi-square over that whole table, the reference level's column included:

\[ \chi^2_P = \sum_{i=1}^{J} \sum_{k=1}^{K} \frac{(O_{ik} - E_{ik})^2}{E_{ik}} \]

on \((K-1)J - (K-1)p\) degrees of freedom, which is the count of free cells less the \((K-1)p\) coefficients the model estimates. A cell whose value is not a number, which is a cell whose observed and expected counts are both zero, is left out of the sum. When the degrees of freedom come out at zero or below, no p value is reported.

How much replication a design has therefore matters, because \(J\) enters the degrees of freedom directly. A design in which no set of factor settings is repeated has \(J\) equal to its number of runs.

Deviance¶

The same question asked from the same table of observed and expected counts, by a different formula: twice the sum, over every cell, of the observed count times the natural log of the observed count divided by the expected count. The report gives the statistic, its degrees of freedom and its p value, and applies no threshold to any of them.

The deviance statistic asks the same question as the Pearson statistic by a different route, comparing the fitted model's log-likelihood with that of a model that reproduces every observed proportion exactly. It is referred to the same chi-square distribution on the same degrees of freedom as the Pearson statistic, and again the nominal engine forms the statistic and chooses the degrees of freedom while the regression library supplies the upper tail probability.

It is formed directly from the same table of observed and expected counts the nominal Pearson statistic uses, with no residual in between:

\[ \chi^2_D = 2 \sum_{i=1}^{J} \sum_{k=1}^{K} O_{ik} \ln \frac{O_{ik}}{E_{ik}} \]

on \((K-1)J - (K-1)p\) degrees of freedom. A cell whose term is not a number, which is any cell holding an observed count of zero, is left out of the sum.

Full details: Logistic Estimation.

It needs replication. When the number of distinct covariate patterns approaches the number of observations the test becomes unreliable, since each pattern has too few observations to judge fit. Several runs per covariate pattern make it useful. A nominal response reaches that limit sooner than a binary one, because the observations are spread across more outcomes.

The predicted probability table¶

Two columns and one row per outcome, the baseline included: the outcome's name, then its probability. The column of probabilities totals exactly 1, reading down the rows. The baseline is the last row, and its cell is not a subtraction from the others: it holds the reference-level formula directly, one divided by one plus the sum of the other outcomes' exponentiated linear predictors, which is what guarantees the total.

For a nominal output with \(L\) non-reference levels there is one sum per level, \(\eta_k = \sum_j c_j b_{jk}\), and the probability of level \(k\) is

\[ p_k = \frac{e^{\eta_k}}{1 + \sum_{l=1}^{L} e^{\eta_l}} \]

The \(1\) in the denominator is the reference level, whose \(e^{\eta}\) is \(1\) by construction. So the reference level's own probability is

\[ p_{\text{ref}} = \frac{1}{1 + \sum_{l=1}^{L} e^{\eta_l}} \]

and the probabilities across all levels sum to \(1\).

Full details: Prediction Equation.

This table, and the prediction block at the top of the report, are the places to read probabilities. The coefficient tables are on the log odds scale and the odds ratio columns are on the odds scale; neither is a probability.

If the model did not converge¶

A warning appears in red at the top of the table and the coefficients are from the last iteration rather than a converged fit, which makes every statistic here unreliable. With a nominal response the usual cause is an outcome that occurs in too few runs. See Nominal Logistic Regression.

References¶

Agresti, A. (2013). Categorical Data Analysis, 3rd edition. Wiley.
Hosmer, D. W., Lemeshow, S., and Sturdivant, R. X. (2013). Applied Logistic Regression, 3rd edition. Wiley.
McFadden, D. (1974). Conditional Logit Analysis of Qualitative Choice Behavior. In Frontiers in Econometrics, Academic Press.