Home / DOE / Additional Information / Binomial Logistic Residuals
Binomial Logistic Residuals¶
A binary logistic regression has no residual in the least squares sense, because the response is an outcome rather than a number and the prediction is a probability. It produces a different set of per-run diagnostics instead, and they are what the residual plots draw for a logistic output.
The axis choices for a logistic output¶
Residual Plots offers a different list for a logistic output than for a quantitative one: Predicted, Leverage, Delta Chi-Squared, Delta Deviance, Delta Beta and Delta Beta Standardized.
| Quantity | What it measures |
|---|---|
| Predicted | the fitted probability |
| Leverage | how unusual the run's factor settings are |
| Delta Chi-Squared | how much the Pearson chi-squared statistic changes when the runs at this covariate pattern are removed |
| Delta Deviance | how much the deviance changes when the runs at this covariate pattern are removed |
| Delta Beta | how much the coefficients move when the runs at this covariate pattern are removed |
| Delta Beta Standardized | the same, scaled, so the values are comparable across patterns |
All four of the delta measures are per covariate pattern, not per row. They ask what happens when every run at one set of factor settings is removed, so on a replicated design one deletion removes several rows at once. See Covariate Pattern.
A large Delta Chi-Squared or Delta Deviance marks a pattern the model fits badly. A large Delta Beta marks a pattern the model depends on, which is not the same thing: a pattern can fit well and still be the one holding a coefficient up.
The regression also produces a deviance residual per pattern, the per-pattern contribution to the deviance. The two deletion measures are not built from the same residual. Delta Chi-Squared is the squared Pearson residual divided by one minus the leverage. Delta Deviance is the squared deviance residual plus a Pearson and leverage term.
Leverage and the two residuals the delta measures are built from are defined as follows.
A binary logistic model has its own leverage, and Quantum XL computes it rather than taking it from the library. Its rows are not runs. The binary engine collapses identical design rows before it fits, so this quantity, like every other quantity of that fit, is one value per covariate pattern. It is the diagonal of the weighted hat matrix
where \(V\) is the diagonal matrix whose entry for pattern \(i\) is \(m_i \hat{\pi}_i (1 - \hat{\pi}_i)\): the number of trials at that pattern, times the fitted probability, times one minus the fitted probability. Where an entry of \(V\) is not positive, the corresponding entry of \(V^{1/2}\) is left at zero rather than having a square root taken.
A logistic fit has no residual in the least squares sense, because the observation is a count of events and the prediction is a probability. The Pearson residual is one of the two standard replacements. It is the difference between the events observed at a covariate pattern and the events the model expects there, divided by the standard deviation of that count under the model:
The denominator is the binomial standard deviation at pattern \(i\), so the residual is in standard deviations of its own count and residuals from patterns of different sizes are comparable with each other. Squaring it and adding over the patterns gives the Pearson goodness of fit statistic.
This is a binary logistic quantity only. The nominal engine produces no Pearson residual, and no nominal output writes one.
The engine also forms a standardized version, which divides by the leverage factor in the same way the studentized residual of a least squares fit does:
Neither form is written to a sheet or drawn on a chart on its own. They are the pieces the reported per-pattern deletion measures are built from.
The deviance residual is the other standard replacement for a least squares residual. It is the signed square root of one covariate pattern's contribution to the deviance, so squaring it and adding over the patterns gives the deviance goodness of fit statistic.
For a pattern where the events observed are neither none nor all of the trials it is
taking the negative sign when \(y_i - m_i \hat{\pi}_i\) is negative, so the residual is negative exactly when the model expected more events than were observed. If the quantity under the root is not strictly positive, which can only happen through rounding at a pattern the model fits almost exactly, the engine reports \(0.0000001\) in place of the root before applying the sign.
The two ends need their own expressions, because a logarithm in the form above would be taken of zero. Where a pattern had no events at all,
and where every trial was an event,
The sign is fixed by which end it is rather than being derived: a pattern with no events can only sit below its expectation, and a pattern of nothing but events can only sit above it.
This is a binary logistic quantity only. The nominal engine produces no deviance residual, and no nominal output writes one. Its deviance statistic is formed straight from observed and expected counts instead.
Full details: Regression Diagnostics for leverage, Logistic Estimation for the two residuals.
The two goodness-of-fit statistics they come from¶
The binary logistic report carries a Pearson chi-square and a deviance chi-square, each with its degrees of freedom and p value. The delta measures are the per-pattern versions of those two totals, which is why they come in pairs.
Both are computed and reported with their degrees of freedom and p values whatever the number of runs per pattern. The software applies no condition and no threshold to either. See Regression Results for Binary Logistic.
When the fit did not converge¶
The residuals are worth nothing if the fit failed, and the report says so plainly rather than leaving you to notice. It is one message, built from sentences, not a list of alternatives:
Model failed to converge. The results of the model are in question. Detected complete separation of data points. Results are based on last iteration; validity of the model is in question.
The middle sentence appears only when separation was detected, and reads Detected quasicomplete separation of data points. for the partial case.
The most common cause is separation, where some combination of factors predicts the outcome perfectly. Quantum XL detects it and distinguishes complete from partial separation. See Binary Logistic Regression.
What a logistic output does not have¶
No S-Hat model, no prediction interval, no Cp or Cpk, and no lack-of-fit block of the kind a quantitative regression has.
Observed vs Predicted is not available for a logistic output either. It takes a quantitative Y-Hat or S-Hat model only, and binary and nominal outputs do not appear in its Outputs tree at all. Residual Plots does accept a binary output, with the logistic axis list above, but not a nominal one.