Skip to content

Home / DOE / Analyze / Binary Logistic Regression / Math Details

Math Details

The mathematics behind the fit. What the engine is for and when Run Regression chooses it are on Binary Logistic Regression, and what governs a run is on Options.

Why not least squares

Least squares would happily predict a probability of 1.4 or of \(-0.2\), which are not probabilities. Logistic regression fits on a scale where that cannot happen, by modelling the log odds of the outcome as linear in the factors and then transforming back:

\[ \ln\!\left(\frac{\pi}{1 - \pi}\right) = \sum_j c_j b_j \qquad \Longleftrightarrow \qquad \pi = \frac{1}{1 + e^{-\sum_j c_j b_j}} \]

where \(\pi\) is the predicted probability, \(c_j\) the coded value of term \(j\) and \(b_j\) its coefficient. The right-hand form is an S curve that approaches 0 and 1 without reaching either, so every prediction is a valid probability.

How the coefficients are found

There is no closed form. The coefficients are found by iteration, maximising the log likelihood:

\[ \ln L = \sum_i \left[ y_i \ln \pi_i + (1 - y_i)\ln(1 - \pi_i) \right] \]

How the fit is computed is not documented. What matters for reading the report is what follows.

The log likelihood at every iteration is kept, so the report can show how the fit progressed, and the convergence line above the coefficient table says whether it settled. See Regression Results for Binary Logistic.

The limits governing the iteration are fixed and there is no control for them, on any dialog or on the Options page. Two of them are visible in their effects rather than as settings: a fit that runs out of them without settling is reported as a failure to converge, and a fit that settles only after being held back is reported as Model Converged (Half stepping limit reached). Both cases, and what to do about them, are on Binary Logistic Regression.

See Also