Skip to content

Home / DOE / Analyze / Binary Logistic Regression / Math Details

Math Details

The mathematics behind the fit. What the engine is for and when Run Regression chooses it are on Binary Logistic Regression, and what governs a run is on Options.

Why not least squares

Least squares would happily predict a probability of 1.4 or of \(-0.2\), which are not probabilities. Logistic regression fits on a scale where that cannot happen, by modelling the log odds of the outcome as linear in the factors and then transforming back:

\[ \ln\!\left(\frac{\pi}{1 - \pi}\right) = \sum_j c_j b_j \qquad \Longleftrightarrow \qquad \pi = \frac{1}{1 + e^{-\sum_j c_j b_j}} \]

where \(\pi\) is the predicted probability, \(c_j\) the coded value of term \(j\) and \(b_j\) its coefficient. The right-hand form is an S curve that approaches 0 and 1 without reaching either, so every prediction is a valid probability.

How the coefficients are found

There is no closed form. The coefficients are found by iteration, maximising the log likelihood:

\[ \ln L = \sum_i \left[ y_i \ln \pi_i + (1 - y_i)\ln(1 - \pi_i) \right] \]

Quantum XL uses iteratively reweighted least squares, a form of Newton's method: it starts from a guess, computes the likelihood, takes a step, and repeats until the coefficients stop changing. The log likelihood at every iteration is kept, so the report can show how the fit converged.

The iteration limits are fixed and there is no control for them. The fit is allowed 20 iterations, with a convergence criterion of 1e-6 and up to 7 half steps. None of the three appears on any dialog or on the Options page.

See Also