Home / Shared Math Details / Shared Estimators and Tests / Weighted Least Squares (QR)

Weighted Least Squares (QR)

This page defines the weighted least-squares fit, solved by QR decomposition, that Quantum XL uses to fit regression models and trend lines.

Notation

Term	Description
\(\mathbf{x}_i\)	the model terms (row of the design matrix) at observation \(i\)
\(X\)	design matrix whose rows are the \(\mathbf{x}_i\)
\(y_i\)	response value at observation \(i\)
\(w_i\)	weight (frequency) of observation \(i\)
\(W\)	diagonal matrix of the weights \(w_i\)
\(\boldsymbol\beta\)	fitted coefficients
\(N\)	total weight \(\sum_i w_i\)
\(p\)	number of parameters (columns of \(X\))

Model

Quantum XL fits \(y \approx X\boldsymbol\beta\) by minimizing the weighted sum of squared residuals:

\[ \min_{\boldsymbol\beta}\ \sum_{i} w_i \left(y_i - \mathbf{x}_i^{\top}\boldsymbol\beta\right)^2 \]

The design-matrix row depends on the model: \([\,1,\ x\,]\) for a line, \([\,1,\ x,\ x^2,\ \dots,\ x^d\,]\) for a degree-\(d\) polynomial, and \([\,1,\ \ln x\,]\) for a logarithmic fit.

QR solution

Quantum XL scales each row by \(\sqrt{w_i}\), forming the weighted system \(X^{*} = \sqrt{W}\,X\) and \(y^{*} = \sqrt{W}\,y\) (so that \(X^{*\top}X^{*} = X^{\top}WX\)), then solves it by Householder QR decomposition:

\[ X^{*} = QR \qquad\Longrightarrow\qquad \hat{\boldsymbol\beta} = R^{-1}Q^{\top}y^{*} \]

The solution is obtained by back-substitution on the first \(p\) rows. A near-singular system (any \(|R_{ii}| < 10^{-14}\)) is rejected rather than solved.

Shared fit statistics

The same weighted fit reports:

\[ R^2 = 1 - \frac{\sum_i w_i\left(y_i - \hat{y}_i\right)^2}{\sum_i w_i\left(y_i - \bar{y}\right)^2} \qquad R^2_{\text{adj}} = 1 - \left(1 - R^2\right)\frac{N - 1}{N - p} \]

\[ s_e = \sqrt{\frac{\sum_i w_i\left(y_i - \hat{y}_i\right)^2}{N - p}} \]

where \(\hat{y}_i\) is the fitted value, \(\bar{y}\) the weighted mean of the response, and \(N = \sum_i w_i\).

Used by

• Scatter Plot: the regression fit. The Scatter Plot page also documents the extras built on this solve (coefficient t-tests, the overall F-test, and VIF).
• Time Series: the trend fit, which uses weights of 1 for valid points and 0 to skip missing points.

See Also

• Student t Distribution
• Shared Math Details

References

1. Golub, G. H., and Van Loan, C. F. (2013). Matrix Computations, 4th ed. Johns Hopkins University Press.
2. Draper, N. R., and Smith, H. (1998). Applied Regression Analysis, 3rd ed. Wiley.