Home / Monte Carlo / Model Building / Correlations / Math Details

Math Details¶

What each family means for the numbers you get, what each parameter is allowed to be, why a set of correlations can be refused, and what the nearest valid matrix is guaranteed to satisfy. What the tool is for is on Correlations, and the controls are on Options.

What this page does not cover

How Quantum XL constructs the correlated values, and how it computes the nearest valid matrix, are not documented. This page states what each control means, what each number you enter has to satisfy, and what the results are guaranteed to have, which is what a reader needs in order to read the dialog and trust its output.

Notation¶

Term	Description
\(k\)	the number of inputs in the correlation
\(\rho\)	one off-diagonal entry of a correlation matrix
\(\theta\)	the strength parameter of the Clayton or Gumbel family
\(\nu\)	the Student-t degrees of freedom
\(R\)	the \(k\) by \(k\) correlation matrix

What a correlation changes and what it leaves alone¶

A correlation changes which values of the inputs turn up together. It does not change what values any one input can take, or how often it takes them.

That is a guarantee rather than an approximation. Each input's own distribution is untouched, so truncation, custom distributions and empirical data all behave exactly as they do on an uncorrelated input, and the marked distribution you see in Mark Input is the distribution the input still has.

A rule that amounts to no dependence reproduces an uncorrelated run exactly, value for value. Two settings do that: an identity matrix, meaning every off-diagonal entry is 0, and a Gumbel theta of exactly 1.

The four families¶

The families differ in where in the range the dependence is concentrated, which is what decides whether extreme values of two inputs tend to occur on the same trial.

Family	Where the dependence sits	Both inputs extreme together
Gaussian	spread evenly over the whole range	no more likely than the middle of the range suggests
Student-t	spread evenly, with both tails strengthened	more likely in both tails, and more so as \(\nu\) falls
Clayton	at the low end	low values occur together more than high values do
Gumbel	at the high end	high values occur together more than low values do

The families are the Gaussian, Student-t, Clayton and Gumbel-Hougaard copulas of the literature; see the references.

What each parameter is allowed to be¶

Every rule below is enforced when Save correlation is pressed, and the sentence quoted is the refusal you get.

A correlation matrix, Gaussian and Student-t¶

\[ \rho_{ii} = 1, \qquad \rho_{ij} = \rho_{ji}, \qquad -1 \le \rho_{ij} \le 1 \]

The diagonal is fixed at 1 by the grid, so only a matrix read from a worksheet can break the first rule, and the grid mirrors what you type, so only a worksheet can break the second.

Correlation group 1: The correlation between inputs 1 and 2 must be a number between -1 and 1.

A correlation of exactly \(+1\) or \(-1\) is allowed. Only a number outside \(-1\) to \(1\) is refused.

Degrees of freedom, Student-t only¶

\[ \nu \ge 1, \qquad \nu \in \mathbb{Z} \]

Degrees of freedom is not a whole number.

Correlation group 1: Degrees of freedom must be a whole number of at least 1 (got 0).

Theta, Clayton¶

\[ \theta > 0, \qquad \theta \text{ finite} \]

Correlation group 1: Clayton theta must be greater than zero.

A larger \(\theta\) ties the inputs more tightly. As \(\theta\) approaches 0 they approach independence, but 0 itself is refused rather than treated as independence.

Theta, Gumbel¶

\[ \theta \ge 1, \qquad \theta \text{ finite} \]

Correlation group 1: Gumbel theta must be a finite number of at least 1 (1 means no correlation).

A larger \(\theta\) ties the inputs more tightly. \(\theta = 1\) is exactly independence and is accepted.

Reflected, Clayton and Gumbel only¶

The dialog describes it as:

Reflected reverses the direction of an input's relationship.

A theta family has no matrix to carry a minus sign per pair, so the direction is set per input instead.

The number you type is Spearman's, not Pearson's¶

The grid holds Spearman's rank correlation coefficient \(\rho_S\), the correlation of the ranks of the two inputs' values. It does not hold Pearson's product-moment correlation, and there is no setting anywhere in the dialog that makes it hold one.

Why a set of correlations can be impossible¶

A correlation matrix cannot hold arbitrary numbers, because the entries constrain each other. If A and B move together strongly and B and C move together strongly, then A and C cannot move strongly in opposition, whatever number you type for that pair.

The exact condition is that \(R\) must be positive semi-definite: every eigenvalue at or above zero, equivalently

\[ \mathbf{x}^{\prime} R \, \mathbf{x} \ge 0 \quad \text{for every } \mathbf{x} \]

A matrix that fails this describes relationships no data could have, and it is refused:

Correlation group 1: The correlations are mathematically inconsistent in that the matrix is not positive semi-definite.

Semi-definite, not definite. A matrix with a zero eigenvalue is accepted. That is what a correlation of exactly \(\pm 1\) produces, and so does a genuinely real case such as three inputs all pairwise at exactly \(-0.5\).

What the nearest valid matrix guarantees¶

When a matrix family's entries fail that condition and a consistent replacement can be found, the refusal is followed by the offer described on Options. What comes back is guaranteed to have two properties.

It is consistent. It satisfies the positive semi-definite condition above, with a small margin rather than sitting exactly on the boundary, so it is safely inside the set of usable matrices rather than on its edge.

It is a correlation matrix. Every diagonal entry is exactly 1 and every off-diagonal entry lies within \([-1, 1]\).

The nearest correlation matrix problem is the subject of Higham (2002), listed in the references.

Three further points:

The numbers you get back are in the units you typed: rank correlations for a matrix headed Rank correlation matrix (Spearman).
The replacement is checked by the same rule that refused the original. If the check does not pass, or no replacement can be found, you are given the original refusal and nothing else, so you never see a matrix that would fail on the next press.
Nothing is saved by accepting. The numbers go into the grid for you to look at, and Save correlation is left for you to press.

Why a discrete input falls short of the requested correlation¶

When any ticked input has a discrete distribution, the dialog says so:

Name is discrete. A correlated discrete input will not reach the requested correlation.

This is a warning, not a refusal: the correlation still saves.

References¶

Copulas, and the separation of dependence from the marginals

Sklar, A. (1959). Fonctions de repartition a n dimensions et leurs marges. Publications de l'Institut de Statistique de l'Universite de Paris, 8, 229-231.
Nelsen, R. B. (2006). An Introduction to Copulas, 2nd edition. Springer.
Joe, H. (2014). Dependence Modeling with Copulas. CRC Press.
Genest, C., & MacKay, J. (1986). The joy of copulas: bivariate distributions with uniform marginals. The American Statistician, 40(4), 280-283.

Rank correlation

Kruskal, W. H. (1958). Ordinal measures of association. Journal of the American Statistical Association, 53(284), 814-861.

The nearest correlation matrix

Higham, N. J. (2002). Computing the nearest correlation matrix: a problem from finance. IMA Journal of Numerical Analysis, 22(3), 329-343.

The individual families

Demarta, S., & McNeil, A. J. (2005). The t copula and related copulas. International Statistical Review, 73(1), 111-129.
Clayton, D. G. (1978). A model for association in bivariate life tables and its application in epidemiological studies of familial tendency in chronic disease incidence. Biometrika, 65(1), 141-151.
Gumbel, E. J. (1960). Bivariate exponential distributions. Journal of the American Statistical Association, 55(292), 698-707.