< List of probability distributions < Bivariate normal distribution
Contents:
- Bivariate normal distribution
- Multivariate normal distribution
- Bravais distribution
- Variance ratio distribution
1. Bivariate normal distribution
A bivariate normal distribution is made up of two independent random variables instead of the single one found with the “regular” normal distribution. The two variables in a bivariate normal are both normally distributed and they have a normal distribution when both are added together.
The bivariate normal distribution can be visualized as a surface — a two-dimensional object that can be embedded in three-dimensional space[1]:
Francis Galton (1822-1911) was one of the first mathematicians to study the bivariate normal distribution in depth, during his study on the heights of parents and their adult children. Bravais, Gauss, Laplace, and Plana also studied the distribution in the early nineteenth century [2].
The bivariate distribution can be described in many different ways and as such, there isn’t a unified agreement for a succinct definition. Some of the more common ways to characterize it include:
- Random variables X & Y are bivariate normal if aX + bY has a normal distribution for all a,b ∈ ℝ.
- X and Y are jointly normal if they can be expressed as X = aU + bV, and Y = cU + dV [3]
- If a and b are non-zero constants, aX + bY has a normal distribution [4]
- If X – aY and Y are independent and if Y – bx and X are independent for all a,b (such that ab ≠ 0 or 1), then (X,Y) has a normal distribution [5].
There are dozens of different variants of these definitions. That’s one reason why the bivariate normal is usually defined in terms of its PDF.
PDF of the bivariate normal distribution.
The bivariate normal distribution can be defined as the probability density function (PDF) of two variables X and Y that are linear functions of the same independent normal random variables [6]:
Where:
- μ = mean
- σ = standard deviation.
And:
And:
For some excellent gifs that show what happens when a few of these parameters are changed, check out Brad Hartlaub’s page at Kenyon college. This one shows what happens when μ1 is changed:
2. Multivariate normal (bivariate normal distribution special case)
The multivariate normal distribution has two or more random variables — so the bivariate normal distribution is a special case of the multivariate normal distribution. That said, while the bivariate normal can be easily visualized (as demonstrated in the gif above), more than two variables poses problems with visualization. Thus, the multinormal can be difficult to wrap your head around — at least, visually. That said, if you’re familiar with matrix algebra it’s fairly easy to work with, and is one of the most important distributions in multivariate statistics.
The multivariate normal distribution is most often described by its joint density function. A multivariate normal p x 1 random vector X, with population mean vector μ and population variance-covariance matrix σ, will have the following joint density function:
Where:
- |Σ| = determinant of the variance-covariance matrix Σ
- Σ-1 = inverse of the variance-covariance matrix Σ.
3. Bravais distribution (another name for bivariate normal distribution)
The Bravais distribution is another name for the bivariate normal distribution, (also sometimes called the bivariate Gaussian or Bivariate Laplace–Gauss distribution).
Teugels & Sundt [7] lists the Bravais distribution as the probability density function of the bivariate normal random vector,
X = (X1, X2)T
which is
Haight [8] published a simple formula for the bivariate normal:
and refers the reader to a version published in an article in the 1958 Volume 19 of Skandinavisk Aktuarietidskrift. The unnamed author (I was unable to locate a copy of the journal to find the authors name) probably named this distribution after Bravais [9], who developed and published his study of normal frequency distributions in two and more variables.
4. Variance Ratio Distribution (historical name for the bivariate normal distribution)
The “variance ratio distribution” refers to the distribution of the ratio of variances of two samples drawn from a normal bivariate correlated population. Today, we call this the bivariate normal distribution.
The Fisher-Snedicor F Distribution is sometimes called the “Variance Ratio” distribution because it is the distribution of the ratio of two independent variance estimates (S12/S22) [10]. However, this is quite different from the variance ratio distribution in historical literature.
Variance Ratio Distribution History
Haight’s entry in [8] provides the formula:
The notation [e]2:65 refers to a 1935 article by Bose [11], titled On the Distribution of the Ratio of Variances of Two Samples Drawn from a Given Normal Bivariate Correlated Population and published in Sankhya, volume 2, 1935. The author, providing a solution to the question of “the distribution function of the ratio of variances obtained from two independent samples,” refers to Fisher’s earlier work [12]
Why the complicated (rarely seen in modern times) formula? The answer is the advent of the computer. Before the computer age (c. 1960s), mathematicians had to refer to tables for the variance ratio distribution F and – sometimes – equations with “great computational difficulty”. In the early days of computers, the distribution also required the “use of excessive amounts of computer time” [13].
Of course, nowadays, we just open our statistics software program and run an algorithm. That’s why you’ll rarely see the actual formula for the variance ratio distribution.
References
[1] Washington U. Contents / Lesson 20: Pattern Classification Tutorial.
[2] Balakrishnan,N. & Lai, C. (2009) Continuous Bivariate Distributions.
[3] Bertsekas & Tsitsiklis (2002). Introduction to Probability (1st ed.).
[4] Johnson & Kotz. (1972) Distributions in Statistics: Continuous Multivariate Distributions.
[5] Rao, C. (1975). Some Problems in the Characterization of the Multivariate Normal Distribution.
[6] Wolfram Mathworld. BND. Retrieved August 4, 2017 from: http://mathworld.wolfram.com/BivariateNormalDistribution.html
[7] Teugels & Sundt. (2004) Encyclopedia of Actuarial Science. Wiley.
[8] Haight, F. (1958). Index to the Distributions of Mathematical Statistics. National Bureau of Standards Report.
[9] Bravais, August, 1846: “Analyse MathCmatique sur les probabilities des erreurs de situation d’un point,” Memoirs Presentes par Divers Savants, 2nd Series, Vol. 9, Institut de France, Acadamie des Sciences, Paris, France, pp. 255-332
[10] Jolicoeur P. (1999) The distribution of the variance ratio, F = S12/S22. In: Introduction to Biometry. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-4777-8_9(KENDALL, M. G. The Advanced Theory of Statistics , Volume 1 , London: Charles Griffin and Co., 1945.
[11] Bose, S., & Mahalanobis, P. C. (1935). On the Distribution of the Ratio of Variances of Two Samples Drawn from a Given Normal Bivariate Correlated Population. Sankhyā: The Indian Journal of Statistics (1933-1960), 2(1), 65–72. http://www.jstor.org/stable/40383720
[12] Fisher, R. (1924). On a distribution yielding the error function of well known statistics. Proceedings of the International Mathematical Congress, Toronto, 05-813.
[13] Box, M. & Box, R. (1969). Computation of the variance ratio distribution. Online: https://academic.oup.com/comjnl/article/12/3/277/363429