< List of probability distributions
The Wishart distribution (also called the Wishart ensemble or Wishart–Laguerre ensemble) is a multivariate continuous distribution which generalizes the gamma distribution. It is named after the Scottish statistician John Wishart, who first formulated the distribution in 1928 [1].
The Wishart distribution comprises a group of probability distributions defined for symmetric, nonnegative-definite random matrices (i.e., matrix-valued random variables). Within random matrix theory, the collection of Wishart matrices is referred to as the Wishart ensemble. These distributions play a crucial role in estimating covariance matrices in multivariate statistics. In the realm of Bayesian statistics, the Wishart distribution serves as the conjugate prior for the inverse covariance matrix of a multivariate-normal random vector [2].
Definition of the Wishart distribution
The Wishart distribution is considered to be one of the more complex distributions in statistics, partly due to its multiple definitions. The simplest definition describes it as the multivariate analog of a chi-squared distribution when the degrees of freedom are non-integers. When the degrees of freedom are integers, it acts as the multivariate counterpart of the gamma distribution.
In the context of inferential statistics, the Wishart distribution is also defined as the distribution of the sample covariance matrix, which is generated from a random sample taken from a multivariate normal distribution. Typically, it is denoted as Wp(Σ, n), where Σ represents the covariance matrix and n signifies the number of degrees of freedom.
Kollo and Von Rosen [3] provide a definition of the Wishart distribution in terms of a matrix distribution:
“The matrix W : p x p is said to be a Wishart distribution if and only if W = XX’ for some matrix X, where X ~ Np,n(M,Σ,I), and Σ is positive definite. If M = 0, the distribution is a central Wishart (W ~ Wp(Σ,n)); if M ≠ 0, the distribution is non central (Wp(Σ,n,Δ)), where Δ = MM’)“
Wishart definition properties
The probability density function (PDF) for the Wishart distribution only exists when the sample size is greater than the number of variables in the model [4]:
Where:
- n = degrees of freedom,
- |X| = determinant of the matrix
- Γp( ) is the multivariate gamma function.
Other important properties include:
- Mean with parameters ν and Σ = Σ.
- Variance with parameters ν and Σ = νΣ.
Advantages and disadvantages
One of the Wishart Distribution’s main uses is to approximate a covariance matrix, as long as the underlying distribution is normal. In addition, many different sum of squares and product matrices also have Wishart distributions. The distribution can also be used to test for statistical significance, and to make predictions about future data. Despite its many applications, it does come with some drawbacks, such as:
- It can be computationally expensive to work with, especially for large covariance matrices.
- The distribution is sensitive to the choice of hyperparameters. This can make it difficult to choose the right hyperparameters for a particular application; if you choose the wrong hyperparameters then the distribution may not be a good fit for the data.
- The Wishart distribution is not always the best choice for modeling covariance matrices. Other distributions, such as the inverse Wishart distribution, may be more appropriate in some cases. The inverse Wishart distribution is often used in Bayesian analysis, where it is used to model the prior distribution of the covariance matrix.
History of the Wishart distribution
The Wishart distribution was initially presented by John Wishart in his 1928 paper titled The generalised product moment distribution in samples from a normal multivariate population. Wishart, a Scottish statistician, worked at the University of Edinburgh and was focused on the challenge of estimating the covariance matrix for a multivariate normal distribution.
Wishart demonstrated that the sample covariance matrix derived from a sample of a multivariate normal distribution follows a Wishart distribution. This discovery marked a significant advancement in the field of multivariate statistics.
Besides Wishart, several other prominent figures have contributed to the growth and development of the Wishart distribution. These individuals include:
- Harold Hotelling: An American statistician based at Columbia University, Hotelling was a pioneer in multivariate statistics and made substantial contributions to the evolution of the Wishart distribution.
- Charles Stein: Stein, an American statistician working at Stanford University, was a top expert in statistical inference and played a crucial role in refining the Wishart distribution.
- George Casella: As an American statistician at the University of California, Berkeley, Casella is a leading authority in Bayesian statistics and has significantly advanced the development of the Wishart distribution.
References
[1] Wishart, J. (1928). “The generalised product moment distribution in samples from a normal multivariate population”. Biometrika. 20A (1–2): 32–52. doi:10.1093/biomet/20A.1-2.32. JFM 54.0565.02. JSTOR 2331939.
[2] Koop, Gary; Korobilis, Dimitris (2010). “Bayesian Multivariate Time Series Methods for Empirical Macroeconomics”. Foundations and Trends in Econometrics. 3 (4): 267–358. doi:10.1561/0800000013. ^
[3] Kollo, T. and Von Rosen, D. (2005). Advanced Multivariate Statistics with Matrices. Springer, Dordrecht.
[4] Abell, M. et. al. (1999). Statistics with Mathematica. Academic Press.