deepdream of
          a sidewalk
Show Question
Math and science::INF ML AI

Multivariate Gaussian distribution

This card derives the general multivariate normal distribution from the standard multivariate normal distribution.

Standard multivariate Gaussian/normal distribution

Let (Ω,F,P) be a probability space. Let X:ΩRK be a continuous random vector. X is said to have a standard multivariate normal distribution iff its joint probability density function is:

fX(x)=(12π)KexTx2

As a vector of random variables

X can be considered to be a vector of independent random variables, each having a standard normal distribution. The proof of this formulation on the reverse side.

General multivariate

The general multivariate normal distribution is best understood as being the distribution that results from applying a linear transformation to a random variable having a multivariate standard normal distribution.

General multivariate normal distribution

Let (Ω,F,P) be a probability space, and let Z:ΩRK be a random vector with a multivariate standard normal distribution. Then let X=μ+ΣZ be another random vector. X has a distribution fX:RKR which is a transformed version of Z's distribution, fZ:RKR:

fX(x)=1rescaling factorfz(z in terms of x)=1|det(Σ)|fz(Σ1(xμ))=1|det(Σ)|(12π)Ke(Σ1(xμ))T(Σ1(xμ))2

Standard multivariate normal as a vector of independent random variables. Proof.

Proof that the above probability density function represents K independent standard normal random variables. For clarity, only the case when K=3 will be highlighted:
fZ(z)=1(2π)32ezTz2=1(2π)121(2π)121(2π)12ezTz2=1(2π)121(2π)121(2π)12ez12+z22+z332=12π12π12πez122ez222ez322=12πez12212πez22212πez322=f(z1)f(z2)f(z3)

Thus, the propability distribution for Z represents 3 independent random variables having a individual distribution of 12πezi22 which is the standard normal distribution for a single random variable.

Elementwise conceptualization

The quantity [Xμ]Σ1[Xμ] can be thought of as a component wise operation, sum([Xμ]2σ2), where σ is the column vector of standard deviations. Another alternative is to imagine the component wise methods that would be called in a linear algebra/DL library: accumulate(mult(invert(pow(σ, 2)), pow(Xμ, 2))). 

V=ΣTΣ form. Proof.

The Statlect formulation simplifies the expression. Copy-pasted here:

Dispelling the mystery of the co-variance matrix

Here is a screenshot of some notes on the perspective of a multivariate Gaussian random variable being a composition of two standard normal variables. It tries to dispell some of the mystery about why the co-variance matrix comes up the way it does.



Example