Math and science::INF ML AI

Sum of Gaussian random variables

Let $X : Ω \to R$ and $Y : Ω \to R$ be two random variables each with a Gaussian distribution:

$X \sim N (μ_{x}, σ_{x}^{2})$
$Y \sim N (μ_{y}, σ_{y}^{2})$

Then the random variable $Z = X + Y$ has a Gaussian distribution:

$Z \sim N (μ_{x} + μ_{y}, σ_{x}^{2} + σ_{y}^{2})$

Follow-up. $Z = X Y$ does not have a Gaussian distribution?

Can you remember the geometric intuition/proof?

This is a somewhat surprising result. It is not true that the product of two Gaussian random variables is Gaussian. The join distribution of two Gaussian distributed random variables is a multivariate Gaussian distribution, but that is a separate matter, dealing with vector valued random variables. The above theorem concerns only scalar random variables.

Intuition

tl;dr. 2D Gaussian is symmetric, and the distribution of $Z$ can be thought of as a line through the origin of the 2D distribution.

One way to prove this result is to take advantage of the symmetry of the Gaussian distribution. First, consider the 2D joint distribution of the random vector $(X, Y)$ , where we make the simplifying assumption that $X$ and $Y$ have zero mean. We are interested in the derived random variable $Z = X + Y$ . The 2D joint distribution is symmetric around the origin. Any line $x + y = c$ determined by $c \in R$ represents a constant sum of the random variables, which is also the value of $Z$ . All possible lines determined by $c$ are parallel to each other, and we can imagine looking side-on such that the join 2D distribution is summed along these lines to give the distribution of $Z$ . Another way of saying this is to consider any line through the origin, and have the probability mass perpendicular to this line be projected onto the line; each point along the line represents a possible value of $Z$ . As the join distribution is symmetric, and both $X$ and $Y$ are Gaussian, then any line through the origin will have a Gaussian distribution. And so $Z$ is Gaussian.

Proofs

The above intuition can be made into a proof. Alternative proofs include carrying out the a convolution integral, but this is very tedious; properties of the characteristic function can also be used, although I find this method doesn't provide much insight.