Independence & Conditional Independence
Two random variables, \(X\) and \( Y \) are independent if the occurrence of one does not give any information on the likelihood of the other event occurring. In other words, their probability distribution can be expressed as a product of two factors, one only involving \(X\), and one only involving \(Y\):
\[\forall x \in X, y \in Y, \, p(X = x, Y= y) = p(X = x)p(Y = y)\]
Two random variables, x and y are conditionally independent if, given knowledge of the occurrence of \(Z\), knowledge of the occurrence of \(X\) provides no information on the likelihood of the occurrence of \(Y\). This can be expressed as:
\[\forall x \in X, y \in Y, z \in Z, \, p(X = x, Y= y | Z = z) = p(X = x | Z = z)p(Y = y | Z = z)\]
Example
Independent events
x: the number on a rolled dice
y: the number on another rolled dice
Conditionally independent
x: a person's height
y: the person's vocabulary
z: the person's age