\( \newcommand{\matr}[1] {\mathbf{#1}} \newcommand{\vertbar} {\rule[-1ex]{0.5pt}{2.5ex}} \newcommand{\horzbar} {\rule[.5ex]{2.5ex}{0.5pt}} \newcommand{\E} {\mathrm{E}} \)
deepdream of a sidewalk
Show Answer
\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \)
Math and science::INF ML AI

Belief networks: independence

It's not immediately obvious how to interpret the conditional relationships represented by a belief network. For example, consider the networks below.

Network(s) b, c and d represent the distribution:

\[\begin{aligned}p(x_1, x_2, x_3) &= p(x_1 \vert x_3)p(x_2 \vert x_3)p(x_3) \\&= p(x_2 \vert x_3)p(x_3 \vert x_1)p(x_1) \\&= p(x_1 \vert x_3)p(x_3 \vert x_2)p(x_2) \end{aligned} \]

Network(s) a represent the distribution:

\[ p(x_1, x_2, x_3) = p(x_3 \vert x_1, x_2)p(x_1)p(x_2)\]

Next, consider conditional independence.


  • In a), \( x \) and \( y \) are unconditionally [...], and conditioned on \( z \) they are [...]. \( p(x, y \vert z) = p(x \vert z)p(y \vert z) \)
  • In b), \( x \) and \( y \) are unconditionally [...], and conditioned on \( z \) they are [...]. \( p(x, y \vert z) \varpropto p(z \vert x)p(x)p(y \vert z) \)
  • In c), \( x \) and \( y \) are unconditionally [...], and conditioned on \( z \) they are [...]. \( p(x, y \vert z) \varpropto p(z \vert x, y)p(x)p(y) \)
  • Id d), \( x \) and \( y \) are unconditionally [...], and conditioned on \( z \) or \( w \), are [...]. See book for full equation.