Skip to content

Commit 8dacfb9

Browse files
committed
Fix formatting
1 parent b3f2d9a commit 8dacfb9

1 file changed

Lines changed: 2 additions & 3 deletions

File tree

content/applied-math/information-theory/01-discrete-entropy.tex

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -656,17 +656,17 @@ \subsection{Inequalities}
656656
\bal
657657
I(X; Y) & = D(p(x,y) \| p(x)p(y)) \geq 0,
658658
\eal
659-
660659
with equality if and only if $p(x,y) = p(x)p(y),$ i.e. $X$ and $Y$ are \@{independent}.
661660

662661
\end{proof}
663662

664663
\end{corollary}
665664

666665
\begin{corollary}[Nonnegativity of Conditional Relative Entropy]
667-
\[ D(p(y|x) \| q(y|x) \geq 0, \]
666+
\[ D(p(y|x) \| q(y|x) \geq 0, \]
668667

669668
with equality if and only if $p(y|x) = q(y|x)$ for all $y$ and $x$ such that $p(x) > 0.$
669+
670670
\begin{proof}
671671

672672
From the definition of \@{conditional relative entropy} we have
@@ -677,7 +677,6 @@ \subsection{Inequalities}
677677
\eal
678678

679679
This is just the average of the \@{relative entropy} over distributions over $y$ for each fixed $x.$ For $p(x) > 0,$ each is $0$ if and only if $p(y|x) = q(y|x),$ by \@{gibbs-inequality}, and positive otherwise.
680-
681680
\end{proof}
682681
\end{corollary}
683682

0 commit comments

Comments
 (0)