Some issue about Problem 4.6 solution

[Heng-Zhou]

Hi, Zhengqi, I found an issue about Problem 4.6 solution.

At the bottom of page 93, the left side of the equation we need to prove is
$\sum\limits_{n=1}^N({\bf x}_n{\bf x}_n^T)-N{\bf mm}^T.$

For the second term, it is a product of a column vector and a row vector, resulting in a $D\times D$ square matrix. As a result, we cannot write it simply as a scalar $(\ldots)^2$. Instead, we should expand it as is:
$-N{\bf mm}^T= -N\bigl(\frac{1}{N}(N_1{\bf m}_1+N_2{\bf m}_2)\frac{1}{N}(N_1{\bf m}_1^T+N_2{\bf m}_2^T)\bigr)$
$=-\frac{1}{N}(N_1^2{\bf m}_1{\bf m}_1^T+N_1N_2{\bf m}_1{\bf m}_2^T+N_1N_2{\bf m}_2{\bf m}_1^T+N_2^2{\bf m}_2{\bf m}_2^T).\quad \quad \quad \quad\quad \quad \quad \quad(1)$
Note that the middle two terms cannot be merged because they are not equal.

Next let us go from the known result ${\bf S}_ W+\frac{N_1N_2}{N}{\bf S}_ B$. If we plug in definition (4.28) and expand it, we have
${\bf S}_ W=\sum\limits_{n\in\mathcal C_1}({\bf x}_ n{\bf x}_ n^T-{\bf x}_ n{\bf m}_ 1^T-{\bf m}_ 1{\bf x}_ n^T+{\bf m}_ 1{\bf m}_ 1^T)+\sum\limits_{n\in\mathcal C_2}({\bf x}_ n{\bf x}_ n^T-{\bf x}_ n{\bf m}_ 2^T-{\bf m}_ 2{\bf x}_ n^T+{\bf m}_ 2{\bf m}_ 2^T)$
$=\sum\limits_{n=1}^N{\bf x}_ n{\bf x}_ n^T -\left(\sum\limits_{n\in\mathcal C_1}{\bf x}_ n\right){\bf m}_ 1^T-{\bf m}_ 1\left(\sum\limits_{n\in\mathcal C_1}{\bf x}_ n^T\right)-\left(\sum\limits_{n\in\mathcal C_2}{\bf x}_ n\right){\bf m}_ 2^T-{\bf m}_ 2\left(\sum\limits_{n\in\mathcal C_2}{\bf x}_ n^T\right)+N_1{\bf m}_1{\bf m}_1^T+N_2{\bf m}_2{\bf m}_2^T.$

Since equation (1) contains only ${\bf m}_ 1$ and ${\bf m}_ 2$, we substitute $\sum\limits_{n\in\mathcal C_1} {\bf x}_ n= N_1{\bf m}_ 1$ and $\sum\limits_{n\in\mathcal C_2}{\bf x}_ n=N_2{\bf m}_ 2$ for the corresponding sums in the last expression, and we can cancel some terms to get
$\sum\limits_{n\in\mathcal {C_1}}{\bf x}_n{\bf x}_n^T-N_1{\bf m}_1{\bf m}_1^T-N_2{\bf m}_2{\bf m}_2^T.\quad \quad \quad \quad\quad \quad \quad \quad\quad \quad \quad \quad\quad \quad \quad \quad(2)$

Treating $\frac{N_1N_2}{N}{\bf S}_ B$ similarly, we get
$\frac{N_1N_2}{N}{\bf S}_ B=\frac{N_1N_2}{N}({\bf m}_2{\bf m}_2^T-{\bf m}_2{\bf m}_1^T-{\bf m}_1{\bf m}_2^T+{\bf m}_1{\bf m}_1^T).\quad \quad \quad \quad\quad \quad \quad \quad(3)$
Likewise, the middle two terms cannot be merged.

Adding (2) and (3) and merging the same terms, we'll get exactly the same expression as (1), except the additional $\sum\limits_{n=1}^N({\bf x}_n{\bf x}_n^T)$ that appears as the first term on the left side of the equation at the bottom of page 93 that we proposed to prove.

[ZhichaoTan]

Thanks for your correction!