【Hackathon 6th No.31】paddle.distribution.Normal support complex normal distribution -part

[NKNaN]

PR Category

User Experience

PR Types

New features

Description

Upgarde paddle.distribution.Normal to support complex normal distribution

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[paddle-ci-bot]

Sorry to inform you that 93adb8c's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

[zhiminzhang0830]

这里没有根号么

[NKNaN]

应该没有的，
若 $Z = X + iY$ ，且 $X \sim N(\mu, \sigma^2)$ , $Y \sim N(\mu, \sigma^2)$ ， $X$ 与 $Y$ 相互独立，则 $Z \sim CN(\mu+i\mu, 2\sigma^2)$
记 $\mu+i\mu = \mu_z \in \mathbb{C}$ ， $2\sigma^2 = \sigma_z^2 \in \mathbb{R}$ ，则其概率密度函数为：

$$ \begin{aligned} p_Z(z) = p_{xy}(x, y) & = \frac{1}{\sqrt{2 \pi \sigma^2}} \exp[-\frac{(x-\mu)^2}{2 \sigma^2}] \cdot \frac{1}{\sqrt{2 \pi \sigma^2}} \exp[-\frac{(y-\mu)^2}{2 \sigma^2}] \\ & = \frac{1}{\pi \cdot 2 \sigma^2} \exp[-\frac{(x-\mu)^2+(y-\mu)^2}{2 \sigma^2}] \\ & = \frac{1}{\pi \cdot 2 \sigma^2} \exp[-\frac{|(x-\mu)+(y-\mu) \cdot i |^2}{2 \sigma^2}] \\ & = \frac{1}{\pi \cdot 2 \sigma^2} \exp[-\frac{|(x+y \cdot i)-(\mu+\mu \cdot i) |^2}{2 \sigma^2}] \\ & =\frac{1}{\pi\sigma_z^2}\exp[-\frac{|z - \mu_z|^2}{\sigma_z^2}] \end{aligned}$$

[zhiminzhang0830]

这种情况下为啥需要转换成float32呢？

[NKNaN]

这里是想把 tuple 和 list 先转成 np.adarray，因为numpy的 np.array 默认是 np.float64，这里是转成paddle的默认类型。是不是这样更好一点：
if isinstance(scale, (tuple, list)):
scale = np.array(scale, dtype=np.float32)

[zhiminzhang0830]

嗯嗯，改成这样把

[NKNaN]

已修改

[zhiminzhang0830]

辛苦确认公式的正确性，最好贴一下推导过程

[NKNaN]

Entopy:

$$ \begin{aligned} H(Z) & = - \int_z p(z)\log p(z) dz \\ & = - \mathbb{E}[\log p(z)] \\ & = - \mathbb{E}[-\log(\pi \sigma_z^2)-\frac{1}{\sigma_z^2}|z-\mu_z|^2] \\ & = \log(\pi \sigma_z^2) + \frac{1}{\sigma_z^2}\mathbb{E}[|z-\mu_z|^2] \\ & = \log(\pi \sigma_z^2) + 1 \end{aligned}$$

KL-DIV:

$$ \begin{aligned} \mathcal{D}_{KL}(p || q) & = \int_z p(z)\log \frac{p(z)}{q(z)} dz \\ & = \mathbb{E}_p[\log \frac{p(z)}{q(z)}] \\ & = \mathbb{E}_p[\log[\frac{\sigma_q^2}{\sigma_p^2} \exp(-\frac{|z-\mu_p|^2}{\sigma_p^2}+\frac{|z-\mu_q|^2}{\sigma_q^2})]] \\ & = -\log \frac{\sigma_p^2}{\sigma_q^2} - \frac{1}{\sigma_p^2}\mathbb{E}_p(z-\mu_p)^2 + \frac{1}{\sigma_q^2}\mathbb{E}_p(z-\mu_q)^2\\ & = -2\log \frac{\sigma_p}{\sigma_q} - 1 + \frac{1}{\sigma_q^2}\mathbb{E}_p(z^cz -\mu_q^cz-z^c\mu_q+\mu_q^c\mu_q) \\ & = -2\log \frac{\sigma_p}{\sigma_q} - 1 + \frac{1}{\sigma_q^2}(\sigma_p^2+\mu_p^c\mu_p -\mu_q^c\mu_p-\mu_p^c\mu_q+\mu_q^c\mu_q) \\ & = -2\log \frac{\sigma_p}{\sigma_q} - 1 + \frac{1}{\sigma_q^2}(\sigma_p^2+|\mu_p-\mu_q|^2) \\ & = \frac{\sigma_p^2}{\sigma_q^2} - 1 + \frac{|\mu_p-\mu_q|^2}{\sigma_q^2}-2\log \frac{\sigma_p}{\sigma_q} \end{aligned} $$

[zhiminzhang0830]

这里为啥scale_np和other_scale_np后面的系数不一样

[NKNaN]

已修改为一致

[zhiminzhang0830]

这里是不是应该是判断other_scale_np

[NKNaN]

应该是的，已修改

[zhiminzhang0830]

LGTM

[luotao1]

请更新对应的中文文档