Mutual Information loss H

[connorlee77]

Can you be a bit more transparent about how you calculate the mutual information loss? Particularly H. How did you guys calculate the joint histogram?

[ekhahniii]

Hello, I too am curious about the formulation for calculating the joint histogram. Maybe you could comment a bit on the motivation for this line:

p_joint = th.mm(p_f, p_m.transpose(0, 1)).div(self._normalizer_2d)

@ChristophJud @RobinSandkuehler

[connorlee77]

@ekhahniii the histograms are calculated via kernel density estimation. If you write out the multivariate version with a gaussian kernel, you'll notice you can decompose it into the product of two univariate kernels (ignoring any scaling terms). The matrix multiply operation combines this product as well as the summation.