Mutual-Information-KDE

Computation of Mutual Information based on Kernel Density Estimation. Implemented in C++ using Eigen.

In order to use the code, compile it with g++ including eigen. No other dependecies needed. Two files are required:

• Input M x 1
• Feature M x N

The Input file acts as reference, against which the N columns of the Feature file will be used to compute the Mutual Information. The Input and Feature file must have the same number of rows.

For now only 1D input are supported.

The program gives as output also the three entropies H(Input), H(Feature), and H(Input, Feature). They are linked to the mutual information by the following:

I(Input, Feature) = H(Feature) + H(Input) - H(Input, Feature);

Relevant references:

• Formula for estimating MI
• Rule for 1D Kernel Width
• Rule for 2D Kernel Width
• Other reference for single/joint pdf estimation