Skip to content

Keras implementation (only for tensorflow backend) of MINE: Mutual Information Neural Estimation

Notifications You must be signed in to change notification settings

ChengzhangZhu/MINE

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 

Repository files navigation

MINE

This code is a Keras implementation (only for tensorflow backend) of MINE: Mutual Information Neural Estimation (https://arxiv.org/pdf/1801.04062.pdf)

Thank mzgubic for providing the Tensorflow implementation

How to use

Option 1: declare the dimension of each input; this build a 3-layer fully-connected network as the statistics network

from mine import MINE
mine = MINE(x_dim=100, y_dim=200)
fit_loss_history, mutual_info = mine.fit(x, y)

Option 2: predefine a statistics network by Keras Model

from mine import MINE
network = Model(inputs=[x_input, y_input], outputs=outputs) # a Keras model
mine = MINE(network=network)
fit_loss_history, mutual_info = mine.fit(x, y)

Several parameters can be set in the fitting function:

fit(x, y, epochs=50, batch_size=100, verbose=1)

paras:

  • x: list or np.array, the input of samples drawn from the first distribution
  • y: np.array, the input of samples drawn from the second distribution
  • epochs: int, the number of training epochs (default: 50)
  • batch_size: int, the training batch size (default: 100)
  • verbose: 0 or 1, whether print training process

returns:

  • fit_loss_history: list, the history of fitting loss values
  • mutual_info: float, the estimated mutual information

A demo has been attached in demo.ipynb. Enjoy :)