KNIFE: kernelized-neural differential entropy estimate

This is the code related to the paper KNIFE (Submitted)

Main Contributors:

• Georg Pichler
• Pierre Colombo
• Malik Boudiaf

Goal :

This repository deals with differential and entropy estimation with application to Computer Vision and NLP. The new estimator proposed in this paper can be used to train neural networks.

Overview

We start by giving an overview of the proposed estimates.

Differential Entropy Estimation

Mutual Information Estimation

References

If you find this repo useful, please cite our paper:

@article{pichler2022knife,
  title={KNIFE: Kernelized-Neural Differential Entropy Estimation},
  author={Pichler, Georg and Colombo, Pierre and Boudiaf, Malik and Koliander, Gunther and Piantanida, Pablo},
  journal={arXiv preprint arXiv:2202.06618},
  year={2022}
}

Usage

Python Function

We provide example inputs under <toto>.py. For example for:

estimator = 

Acknowledgements

Our research was granted access to the HPC resources of IDRIS under the allocation 2021-AP010611665 as well as under the project 2021-101838 made by GENCI.