Infotheory, written in C++, is a software package to perform information theoretic analysis, especially on high-dimensional data. While inferring the data distribution from samples, we've utilized sparse representations thus avoiding the catastrophic explosion of bin counts with increased dimensions. Moreover, this package enables better distribution estimation by employing averaged shifted histograms .
The following information theoretic quantities can be estimated using this tool as of now and follow this repo for more to come.
- Entropy 
- Mutual Information 
- Partial Information Decomposition measures 
- Unique Information
- Redundant Information
- Synergistic Information
The package can be used in Python or C++. While the C++ headers should function well on all platforms, the python package has currently only been tested on MacOS and Linux.
pip install infotheory
On MacOS, upgrade to MacOS-Mojave and update Xcode. You might have to set these environment variables from your terminal.
export CXXFLAGS="-mmacosx-version-min=10.9" export LDFLAGS="-mmacosx-version-min=10.9"
Using this package in your own code involves three steps.
See colab demo here
Created by Madhavun Candadai and Eduardo J. Izquierdo. Contact Madhavun at madvncv[at]gmail.com
- Scott, D. W. (1985). Averaged shifted histograms: effective nonparametric density estimators in several dimensions. The Annals of Statistics, 1024-1040.
- Williams, P. L., & Beer, R. D. (2010). Nonnegative decomposition of multivariate information. arXiv preprint arXiv:1004.2515.