Deep exponential families (DEFs)
C++ Jupyter Notebook Python
Switch branches/tags
Nothing to show
Clone or download
altosaar Merge pull request #3 from blei-lab/obs-weights-config
fix obs_weights being ignored in config
Latest commit 93d8456 Jan 29, 2018
Permalink
Failed to load latest commit information.
scripts DEF release Jun 19, 2015
wikipedia Fix path to validation so example in readme works Nov 13, 2017
README.md Update README.md Dec 22, 2017
bbvi.cpp DEF release Jun 19, 2015
bbvi.hpp DEF release Jun 19, 2015
def.cpp DEF release Jun 19, 2015
def.hpp DEF release Jun 19, 2015
def_2_model.cpp DEF release Jun 19, 2015
def_2_model.hpp DEF release Jun 19, 2015
def_bernoulli_layer.cpp DEF release Jun 19, 2015
def_bernoulli_layer.hpp DEF release Jun 19, 2015
def_bernoulli_y_layer.cpp DEF release Jun 19, 2015
def_bernoulli_y_layer.hpp DEF release Jun 19, 2015
def_data.cpp DEF release Jun 19, 2015
def_data.hpp DEF release Jun 19, 2015
def_example.ini DEF release Jun 19, 2015
def_exponential_layer.cpp DEF release Jun 19, 2015
def_exponential_layer.hpp DEF release Jun 19, 2015
def_gamma_layer.cpp DEF release Jun 19, 2015
def_gamma_layer.hpp DEF release Jun 19, 2015
def_gaussian_layer.cpp DEF release Jun 19, 2015
def_gaussian_layer.hpp DEF release Jun 19, 2015
def_layer.hpp DEF release Jun 19, 2015
def_main.cpp DEF release Jun 19, 2015
def_model.cpp fix obs_weights being ignored in config Jan 29, 2018
def_model.hpp DEF release Jun 19, 2015
def_poisson_layer.cpp DEF release Jun 19, 2015
def_poisson_layer.hpp DEF release Jun 19, 2015
def_poisson_y_layer.cpp DEF release Jun 19, 2015
def_poisson_y_layer.hpp DEF release Jun 19, 2015
def_y_layer.hpp DEF release Jun 19, 2015
layer_factory.cpp DEF release Jun 19, 2015
layer_factory.hpp DEF release Jun 19, 2015
link_function.cpp DEF release Jun 19, 2015
link_function.hpp DEF release Jun 19, 2015
optimizer.cpp DEF release Jun 19, 2015
optimizer.hpp DEF release Jun 19, 2015
random.hpp DEF release Jun 19, 2015
serialization.cpp DEF release Jun 19, 2015
serialization.hpp DEF release Jun 19, 2015
utils.cpp DEF release Jun 19, 2015
utils.hpp DEF release Jun 19, 2015
utils.py DEF release Jun 19, 2015
vocab.dat DEF release Jun 19, 2015
waf DEF release Jun 19, 2015
wscript DEF release Jun 19, 2015

README.md

Deep Exponential Family

Reference

Deep Exponential Families by Rajesh Ranganath, Linpeng Tang, Laurent Charlin, and David M. Blei, AISTATS 2015.

Requirements

  • armadillo
  • boost 1.55
  • OpenMP
  • GSL
  • g++ >= 4.7

Instructions to Build and Run

Configuring: ./waf configure

Building: ./waf build (binary is build/def_main)

Running: def reads its options from a config file and from the command line. ./build/def_main --help shows the command line options.

We give a full example below (including a sample config file) of a def running a dataset of wikipedia articles.

Input Format for Text

The header:

n_examples n_words

Followed by n_examples examples, each has two lines:

example_ind example_words
word_ind0 word_count0 word_ind1 word_count1 ...

Comprehensive Example

  1. Data. We have pre-processed a corpus of wikipedia articles containing 1000 train articles, 500 validation and, 500 test articles. The data are available in folder wikpedia

  2. Configuration file. The configuration file will be read by def and contains all options regarding the model to be trained (e.g., number of layers, size of layers, distribution of global and local variables). Example configuration for the wikipedia dataset is available in folder wikipedia. Note that this file will look for the dataset in a directory specified by the WIKIPEDIA_DEF environment variable.

  3. Running. Here is an example invocation:

cd deep-exponential-families
# define environment variable used in def_wikipedia_50_25_10.ini
export WIKIPEDIA_DEF=`pwd`/wikipedia
./build/def_main --v=3 --folder=experiments/def_wikipedia --algo=rmsprop --rho=.2 --samples=64 --max_examples=1000000 --model=wikipedia/def_wikipedia_50_25_10.ini --batch=10000 --batch_order=rand --threads=5 --test_interval=5 --iter=2000

The above settings (including the values provided in the configuration file) are the ones we used in the paper. We have found these settings to be useful across several datasets.

Topic visualization

We show some of the tools that we have used to explore the def fits in this DEF IPython Notebook.