Reference implementation for Structured Prediction with Deep Value Networks
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.

Deep Value Network (DVN)

This code is a python reference implementation of DVNs introduced in

Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs. Michael Gygli, Mohammad Norouzi, Anelia Angelova. ICML 2017. PDF

Note: This code implements the multi-layer perceptron version used for the multi-label classification experiments only (Section 5.1). The segmentation code was written while inside Google and thus not available.


To run this code you need to have tensorflow, numpy, liac-arff, scikit-learn and torchfile installed. Install with

pip install -r requirements.txt

Playing around with a pre-trained Value Net

The pre-trained model for the Bibtex dataset is included in this repository. This allows you do play around with it and it's predictions, using our jupyter notebook.

Replicating the experiments in the paper


To replicate the numbers for bibtex provided in the paper, run:

import reproduce_results
# Reproduce results on the bibtex dataset

By default, the model weights and logs are stored to ./bibtex_dvn. You can monitor the process using tensorboard with

tensorboard --logdir ./bibtex_dvn/

In order to understand the training process two quantities are important:

  1. loss: The loss in estimating the true value of an output hypothesis
  2. gt_f1_scores: The true f1 scores of the generated output hypothesis.

As training progresses, the generated output hypothesis should get better and better. As such, the validation performance reported here closely matches the performance of the test set. The curve should look something like this: Training curve


For Bookmarks the splits are not provided on Thus, we use the splits provided by SPEN. To get the data, run:

cd mlc_datasets
tar -xvf icml_mlc_data.tar.gz
cd ..

Then, you can reproduce the results with

import reproduce_results
# Reproduce results on the bookmarks dataset

The model weights and logs are stored to ./bookmarks_dvn/.


Michael Gygli, Mohammad Norouzi, Anelia Angelova

Code by Michael Gygli