Skip to content
Knowledge Extraction with No Observable Data (NeurIPS 2019)
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs Update the slides Dec 3, 2019
pretrained Initial commit Sep 30, 2019
src Update the main script Oct 1, 2019
LICENSE Update LICENSE Jan 9, 2020 Update Jan 9, 2020 Add a demo script Oct 1, 2019
requirements.txt Initial commit Sep 30, 2019


This project is a PyTorch implementation of Knowledge Extraction with No Observable Data (NeurIPS 2019). This paper proposes a novel approach that extracts the knowledge of a deep neural network without any given data.



You can run the demo script by bash, which simply runs src/ It trains five generators based on the pretrained LeNet5 model of pretrained/mnist.pth.tar and then uses it to train a student network from its extracted knowledge. You can change the dataset, pretrained model, and number of generators by modifying src/ It is also possible to modify directly our model package such to change the hyperparameters such as learning rates, network structures, sampling distributions, and so on.

The following figure visualizes artificial images for the SVHN dataset, generated from our five generators. The digits are clearly recognizable despite the noises, representing that our generators are capable of capturing essential properties of the data distribution given a trained teacher network.


You can download this bib file or copy the following information:

title = {Knowledge Extraction with No Observable Data},
author = {Yoo, Jaemin and Cho, Minyong and Kim, Taebum and Kang, U},
booktitle = {Advances in Neural Information Processing Systems 32},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {2701--2710},
year = {2019},
publisher = {Curran Associates, Inc.},
url = {}
You can’t perform that action at this time.