Skip to content
Lifelong Learning with Dynamically Expandable Networks
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
DEN.py . Jan 13, 2018
DEN_run.py . Jan 13, 2018
README.md Update README.md Apr 24, 2018
ops.py cleaning Mar 21, 2018

README.md

Lifelong Learning with Dynamically Expandable Networks

  • Jaehong Yoon(KAIST, AItrics), Eunho Yang(KAIST, AItrics), Jeongtae Lee(UNIST), and Sung Ju Hwang(KAIST, AItrics)

This project hosts the code for our ICLR 2018 paper.

We propose a novel deep network architecture for lifelong learning which we refer to as Dynamically Expandable Network (DEN), that can dynamically decide its network capacity as it trains on a sequence of tasks, to learn a compact overlapping knowledge sharing structure among tasks. DEN is efficiently trained in an online manner by performing selective retraining, dynamically expands network capacity upon arrival of each task with only the necessary number of units, and effectively prevents semantic drift by splitting/duplicating units and timestamping them. We validate DEN on multiple public datasets in lifelong learning scenarios on multiple public datasets, on which it not only significantly outperforms existing lifelong learning methods for deep networks, but also achieves the same level of performance as the batch model with substantially fewer number of parameters.

Reference

If you use this code as part of any published research, please refer the following paper.

@misc{yoon2018lifelong,
  title={Lifelong Learning with Dynamically Expandable Networks},
  author={Yoon, Jaehong and Yang, Eunho and Lee, Jeongtae and Hwang, Sung Ju},
  year={2018},
  publisher={ICLR}
}

Running Code

We implemented the model as described in the paper based on Tensorflow library, Tensorflow.

Get our code

git clone --recursive https://github.com/jaehong-yoon93/DEN.git DEN

Run examples

In this code, you can run our model on MNIST dataset with permutation. Then, you don't need to download dataset on your own, just you get the dataset when you run our code.

For convinence, we added the logs that are printed out validation & test accuracy, and several process. If you execute DEN_run.py, you can reproduce our model.

python DEN_run.py

Authors

Jaehong Yoon13*, Eunho Yang13, Jeongtae Lee2, and Sung Ju Hwang13

1KAIST @ School of Computing, KAIST, Daejeon, South Korea

2UNIST @ School of Electrical and Computer Engineering, UNIST, Ulsan, South Korea

3AItrics @ Seoul, South Korea

* Work done while at UNIST

You can’t perform that action at this time.