Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Lifelong Learning with Dynamically Expandable Networks

  • Jaehong Yoon(KAIST, AItrics), Eunho Yang(KAIST, AItrics), Jeongtae Lee(UNIST), and Sung Ju Hwang(KAIST, AItrics)

This project hosts the code for our ICLR 2018 paper.

We propose a novel deep network architecture for lifelong learning which we refer to as Dynamically Expandable Network (DEN), that can dynamically decide its network capacity as it trains on a sequence of tasks, to learn a compact overlapping knowledge sharing structure among tasks. DEN is efficiently trained in an online manner by performing selective retraining, dynamically expands network capacity upon arrival of each task with only the necessary number of units, and effectively prevents semantic drift by splitting/duplicating units and timestamping them. We validate DEN on multiple public datasets in lifelong learning scenarios on multiple public datasets, on which it not only significantly outperforms existing lifelong learning methods for deep networks, but also achieves the same level of performance as the batch model with substantially fewer number of parameters.


If you use this code as part of any published research, please refer the following paper.

  title={Lifelong Learning with Dynamically Expandable Networks},
  author={Yoon, Jaehong and Yang, Eunho and Lee, Jeongtae and Hwang, Sung Ju},

Running Code

We implemented the model as described in the paper based on Tensorflow library, Tensorflow.

Get our code

git clone --recursive DEN

Run examples

In this code, you can run our model on MNIST dataset with permutation. Then, you don't need to download dataset on your own, just you get the dataset when you run our code.

For convenience, we added the logs that are printed out validation & test accuracy, and several process. If you execute, you can reproduce our model.



Jaehong Yoon13*, Eunho Yang13, Jeongtae Lee2, and Sung Ju Hwang13

1KAIST @ School of Computing, KAIST, Daejeon, South Korea

2UNIST @ School of Electrical and Computer Engineering, UNIST, Ulsan, South Korea

3AItrics @ Seoul, South Korea

* Work done while at UNIST


Lifelong Learning with Dynamically Expandable Networks



No releases published


No packages published