GLAC Net: GLocal Attention Cascading Network for the Visual Storytelling Challenge
Switch branches/tags
Nothing to show
Clone or download
Latest commit 53a6d5f Nov 29, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data Change vocab.pkl path Jul 30, 2018
misc Delete readme.txt Jul 5, 2018
models Change vocab.pkl path Jul 30, 2018
.gitignore Apply .gitignore Aug 1, 2018
LICENSE Create LICENSE Jul 30, 2018
README.md Update README.md Nov 29, 2018
build_vocab.py Change vocab.pkl path Jul 30, 2018
data_loader.py Initial commit Jul 5, 2018
disp_multiple_images.py Initial commit Jul 5, 2018
eval.py Fix some bugs Jul 30, 2018
inference.ipynb Update sample result Aug 4, 2018
model.py Modify function words list Aug 4, 2018
requirements.txt Initial commit Jul 5, 2018
resize.py Update resize.py Jul 18, 2018
train.py Fix some bugs Jul 30, 2018
vist.py Initial commit Jul 5, 2018

README.md

GLAC Net: GLocal Attention Cascading Network

This repository is the implementation of GLAC Net: GLocal Attention Cascading Network for the Visual Storytelling Challenge 2018 as a team SnuBiVtt. Our model got the highest score in the human evaluation of the challenge.

Architecture of GLocal Attention Cascading Network


Dependencies

Python 2.7
Pytorch >= 0.4.0


Prerequisites

1. Clone the repository
git clone https://github.com/tkim-snu/GLACNet.git
cd GLACNet
2. Download requirements
pip install -r requirements.txt
3. Download sentence tokenizer
python
>>> import nltk
>>> nltk.download('punkt')
>>> exit()

Preprocessing

1. Download the dataset

VIST homepage

2. Resize images and build vocabulary

All the images should be resized to 256x256.

python resize.py --image_dir [train_image_dir] --output_dir [output_train_dir]
python resize.py --image_dir [val_image_dir] --output_dir [output_val_dir]
python resize.py --image_dir [test_image_dir] --output_dir [output_test_dir]
python build_vocab.py

Training & Validation

python train.py

Evaluation

1. Download the evaluation tool (METEOR score) for the VIST Challenge
git clone https://github.com/windx0303/VIST-Challenge-NAACL-2018 ../VIST-Challenge-NAACL-2018
2. Install Java
sudo apt-get install default-jdk
3. Run eval.py script
python eval.py --model_num [my_model_num]

The result.json file will be found in the root directory.


Pretrained model

We provide the pretrained model. Please download the link and move to <GLACNet root>/models/.


Citation

@article{Kim2018GLAC,
title={GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generation},
author={Taehyeong Kim and Min-Oh Heo and Seonil Son and Kyoung-Wha Park and Byoung-Tak Zhang},
journal={CoRR},
year={2018},
volume={abs/1805.10973}
}

License

MIT License
This repository refer to pytorch tutorial by yunjey.