Skip to content
Code for Interpretable Adversarial Perturbation in Input Embedding Space for Text, IJCAI 2018.
Python Shell
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
README.md
Setup.md add how to build enviroment Jun 24, 2018
adaptive_softmax.py add training code Jun 24, 2018
analysis.py
lm_nets.py
nets.py Fix iAdv Jun 24, 2018
pretrain.py
train.py
utils.py
utils_pretrain.py add training code Jun 24, 2018
visualize.py update visualize code Jun 25, 2018
visualize_sample01.png
visualize_sample02.png update visualize code Jun 25, 2018

README.md

Interpretable Adversarial Perturbation

Code for Interpretable Adversarial Perturbation in Input Embedding Space for Text, IJCAI 2018.

This code reproduce the our paper with Chainer.

Setup Environment

Please install Chainer and Cupy.

You can set up the environment easily with this Setup.md.

Download Pretrain Model

Please download pre-trained model. From http://sato-motoki.com/research/vat/imdb_pretrained_lm_ijcai.model

$ wget http://sato-motoki.com/research/vat/imdb_pretrained_lm_ijcai.model

Run

Pretrain

$ python -u pretrain.py -g 0 --layer 1 --dataset imdb --bproplen 100 --batchsize 32 --out results_imdb_adaptive --adaptive-softmax

Note that this command takes about 30 hours with single GPU.

Train (iVAT: Interpretable Semi-supervised setting)

Please add --use_semi_data 1 and --use_attn_d 1 to use iVAT (ours).

$ python train.py --gpu=0 --n_epoch=30 --batchsize 32 --save_name=imdb_model_vat --lower=0 --use_adv=0 --xi_var=15.0  --use_unlabled=1 --alpha=0.001 --alpha_decay=0.9998 --min_count=1 --ignore_unk=1 --pretrained_model imdb_pretrained_lm_ijcai.model --use_exp_decay=1 --clip=5.0 --batchsize_semi 96 --use_semi_data 1 --use_attn_d 1 

Train (VAT: Semi-supervised setting)

Please add --use_semi_data 1 to use VAT.

$ python train.py --gpu=0 --n_epoch=30 --batchsize 32 --save_name=imdb_model_vat --lower=0 --use_adv=0 --xi_var=5.0  --use_unlabled=1 --alpha=0.001 --alpha_decay=0.9998 --min_count=1 --ignore_unk=1 --pretrained_model imdb_pretrained_lm_ijcai.model --use_exp_decay=1 --clip=5.0 --batchsize_semi 96 --use_semi_data 1

Note that this command takes about 8 hours with single GPU.

Train (iAdv: Interpretable Supervised setting)

Please add --use_adv 1 and --use_attn_d 1 to use iAdv.

$ python train.py --gpu=0 --n_epoch=30 --batchsize 32 --save_name=imdb_model_adv --lower=0 --use_adv 1 --xi_var=15.0  --use_unlabled=1 --alpha=0.001 --alpha_decay=0.9998 --min_count=1 --ignore_unk=1 --pretrained_model imdb_pretrained_lm_ijcai.model --use_exp_decay=1 --clip=5.0 

Train (Adv: Supervised setting)

Please add --use_adv 1 to use Adv.

$ python train.py --gpu=0 --n_epoch=30 --batchsize 32 --save_name=imdb_model_adv --lower=0 --use_adv 1 --xi_var=5.0  --use_unlabled=1 --alpha=0.001 --alpha_decay=0.9998 --min_count=1 --ignore_unk=1 --pretrained_model imdb_pretrained_lm_ijcai.model --use_exp_decay=1 --clip=5.0

Note that this command takes about 6 hours with single GPU.

Visualize Example

Visualize (iAdv)

Please change [trained_model_filename].

python analysis.py --gpu 0 --n_epoch 1 --load_trained_lstm [trained_model_filename] --use_attn_d=1 --use_adv 1 --analysis_mode 1 --lower 0 --batchsize 1 --save_name visualize_imdb_baseline.pickle
python visualize.py --pickle_filename visualize_imdb_baseline.pickle --savefig_dir figs

Authors

We thank Takeru Miyato (@takerum) who suggested that we reproduce the result of a [Miyato et al., 2017].

Reference

[Miyato et al., 2017]: Takeru Miyato, Andrew M. Dai and Ian Goodfellow
Adversarial Training Methods for Semi-Supervised Text Classification.
International Conference on Learning Representation (ICLR), 2017

[Sato et al., 2018]: Motoki Sato, Jun Suzuki, Hiroyuki Shindo, Yuji Matsumoto
Interpretable Adversarial Perturbation in Input Embedding Space for Text.
IJCAI-ECAI-2018
You can’t perform that action at this time.