Probability Weighted Word Saliency(PWWS)

We add prediction of a victim classifier from the Probability Weighted Word Saliency(PWWS)(Github). The implementations of the ACL2019 paper Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency.

Overview

• data_set/aclImdb/ , data_set/ag_news_csv/anddata_set/yahoo_10 are placeholder directories for the IMDB Review, AG's News and Yahoo! Answer, respectively.
• word_level_process.pyandchar_level_process.py contain two different prepressing methods of dataset for word-level and char-level, respectively.
• neural_networks.py contain implementations of four neural networks(word-based CNN, Bi-directional LSTM, char-based CNN, LSTM) used in paper.
• Use training.pyto train four NN in neural_networks.py.
• fool.py, evaluate_word_saliency.py, get_NE_list.py,adversarial_tools.pyandparaphrase.pybuild the experiment pipeline.

Dependencies

• Python 3.7.1.
• Versions of all depending libraries are specified in requirements.txt. To reproduce the reported results, please make sure that the specified versions are installed.
• If you did not download WordNet(a lexical database for the English language), use nltk.download('wordnet') to do so.(Cancel the code comment on line 14 in paraphrase. py)

Usage

• Download dataset files from google drive , which include
  • IMDB: aclImdb.zip. Decompression and place the folderaclImdb indata_set/.
  • AG's News: ag_news_csv.zip. Decompression and place the folder ag_news_csv indata_set/.
  • Yahoo Answers: yahoo_10.zip. Decompression and place the folder yahoo_10 indata_set/.
• Download glove.6B.100d.txtfrom google drive and place the file in /.
• Run training.py or use command likepython3 training.py --model word_cnn --dataset imdb --level word. You can reset the model hyper-parameters in neural_networks.py and config.py.Note that neither this repository nor the paper provides an implementation of char_cnn on IMDB and Yahoo! Answers datasets.
• Run fool.py or use command likepython3 fool.py --model word_cnn --dataset imdb --level wordto generate adversarial examples using PWWS.
• If you want to train or fool different models, reset the argument in training.pyandfool.py.

Result on adversarial generation

runs/contains some pretrained NN models.

• adv.txt means adversarial texts.
• org.txt means original texts.
• adv_predict.txt means prediction of victim classifier for adversarial texts.
• org_predict.txt means prediction of victim classifier for original texts.

Contact

• If you have any questions regarding the code, please create an issue or contact the owner of this repository.

Acknowledgments

• Code refer to: PWWS (Github).