A Model for Natural Language Attack on Text Classification and Inference
This is the source code for the paper: Jin, Di, et al. "Is BERT Really Robust? Natural Language Attack on Text Classification and Entailment." arXiv preprint arXiv:1907.11932 (2019).
- Pytorch >= 0.4
- Tensorflow >= 1.0
- Python >= 3.6
How to use
- Run the following code to install the esim package:
cd ESIM python setup.py install cd ..
- (Optional) Run the following code to pre-compute the cosine similarity scores between word pairs based on the counter-fitting word embeddings.
python comp_cos_sim_mat.py [PATH_TO_COUNTER_FITTING_WORD_EMBEDDINGS]
- Run the following code to generate the adversaries for text classification:
For Natural langauge inference:
- --dataset_path: The path to the dataset. We put the 1000 examples for each dataset we used in the paper in the folder data.
- --target_model: Name of the target model such as ''bert''.
- --target_model_path: The path to the trained parameters of the target model. For ease of replication, we shared the trained BERT model parameters on each dataset we used.
- --counter_fitting_embeddings_path: The path to the counter-fitting word embeddings.
- --counter_fitting_cos_sim_path: This is optional. If given, then the pre-computed cosine similarity scores based on the counter-fitting word embeddings will be loaded to save time. If not, it will be calculated.
- --USE_cache_path: The path to save the USE model file (Downloading is automatic if this path is empty).