This is code for the paper Difference-Masking (under submission to EMNLP).
We reproduce the experiments from Don't Stop Pretraining using the setting from AANG. Our modifications on the repositories for these papers will be posted soon.
All of these experiments were run using MERLOT-Reserve on Google Cloud TPUs. If you do not have access to Cloud TPU's, this is a good resource. Here is the github for MERLOT-Reserve that our code is based on.
To run TVQA in the finetuning setting, please first get the data from the TVQA dataset. You'll need to fill out a form to access the raw data. Then, code will be posted soon.
At this time, we are not permitted to share the raw Social-IQ data, though we plan to process and release features that can be used to reproduce this code.