QKVAE : A model for Unsupervised Disentanglement of Syntax and Semantics

This repo contains the code for our paper Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs

The ParaNMT data files we use can be found in the Google Drive hosted by VGVAE Authors, except for two files which are here. The content of both Google Drives must be downloaded and placed in .data/paranmt/.

After adding the data and installing the dependencies listed in requirements.txt, the model can be trained by running:

python qkv_train.py

A checkpoint corresponding to the best model we obtained among the 5 instances we ran in our paper can be found in this google drive. Note that using this checkpoint requires putting it in a directory named checkpoints under the root of this repo, and setting the run name with the CLI option --test_name QKVBest.

To Be Done:

• Provide usage instructions for syntactic/semantic transfer with the provided checkpoint.
• Package and add evaluation code.