FLONET

This project contains implementations of FloNet described in the paper End-to-End Learning of Flowchart Grounded Task Oriented Dialogs.

Reproduce Validation Numbers Reported in the Paper:

1. Clone this repo

2. Set up a python environment using the requirements.txt

3. Download the pre-trained Glove embeddings and unzip the contents to the folder code/glove6B/

4. Download the pretrained checkpoints. The compressed file contains the pretrained retriever and generator models for both the S-Flo and the U-Flo settings

5. Run the inference script

   a. S-Flo setting:

    python flonet.py --save-name='FlonetInferValS' --retriever_checkpoint=path-to-the-sflo-pretrained-retriever-checkpoint.pth.tar  --gpt_model_checkpoint=path-to-the-sflo-pretrained-generator-folder --dialog-dir='../data/dialogs/' --cached-dialog-path='../data/saved_data/cached_in_domain_hard_dialogs.pkl' --domain='in_domain_hard' --saved-glove-path=./glove6B/  --inference=1 --num-epochs=0 --max_length=60
   

   b. U-Flo setting:

   python flonet.py --save-name='FlonetInferValU' --si_model_checkpoint=path-to-the-uflo-pretrained-retriever-checkpoint.pth.tar --gpt_model_checkpoint=path-to-the-uflo-pretrained-generator-folder --dialog-dir='../data/dialogs/' --cached-dialog-path='../data/saved_data/cached_out_domain_dialogs.pkl' --domain='out_domain' --saved-glove-path=./glove6B/  --inference=1 --max_length=60 --num-epochs=0 --emb-size=200 --hidden-size=600
   

Training FloNet from Scratch

1. Clone this repo

2. Set up a python environment using the requirements.txt

3. Download the pre-trained Glove embeddings and unzip the contents to the folder code/glove6B/

4. Pre-train the retriever using retriever.py. Example command shown below:

   python retriever.py --cached-dialog-path='../data/saved_data/flodial_out.pkl' --domain=out_domain --hidden-size=600 --emb-size=200
   

5. Pre-train the generator using generator.py (input the data/gpt_input/ file generated by retriever.py). Example command shown below:

   python generator.py --dataset_path="../data/gpt_data/Retriever_out_domain.json" --dataset_cache="../data/saved_data/flonet_out_cache" 
   

6. Rename generator's last checkpoint (in code/generator/*model_name*/) to pytorch_model.bin

7. Feed pre-trained retriever checkpoint (in data/model/proxybest_checkpoint...), pre-trained generator checkpoint (in code/generator/*model_name*/) and retriever input to flownet.py

   python flonet.py --cached-dialog-path='../data/saved_data/flodial_out.pkl' --domain=out_domain --hidden-size=600 --emb-size=200  --si_model_checkpoint='../data/model/Retriever_checkpoint_out_domain_600_0.0001_16.pth.tar' --gpt_model_checkpoint='../data/generator/_gpt2_flowchart_out_cache_BLEU_1628354260/'
   

Scripts

• retriever.py : Used to pretrain the retriever of FloNet.

  • need to change the following arguments
    • cached-dialog-path : path to a processed copy of input data
    • domain : in_domain (s-flo) or out_domain (u-flo)
    • save-name : prefix for the saved data and logs
    • dialog-dir : path to the dataset folder, ../data/flodial/
    • cached-scores-path : path for storing the proxy scores
    • saved-glove-path : point it to the glove embedding folder
    • hidden-size and emb-size according to the domain as explained in the paper
  • Saves the data required for pretraining GPT in data/gpt_data/ folder
  • model checkpoint saved in path pointed by mode-dir argument
  • logs are saved in log-dir argument

• generator.py - pretrain the generator. Needs the gpt input generated by retriever for training (for the case of training using only flowchart+dialog history or only dialog history, use the GPT input file generated by generate_data_for_generator.py)

  • dataset_path : path of the GPT input file
  • dataset_cache : path for saving a cache of processed GPT input (tokenization and other things)
  • use_flowchart : 1, =0 when training for only dialog history
  • max_length : max decode length

• flonet.py : trains FloNet, if not given checkpoints of retriever and generator, it trains the NoPretrain version. Combines arguments of generator and retriever. has additional below two arguments:

  • si_model_checkpoint : retriever checkpoint file path
  • gpt_model_checkpoint : generator checkpoint folder path