File Structure

• assets
• data
  • 2WikiMultihopQA
  • CompositionalCelebrities
  • FinetuningData
    • direct_dev_130.json: Direct prompting validation dataset with token count up to 130.
    • direct_train_130.json: Direct prompting training dataset with token count up to 130.
    • self_ask_dev_300.json: Self-ask prompting validation dataset with token count up to 300.
    • self_ask_train_300.json: Self-ask prompting training dataset with token count up to 300.
    • Training and validation (dev) data used in this paper can be downloaded here
  • MultihopEvaluation
• logs
  • data_generation.log
  • evaluation.log
  • token_stats.log
• models
  • flan-t5-small-direct.h5: Flan-T5 small model fine-tuned with direct prompting.
  • flan-t5-small-self-ask.h5: Flan-T5 small model fine-tuned with self-ask prompting.
  • t5-small-direct.h5: T5-small model fine-tuned with direct prompting.
  • t5-small-self-ask.h5: T5-small model fine-tuned with self-ask prompting.
  • Trained models can be downloaded here
• responses
  • Naming convention: model - finetune method (if any) - with / without examplars - responses.json
  • Responses used in this paper can be downloaded here
• results
  • Plots
    • Plots used in the paper.
  • Naming convention: model - finetune method (if any) - with / without examplars - results.json
  • Results used in this paper can be downloaded here
• samples
  • Qualitative Analysis Samples
• compositional-reasoning-paper.pdf
• compositional-reasoning-proposal.pdf
• compositional-reasoning-slides.pdf
• evaluation.py: command line file to evaluate model performance.
• training_demo.ipynb: Demo notebook for fine-tuning the baseline models.
• training_utils.py: Utility tools to generate dataset, train keras model with limited ram, and etc.