Fact Verification and Evidence Finding in Tabular Data

Code for our paper AttesTable at SemEval-2021 Task 9: Extending Statement Verification with Tables for Unknown Class, and Semantic Evidence Finding

Tasks

• Task-A (Statement Verification) : Given a table and a statement, we need to determine whether the table entails the statement, refutes the statement, or has insufficient information about it ("unknown")
• Task-B (Evidence Finding) : When the table entails or refutes the statement, we need to select which cells in the table provide the evidence for it

Data and other competition related details can be found at the SEM-TAB-FACTS website and on the corresponding Codalab competition

Description

• models/
  • Task-A.ipynb : Notebook used for training/validating the models for Task-A
  • Task-B.ipynb : Notebook used for training/validating the models for Task-B
• utils/
  • Task-A/
    • augment_df.py : Synthesizes the unknown statements as proposed in our paper
    • augment_df_2.py : Augments the DF directly and inserts unknown statements randomly in each table from other tables in the same file, if no other tables, then it randomly selects the statements from other files. Remove some tables of autogenerated folder so that number of tables from manual and autogenerated folders are same.
    • transform_data.py : Transforms the data from the given XML format to CSV format
    • transform_data_2.py : Given a (table, statement) DF, generates a (table, statements) DF directly
    • unaugment_df.py : Removes the unknown statements from a DF
  • Task-B/
    • create_data.py : Creates the DF used while training the models for task-B in a (cell, statement) format
    • create_data_list.py : Creates the DF used while training the models for task-B in a (statement, [cells]) format
    • create_data_undersampled.py : Creates the DF used while training the models for task-B in a (statement, [cells]) format, while simultaneously undersampling the "irrelevant" class
  • merge_df.py : Merges/stacks two given CSV files

License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

Authors

• Harshit Varma
• Aadish Jain
• Pratik Ratadiya
• Abhishek Rathi

Citation

If you find this work useful, please cite:

@inproceedings{varma-etal-2021-attestable,
    title = "{A}ttes{T}able at {S}em{E}val-2021 Task 9: Extending Statement Verification with Tables for Unknown Class, and Semantic Evidence Finding",
    author = "Varma, Harshit  and
      Jain, Aadish  and
      Ratadiya, Pratik  and
      Rathi, Abhishek",
    booktitle = "Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.semeval-1.182",
    doi = "10.18653/v1/2021.semeval-1.182",
    pages = "1276--1282"
}

Contact

Feel free to contact Pratik Ratadiya (pratik.r@vcreatek.com) for any queries