Skip to content

awslabs/hypergraph-tabular-lm

HyTrel

A hypergraph-based tabular language model.

Introduction

This repository contains the official implementation for the paper HyTrel: Hypergraph-enhanced Tabular Data Representation Learning with code, data, and checkpoints. figure1

Installation

It's recommended to use python 3.9.

Here is an example of creating the environment using Anaconda.

  • Create the virtual environment using conda create -n hytrel python=3.9
  • Install the required packages with the corresponding versions from requirements.txt

Note: If you encounter difficulty installing torch_geometric, please refer here to install it according to your environment settings.

Pretraining

  • Pre-process the raw data, slicing the big file into chunks, and put the *.jsonl files into the directory /data/pretrain/chunks/. Sample data is present here and the files can be used as reference.
    Note: Pretraining data *.jsonl are acquired and preprocessed by using the scripts from the TaBERT.

  • Run python parallel_clean.py to clean and serialize the tables.
    Note: We serialize the tables as arrow in consideration of memory usage.

  • Run sh pretrain_electra.sh to pretrain HyTrel with the ELECTRA objective.

  • Run sh pretrain_contrast.sh to pretrain HyTrel with the Contrastive objective.

Evaluation

First put the ELECTRA-pretrained checkpoint to /checkpoints/electra/, and Contrast-pretrained checkpoint to /checkpoints/contrast/.

Column Type Annotation

  • Put the data {train, dev, test}.table_col_type.json and type_vocab.txt into the directory /data/col_ann/.

  • Run sh evaluate_cta_electra.sh with ELECTRA-pretrained checkpoint.

  • Run sh evaluate_cta_contrast.sh with Contrast-pretrained checkpoint.

Column Property Annotation

  • Put the data {train, dev, test}.table_rel_extraction.json and relation_vocab.txt into the directory /data/col_rel/.

  • Run sh evaluate_cpa_electra.sh with ELECTRA-pretrained checkpoint.

  • Run sh evaluate_cpa_contrast.sh with Contrast-pretrained checkpoint.

Table Type Annotation

  • Decompose ttd.tar.gz into train, dev, test data folders under the directory /data/ttd/.

  • Run sh evaluate_ttd_electra.sh with ELECTRA-pretrained checkpoint.

  • Run sh evaluate_ttd_contrast.sh with Contrast-pretrained checkpoint.

Reference

Please cite our paper.

@article{chen2023hytrel,
 archiveprefix = {arXiv},
 author = {Pei Chen and Soumajyoti Sarkar and Leonard Lausen and Balasubramaniam Srinivasan and Sheng Zha and Ruihong Huang and George Karypis},
 eprint = {2307.08623},
 primaryclass = {cs.LG},
 title = {HYTREL: Hypergraph-enhanced Tabular Data Representation Learning},
 year = {2023}
}

Contact

For the data and model checkpoints, please email: chen.pei518@163.com (Pei Chen)

About

No description, website, or topics provided.

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published