This model gives roughly same performance as the Tensorflow baseline (0.57~0.58 public score).
This is intended to serve as a reference implementation for more complicated QA tasks.
Example inference notebook on Kaggle.
The GCP pre-emptible GPUs became too hard to use (i.e., got pre-empted too often) in mid-December 2019, so I dropped this PyTorch version and started to work on the Tensorflow version using TPU.
- pytorch_helper_bot==0.4.0
- transformer==2.2.1
- fire
- The
master
branch uses BERT pretrained models. Thealbert
branch uses ALBERT pretrained models. - don't worry about the Telegram bot token in train.py. I've already expired the token.)