Skip to content

Initial models compatible with latest release of transformers library
Compare
Choose a tag to compare
@tingofurro tingofurro released this 08 Sep 17:15
· 7 commits to master since this release
0.2
533bb59

We release an upgraded set of initial models for the training script that are compatible with transformers==3.1.0 to make it easier to get started.
The original release (0.1) used version 2.8.0 of transformers, and there were some breaking changed introduced since, which leads to some model loading failing.
The requirements.txt in the latest release has been updated with compatible library versions to simplify installation.

Initial Models

These sets of models work using Python 3.6.10, Transformers 3.1.0 and Sklearn 0.22.1:

  • keyword_extractor.joblib: An sklearn pipeline that will extract can be used to compute tf-idf scores of words according to the BERT vocabulary, which is used by the Masking Procedure,
  • bert_coverage.bin: A bert-base-uncased finetuned model on the task of Coverage for the news domain,
  • fluency_news_bs32.bin: A GPT2 (base) model finetuned on a large corpus of news articles, used as the Fluency model,
  • gpt2_copier23.bin: A GPT2 (base) model that can be used as an initial point for the Summarizer model.

Final Models

Unfortunately, the three final models (trained summarizers) released in v0.1 do not work anymore in the latest transformers library, and only work in versions 2.8.0 and before. Once we retrain these models, we will reupload them. If this is of interest, feel free to add an issue or contact us.

Assets 6