Implements entity pre-training

[JohnGiorgi]

Overview

This PR implements entity pre-training. Simply put, entity pre-training involves pre-training the entity module before training the entire NER + RE model jointly.

Previously, we were implementing this by delaying the training of the RE module by a full epoch. I have found a different yet equally simple technique that works quite well. Basically, the loss function is weighted according to

loss = ner_loss + decay_coef * re_loss

where 0 <= delay_coef <= 1 and is computed as current_step / total_steps if this is the first epoch, or 1 otherwise. I saw gains up to 1% on the validation set using this scheme.

[coveralls]

Pull Request Test Coverage Report for Build 502

• 3 of 3 (100.0%) changed or added relevant lines in 1 file are covered.
• 2 unchanged lines in 1 file lost coverage.
• Overall coverage increased (+0.007%) to 88.091%

---

Files with Coverage Reduction	New Missed Lines	%
saber/models/modules/bert_for_entity_and_relation_extraction.py	2	95.15%

Totals
Change from base Build 500:	0.007%
Covered Lines:	1435
Relevant Lines:	1629

---

💛 - Coveralls