A BERT-variant language model for oil and gas domain by adapting the BERT model through adding extended modules and OOVs to the pretrained dictionary.
The whole process for this part is explained in chapter 5 of my PhD dissertation.
For brief explanation you can visit here: