We are currently in process of uploading our models to Hugging Face...... and code to this repo. Some of our already uploaded models can be accessed using following links:
| Model | IndicBERT | mBERT | XLM-RoBERTa |
|---|---|---|---|
| Joint MLM training on Hindi (hi) and Bengali (bn) | indic-hi-bn-MLM | mBERT-hi-bn-MLM | xlmr-large-hi-bn-MLM |
| Joint MLM training on Hindi (hi) and Telugu (te) | indic-hi-te-MLM | mBERT-hi-te-MLM | xlmr-large-hi-te-MLM |
| Joint MLM training on Hindi (hi) and Bengali (bn) follwed by QA larning on SQuAD-TyDi-MLQA (in sequence) | indic-hi-bn-MLM-SQuAD-TyDi-MLQA | mBERT-hi-bn-MLM-SQuAD-TyDi-MLQA | xlmr-large-hi-bn-MLM-SQuAD-TyDi-MLQA |
| Joint MLM training on Hindi (hi) and Telugu (te) follwed by QA larning on SQuAD-TyDi-MLQA (in sequence) | indic-hi-te-MLM-SQuAD-TyDi-MLQA | mBERT-hi-te-MLM-SQuAD-TyDi-MLQA | xlmr-large-hi-te-MLM-SQuAD-TyDi-MLQA |