Fine-tuning AraBERT for Machine Translation

[salma-elshafey]

Hello, I want to fine-tune AraBERT for Ar-En Machine Translation. Do you have any idea how can I do so?
Thanks in advance.

[WissamAntoun]

Hey,

AraBERT for translation isn't a standard use case. Basically, it is not directly supported.

What you can do instead is use a BERT-to-BERT architecture as shown in this paper here https://arxiv.org/pdf/1907.12461.pdf , and for implementation check the thread here huggingface/transformers#8944. Or our other repo that uses BERT2BERT in a chatbot setting https://github.com/aub-mind/Arabic-Empathetic-Chatbot