Marian RNN conversion support

[FricoRico]

Feature request

Add support for converting Marian Transformer RNN models.

Motivation

Firefox is doing great work at training new models. Their teacher models are able to be converted to PyTorch models using existing conversion tooling.

However their student models which are way smaller, and more efficient, are structured as Transformer-RNN. This is currently not supported by the conversion tools. Would it be even possible to add support for this?

Your contribution

At this point I'm just a parrot trying to seek help and information on this topic. It is far outside my current knowledge on what this exactly means. Perhaps if someone could point me in the right direction I could figure this out.

[Rocketknight1]

Hi @FricoRico, I don't know if the Marian modeling code in Transformers supports Transformer-RNN architectures either! This means that you'd need to either:

1. Convert the models as "custom code" models: https://huggingface.co/docs/transformers/en/custom_models
2. Write a full PR to add the Transformer-RNN architecture to Transformers

[FricoRico]

@Rocketknight1 Yeah you are right, ideally the Marian tools would also need to be expanded to support Transformer-RNN for inference. But I guess step one would be to even allow to export the models in the first place. But perhaps I'm over simplifying things.

[Rocketknight1]

In general, we need modeling code first in order to support conversion of model checkpoints, rather than the other way around! The model code provides the "architecture" that runs a particular set of weights.

[FricoRico]

I guess the two will have to go hand-in-hand then? Because you will need to have a model to test with? The original models are in npz format.

[Rocketknight1]

Usually, yes! You'd do something like:

1. Make a PR branch that adds the modeling architecture
2. Add a conversion script to that branch
3. Convert the models and add them to the Hub (they won't work in normal Transformers until the PR is merged)
4. Add tests to the PR branch to make sure everything's working okay
5. Get reviews + merge the PR

[FricoRico]

As much as I'd love to write an implementation, I'm by far not knowledgable enough to even comprehend where to start or what to do. Perhaps one day in the future I could invest time to dive deep into the subject. Hopefully there is someone out there more skilled than me that is also interested in this implementation.

Alternatively perhaps it is possible to train MarianMT student models in a regular (non-RNN) Transformer model. But I can't find much information about this, probably for good reasons.