Add Luke to Onnx

[aakashb95]

What does this PR do?

I added lines to make Luke models available for Onnx conversion.

Who can review?

• albert, bert, xlm: @LysandreJik

adding @NielsRogge since he worked on luke

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[lewtun]

Thanks for this contribution @aakashb95 🤗 ! Overall this is looking good and I think we can minimally add support for the masked-lm feature (the other features need a discussion)

Can you also please fix the merge conflicts and test that the slow tests pass with:

RUN_SLOW=1 pytest tests/onnx/test_onnx_v2.py -k "luke"

Thanks!

[lewtun]

We should include the other heads that LUKE supports:

• masked-lm
• entity-classification
• entity-pair-classification
• entity-span-classification

Now, masked-lm is easy to add, but for the entity-* tasks you'll need to add new entries in the _TASKS_TO_AUTOMODELS mapping in the FeaturesManager here. One somewhat annoying thing about LUKE is that these new mappings are specific to just one model and we lose the nice correspondence with autoclasses.

@michaelbenayoun do you think it's OK to include LUKE-specific classes inside the FeaturesManager?

[michaelbenayoun]

I'm not very familiar with LUKE, but if it does not fit what is currently being done, I think it is okay to add those classes in the FeaturesManager

[aakashb95]

RUN_SLOW=1 pytest tests/onnx/test_onnx_v2.py -k "luke"

I have all requirements installed and yet, I get the following output when I run the slow test:

RUN_SLOW=1 pytest tests/onnx/test_onnx_v2.py -k "luke"
================= test session starts==============================
platform darwin -- Python 3.9.12, pytest-7.1.1, pluggy-1.0.0
rootdir: /Users/aakash/work/transformers, configfile: setup.cfg
collected 141 items / 141 deselected / 0 selected                                                                                                                                                                                             

What am I missing?

[michaelbenayoun]

You need to add the luke architecture to be tested in tests/onnx/test_onnx_v2.py.
Currently you are running those tests and filtering for the ones containing "luke" in their names, but none is found.

[aakashb95]

the slow tests resulted in the following errors:
luke_onnx_test.log

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[NielsRogge]

@lewtun could you revive this PR? Seems like it was already in a good state.

[lewtun]

Hey @aakashb95 sorry for the slow reply!

The reason your tests are failing is because:

• LUKE can't be loaded with AutoModelForMaskedLM. To solve this, please rebase your branch on main to include the recent changes in Debug LukeForMaskedLM #17499
• You need to import the LukeForXxx PyTorch classes in features.py here
• You need to specify the outputs for the new features in config.py in the _tasks_to_common_outputs dictionary, e.g. add this:

  "entity-classification": OrderedDict({"logits": {0: "batch"}}),
  "entity-pair-classification": OrderedDict({"logits": {0: "batch"}}),
  "entity-span-classification": OrderedDict({"logits": {0: "batch"}}),

After that, I think this PR should be in a good shape and the tests will pass :)

[aakashb95]

Thanks for reviving this PR!
I have incorporated the changes you mentioned and run the slow tests.

The tests fail with errors similar to the one below:

self = <tests.onnx.test_onnx_v2.OnnxExportTestCaseV2 testMethod=test_pytorch_export_071_luke_entity_pair_classification>, test_name = 'luke_entity-pair-classification'
name = 'luke', model_name = 'studio-ousia/luke-base', feature = 'entity-pair-classification'
onnx_config_class_constructor = functools.partial(<bound method OnnxConfig.from_model_config of <class 'transformers.models.luke.configuration_luke.LukeOnnxConfig'>>, task='entity-pair-classification')
device = 'cpu'

    def _onnx_export(self, test_name, name, model_name, feature, onnx_config_class_constructor, device="cpu"):
        from transformers.onnx import export
    
        model_class = FeaturesManager.get_model_class_for_feature(feature)
        config = AutoConfig.from_pretrained(model_name)
>       model = model_class.from_config(config)
E       AttributeError: type object 'LukeForEntityPairClassification' has no attribute 'from_config'

tests/onnx/test_onnx_v2.py:266: AttributeError

If my understanding is correct, the AutoModelFor* classes have from_config implemented and since luke is unique, a from_config would have to be implemented for corresponding LukeFor* classes, right?

Attaching logs for reference:
luke_onnx_slow_test.log

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[NielsRogge]

cc'ing @lewtun here

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.