Converting models from Huggingface?

[mkserge]

Hello,

I am trying to find a way to convert models trained using Huggingface.

Using Python 3.8.6, PyTorch 1.9.0.

Step 1: Save the model in torch.

(venv) sergey_mkrtchyan browse_reader (master) $ python
Python 3.8.6 (v3.8.6:db455296be, Sep 23 2020, 13:31:39)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> from transformers import RobertaForSequenceClassification
>>> model = RobertaForSequenceClassification.from_pretrained('/Users/sergey_mkrtchyan/workspace/mrc/browse_models/tanda_roberta_large_asnq_orig/')
Some weights of the model checkpoint at /Users/sergey_mkrtchyan/workspace/mrc/browse_models/tanda_roberta_large_asnq_orig/ were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
>>> torch.save(model, '/Users/sergey_mkrtchyan/workspace/mrc/browse_models/tanda_roberta_large_asnq_pt/tanda.pt')
>>>

Step2: Convert the model using OLive's ONNX Converter Image

sergey_mkrtchyan OLive (master) $ docker run -v /Users/sergey_mkrtchyan/workspace/mrc/browse_models:/mnt/ onnx-converter --model /mnt/tanda_roberta_large_asnq_pt/tanda.pt --output_onnx_path /mnt/tanda_roberta_large_asnq_pt/tanda.onnx --model_type pytorch --model_input_shapes "[(1,7),(1,7)]"
WARNING:root:scikit-learn version 0.24.2 is not supported. Minimum required version: 0.17. Maximum required version: 0.19.2. Disabling scikit-learn conversion API.

-------------
Model Conversion

Conversion error occurred. Abort.

-------------
MODEL CONVERSION SUMMARY (.json file generated at /mnt/tanda_roberta_large_asnq_pt/output.json )

{'conversion_status': 'FAILED',
 'correctness_verified': 'FAILED',
 'error_message': "No module named 'transformers'",
 'input_folder': '',
 'output_onnx_path': ''}
Traceback (most recent call last):
  File "src/onnx_converter.py", line 348, in <module>
    main()
  File "src/onnx_converter.py", line 312, in main
    raise e
  File "src/onnx_converter.py", line 302, in main
    convert_models(args)
  File "src/onnx_converter.py", line 276, in convert_models
    converter(args)
  File "src/onnx_converter.py", line 179, in pytorch2onnx
    model = torch.load(args.model, map_location="cpu")
  File "/usr/local/lib/python3.6/site-packages/torch/serialization.py", line 607, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/usr/local/lib/python3.6/site-packages/torch/serialization.py", line 882, in _load
    result = unpickler.load()
  File "/usr/local/lib/python3.6/site-packages/torch/serialization.py", line 875, in find_class
    return super().find_class(mod_name, name)
ModuleNotFoundError: No module named 'transformers'
sergey_mkrtchyan OLive (master) $

It seems like the model somehow preserves the information about the transformers package where it was trained at. Is there any way to get rid of this?

Note that directly loading the pytorch_model.bin file results in another exception.

sergey_mkrtchyan OLive (master) $ docker run -v /Users/sergey_mkrtchyan/workspace/mrc/browse_models:/mnt/ onnx-converter --model /mnt/tanda_roberta_large_asnq_pt/pytorch_model.bin --output_onnx_path tanda_roberta_large_asnq_amazon/tanda.onnx --model_type pytorch --model_input_shapes "[(1,7),(1,7)]"
WARNING:root:scikit-learn version 0.24.2 is not supported. Minimum required version: 0.17. Maximum required version: 0.19.2. Disabling scikit-learn conversion API.

-------------
Model Conversion

Conversion error occurred. Abort.

-------------
MODEL CONVERSION SUMMARY (.json file generated at tanda_roberta_large_asnq_amazon/output.json )

{'conversion_status': 'FAILED',
 'correctness_verified': 'FAILED',
 'error_message': "'collections.OrderedDict' object has no attribute "
                  "'training'",
 'input_folder': '',
 'output_onnx_path': ''}
Traceback (most recent call last):
  File "src/onnx_converter.py", line 348, in <module>
    main()
  File "src/onnx_converter.py", line 312, in main
    raise e
  File "src/onnx_converter.py", line 302, in main
    convert_models(args)
  File "src/onnx_converter.py", line 276, in convert_models
    converter(args)
  File "src/onnx_converter.py", line 182, in pytorch2onnx
    torch.onnx.export(model, dummy_model_input, args.output_onnx_path)
  File "/usr/local/lib/python3.6/site-packages/torch/onnx/__init__.py", line 280, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/usr/local/lib/python3.6/site-packages/torch/onnx/utils.py", line 94, in export
    use_external_data_format=use_external_data_format)
  File "/usr/local/lib/python3.6/site-packages/torch/onnx/utils.py", line 674, in _export
    with select_model_mode_for_export(model, training):
  File "/usr/local/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/site-packages/torch/onnx/utils.py", line 38, in select_model_mode_for_export
    is_originally_training = model.training
AttributeError: 'collections.OrderedDict' object has no attribute 'training'

[leqiao-1]

If you have any further concerns or questions, please reopen this issue.