Huggingface model not being torch scriptable

[HamidShojanazeri]

❓ Questions and Help

Referring to the recommendation of trying script API for Huggingface (HF) models in issue, here this has been tried and facing the issue as reported in the logs. Generally, HF models are not scriptable, there has been feature request to achieve this, some work has been started in this PR, however there is not much progress updated. Here is code snippet to reproduce the issue with latest transformers version. Any thoughts/suggestions is appreciated.

import transformers
import torch
from transformers import (AutoModelForSequenceClassification, AutoTokenizer, AutoModelForQuestionAnswering,
 AutoModelForTokenClassification, AutoConfig)

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
config = AutoConfig.from_pretrained('roberta-base',num_labels=2,torchscript=True)
model = AutoModelForSequenceClassification.from_pretrained('roberta-base', config=config)
tokenizer = AutoTokenizer.from_pretrained('roberta-base',do_lower_case=True)
dummy_input = "This is a dummy input for torch jit script"
max_length = 20
inputs = tokenizer.encode_plus(dummy_input,max_length = int(max_length),pad_to_max_length = True, add_special_tokens = True, return_tensors = 'pt')
input_ids = inputs["input_ids"]
attention_mask = inputs["attention_mask"]
outputs = model(**inputs)
model.to(device).eval()
scripted_model = torch.jit.script(model)

Error logs
HF-scripting-error.logs.txt

cc @gmagogsfm

[eellison]

Work is ongoing in this direction by @nikithamalgifb

[isgursoy]

following

[issamemari]

Very interested

[adampauls]

following

[MrRace]

Sad ~ Who can share the solution? Thanks a lot!