a problem of code

[tengwang0318]

I just try to run the code, I download the dataset, but I got an error about "TypeError: linear(): argument 'input' (position 1) must be Tensor, not str". I check the code, and I find the two lines code maybe have a mistake.

I just change them to

output = self.bert(input_ids=input_ids)
return output.last_hidden_state

but I still got the same mistake, I don't know why.
Here is my colab code
I would appreciate it if you could help me again!

[hasanhuz]

Hi, are you using the same Transformer version as indicated in the requirement file?

[tengwang0318]

Because the Transformer version is too old, so I can't install it.

[hasanhuz]

I cannot replicate the error because I still have the old version. I suggest you the following and hope it helps: change line#26 in the above image as
"last_hidden_state = self.bert(input_ids)[0]".

Please, make sure the shape of a returned tensor is (batch, seq_length, hidden_dim).

[tengwang0318]

OK, thank you! I will try this!

[hasanhuz]

You're welcome, please let me know if it works. I may update my code for those who have newer versions of the Transformer!

Thanks :)

[tengwang0318]

I think it doesn't work.
For example,

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = BertModel.from_pretrained('bert-base-uncased')
input_ids = torch.LongTensor([[31, 51, 99], [15, 5, 0]])
output = model(input_ids)
print(output)

then output is

BaseModelOutputWithPoolingAndCrossAttentions(last_hidden_state=tensor([[[-0.3521, -0.7047, -0.6663,  ...,  0.4252,  0.7841,  0.9555],
         [-0.1795, -0.7452, -0.3822,  ...,  0.2929,  0.9957,  0.9606],
         [-0.2068, -0.6080, -0.3451,  ...,  0.1681,  0.9724,  0.7488]],

        [[-0.3532, -0.0907, -0.2980,  ...,  0.1295,  0.4900,  0.3955],
         [-0.3334, -0.0261, -0.1224,  ...,  0.2188,  0.4691,  0.3899],
         [-0.2931,  0.0495, -0.2581,  ...,  0.2843,  0.5286,  0.3704]]],
       grad_fn=<NativeLayerNormBackward>), pooler_output=tensor([[-0.0852,  0.0362,  0.1649,  ..., -0.4534, -0.2865,  0.4123],
        [ 0.0966,  0.0838,  0.0168,  ..., -0.4515, -0.2616,  0.1139]],
       grad_fn=<TanhBackward>), hidden_states=None, past_key_values=None, attentions=None, cross_attentions=None)

you will get a dict type of output, and you can output.last_hidden_state to get last hidden state

[hasanhuz]

I have just updated the model script to handle both old and new versions of Transformers.
Please, test it and let me know if it doesn't work!

[hasanhuz]

It seems to me that you have added something to the learner script, right? This error is quite weird since it's not relevant to any part of my codes!

[tengwang0318]

my fault! Sorry, maybe I have made a mistake! I'm sorry!
The code after updating can run without error. Thank you so much! You are the most awesome people I have seen on Github!

[tengwang0318]

I'm very curious about why your code can work well, but my code can't. Do you just modify this place? Maybe when I use colab to run code, I don't update the edited file successfully.

Have a good day!

[hasanhuz]

Glad to heat that it works fine for you now :)
All the best.

[tengwang0318]

I just modify your code to run SemEval 2014 task4, I use ['food', 'anecdotes', 'service', 'ambience', 'price'] as aspects like emotions, but I can't use encode_plus method like this

I don't know why it split 'anecdotes' to 'an', '##ec', '##dote', and '###s'. I use encode_plus for a long time and I never met this problem.

[tengwang0318]

After I change anecdotes and ambience to another word, respectively. It works well! I think maybe its vocabulary doesn't have these words.

[hasanhuz]

Yes, Transformers-based models use Word-Piece tokenizer, which can split words that don't exit in the vocabulary set into pieces as you got in your example ('anecdotes' to 'an', '##ec', '##dote', and '###s'). This is how such models deal with OOV!

[tdlbj]

The current time to open the file name. Why does it exist