about transformers version compatibility

[jianbohuang]

Dose it compatible with higher transformers version?

if run it in my env of verison : transformers 4.28.0
then it occurs an error:

./models/bert.py", line 229, in forward
RuntimeError: The size of tensor a (3) must match the size of tensor b (9) at non-singleton dimension 0

[xinyu1205]

Thank you for your very valuable feedback. I will check this issue and give you a response as soon as possible in next few days.

[deyiluobo]

I had the same problem, I lowered the version of transformers to no avail

[xinyu1205]

The reason behind the issue is that our code has been modified based on BLIP. To resolve the issue quickly, you can refer to a simple solution provided in this GitHub comment: salesforce/BLIP#142 (comment).

Further modifications are required in the Tag2Text/models/bert.py file to align it with the new version of the transformer. I have added this to my pending tasks list, but due to my current workload, I cannot ensure completion as soon as possible. Really hope you can understand.

Sincerely Thank you once again for bringing this issue to my attention.

[bloodhunt3r]

replace the following code:

https://github.com/xinyu1205/Tag2Text/blob/9f6866e115ed3026d748bc67de67a9d428df1016/models/bert.py#L224-L229

with the following code to do alignment:

        if key_layer.shape[0] > query_layer.shape[0]:
            key_layer = key_layer[:query_layer.shape[0], :, :, :]
            attention_mask = attention_mask[:query_layer.shape[0], :, :]
            value_layer = value_layer[:query_layer.shape[0], :, :, :]
        attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))

[xinyu1205]

Thank you very much for sharing! If you have already tested the corresponding version, you are very welcome to be one of the contributors to this project by initiating Pull Requests (please add comments in the corresponding area). We appreciate your willingness for sharing and look forward to your contributions.

[Qiliqing]

replace the following code:

https://github.com/xinyu1205/Tag2Text/blob/9f6866e115ed3026d748bc67de67a9d428df1016/models/bert.py#L224-L229

with the following code to do alignment:

        if key_layer.shape[0] > query_layer.shape[0]:
            key_layer = key_layer[:query_layer.shape[0], :, :, :]
            attention_mask = attention_mask[:query_layer.shape[0], :, :]
            value_layer = value_layer[:query_layer.shape[0], :, :, :]
        attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))

It works, thx!