hi,when I train wav2vec-xlsr, prompt “you should pass attention_mask when working with Wav2Vec2 new checkpoints”，what should I do? When do you plan to add the attention_mask for train?

[dengcunqin]

hi,when I train wav2vec-xlsr, prompt “you should pass attention_mask when working with Wav2Vec2 new checkpoints”，what should I do? When do you plan to add the attention_mask for train?

[thevasudevgupta]

Hello @dengcunqin,

Are you using this script?
If so, this script won't work with gsoc-wav2vec2-xlsr-53 and is only compatible with gsoc-wav2vec2-960h.

The major difference in both ckpts (in terms of usage) is that gsoc-wav2vec2-xlsr-53 also accepts attention_mask while gsoc-wav2vec2-960h doesn't.

You can use gsoc-wav2vec2-xlsr-53 like this:

from wav2vec2 import Wav2Vec2ForCTC

model = Wav2Vec2ForCTC.from_pretrained("vasudevgupta/gsoc-wav2vec2-xlsr-53")

speech: tf.Tensor
attention_mask: tf.Tensor # 0 for padding positions & 1 otherwise

outputs = model(speech, attention_mask=attention_mask)

In case you are using this script, you will have to change the dataloading part appropriately so that dataset object also returns attention_mask; then then probably you will be able to use that script.

[dengcunqin]

Do you have any plans for update training code for gsoc-wav2vec2-xlsr-53?

[thevasudevgupta]

Hey, I am little busy with some other project. So, wouldn’t be able to find time for this sometime soon.

[dengcunqin]

get

[thevasudevgupta]

Closing this issue then!