DataCollatorForWholeWordMask does not work with return_tensors = "np" and return_tensors = "tf"

[dwyatte]

Environment info

• transformers version: 4.11.2
• Platform: Linux-4.19.0-14-cloud-amd64-x86_64-with-Ubuntu-18.04-bionic
• Python version: 3.7.5
• PyTorch version (GPU?): 1.9.1+cu102 (False)
• Tensorflow version (GPU?): 2.6.0 (False)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: No
• Using distributed or parallel set-up in script?: No

Who can help

@Rocketknight1

Information

The problem arises when using:

• the official example scripts: (give details below)
• my own modified scripts: (give details below)

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name)
• my own task or dataset: (give details below)

I'd like to use DataCollatorForWholeWordMask for masked language modeling in TensorFlow.

To reproduce

return_tensors = "np"

from transformers.data import DataCollatorForWholeWordMask
from transformers import BertTokenizerFast

return_tensors = "np"
tokenizer = BertTokenizerFast.from_pretrained("bert-base-uncased")
collate_fn = DataCollatorForWholeWordMask(tokenizer, return_tensors=return_tensors, mlm=True)
collate_fn([{"input_ids": list(range(10))}, {"input_ids": list(range(10))}])

TypeError: numpy_mask_tokens() got an unexpected keyword argument 'special_tokens_mask'

This is at least partly due to not implementing DataCollatorMixin.numpy_call in DataCollatorForWholeWordMask (np_call is implemented instead). There are also some issues with using TensorFlow tensors in the np_call function.

return_tensors = "tf"

from transformers.data import DataCollatorForWholeWordMask
from transformers import BertTokenizerFast

return_tensors = "tf"
tokenizer = BertTokenizerFast.from_pretrained("bert-base-uncased")
collate_fn = DataCollatorForWholeWordMask(tokenizer, return_tensors=return_tensors, mlm=True)
collate_fn([{"input_ids": list(range(10))}, {"input_ids": list(range(10))}])

AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'clone'

inputs.clone() appears to have been copied from torch_call which is not valid in TensorFlow.