num_proc not specified in .map functions

[sandstromviktor]

In trainer.py, there are three .map functions where num_proc is not specified.
It should be possible to set this because it speeds up tokenization, spreading and normalizations by a significant amount.

Example from trainer.py row 227:

        with tokenizer.entity_tracker(split=dataset_name):
            dataset = dataset.map(
                tokenizer,
                batched=True,
                remove_columns=set(dataset.column_names) - set(self.OPTIONAL_COLUMNS),
                desc=f"Tokenizing the {dataset_name} dataset",
                fn_kwargs={"return_num_words": is_evaluate},
                num_proc=4, # Added this - should be specifiable
            )

This sped up tokenization by about 4 times.

[tomaarsen]

Woah, good call! I'd love to look into this.

[sandstromviktor]

I think that this can be set using the huggingface trainingargs dataloader_num_workers argument. If not please tell me why :)

[tomaarsen]

That makes sense, I should've thought of that!