TypeError when training a xlmr zero-shot encoder

[JiunHaoJhan]

Hi,

When I try to train an xlmr-base zero-shot encoder, a TypeError occurs. Here is the error message:

Traceback (most recent call last):
  File "/home/massive/scripts/train.py", line 99, in <module>
    main()
  File "/home/massive/scripts/train.py", line 96, in main
    trainer.train()
  File "/home/miniconda3/envs/massive/lib/python3.9/site-packages/transformers/trainer.py", line 1392, in train
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
  File "/home/miniconda3/envs/massive/lib/python3.9/site-packages/transformers/trainer.py", line 1514, in _maybe_log_save_evaluate
    metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
  File "/home/massive/src/massive/utils/trainer.py", line 110, in evaluate
    output = eval_loop(
  File "/home/miniconda3/envs/massive/lib/python3.9/site-packages/transformers/trainer.py", line 2392, in evaluation_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=all_preds, label_ids=all_labels))
  File "/home/massive/src/massive/utils/training_utils.py", line 324, in compute_metrics
    return eval_preds(
  File "/home/massive/src/massive/utils/training_utils.py", line 475, in eval_preds
    convert_to_bio(lab, outside=labels_ignore, labels_merge=labels_merge)
  File "/home/massive/src/massive/utils/training_utils.py", line 423, in convert_to_bio
    bio_tagged.append('I-' + prev_tag)
TypeError: can only concatenate str (not "NoneType") to str

I trace the code and found that prev_tag is None concatenating with the string 'I-' when tag is -100 and labels_merge is '-100'. Could you help check this error?

[FeiWang96]

It seems to be mentioned in a former issue #17 and fixed in latest code #18 .

[cperiz]

Hi @JiunHaoJhan, as @FeiWang96 mentioned, this is addressed in #18. If you pull the latest mainline it should work. Let us know if you still face any issues.

[JiunHaoJhan]

Thanks. It works.