KeyError '23' while running examples

[cipherome-minkim]

Has anyone else seen this while running the examples?

export MODEL_PATH=../examples/ft/prom-core/$KMER
export DATA_PATH=examples
export PREDICTION_PATH=examples
python ../examples/run_finetune.py \
    --model_type dna \
    --tokenizer_name=dna$KMER \
    --model_name_or_path $MODEL_PATH \
    --task_name dnaprom \
    --do_predict \
    --data_dir $DATA_PATH  \
    --max_seq_length 75 \
    --per_gpu_pred_batch_size=128   \
    --output_dir $MODEL_PATH \
    --predict_dir $PREDICTION_PATH \
    --fp16 \
    --n_process 48

error message we are seeing is

12/14/2020 01:54:26 - INFO - transformers.data.processors.glue -   Writing example 0/5
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/ubuntu/DNABERT/src/transformers/data/processors/glue.py", line 120, in glue_convert_examples_to_features
    label = label_map[example.label]
KeyError: '23'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "../examples/run_finetune.py", line 1281, in <module>
    main()
  File "../examples/run_finetune.py", line 1152, in main
    prediction = predict(args, model, tokenizer, prefix=prefix)
  File "../examples/run_finetune.py", line 484, in predict
    pred_dataset = load_and_cache_examples(args, pred_task, tokenizer, evaluate=True)
  File "../examples/run_finetune.py", line 761, in load_and_cache_examples
    features.extend(result.get())
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
KeyError: '23'

[cipherome-minkim]

@Zhihan1996 halp?

[Zhihan1996]

Do you use your own data to finetune the model?

[cipherome-minkim]

Nope. We were running your example with KMER=6.

…

On Wed, Dec 16, 2020 at 5:18 PM Zhihan Zhou ***@***.***> wrote: Do you use your own data to finetune the model? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMR7EOZCVWIE6H4Z5BRC343SVFL6RANCNFSM4U3QHTOA> .

[jerryji1993]

Hi @cipherome-minkim,

Sorry about the delay in response. We have recently performed large-scale bug fixes and updated the test data. Please kindly try and see if the bug still occurs. Thanks!

Jerry