Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IndexError in run_classifier.py::MrpcProcessor::_create_examples (2) #551

Open
alter-bug-tracer opened this issue Apr 4, 2019 · 2 comments
Open

Comments

@alter-bug-tracer
Copy link

@alter-bug-tracer alter-bug-tracer commented Apr 4, 2019

What: "IndexError: list index out of range"

Location: MrpcProcessor::_create_examples function from run_classifier.py

Reason: Missing input validation when reading the lines: interpreting a newline as a line with at least 5 elements.

Steps to reproduce:
Corrupt the "glue_data/MRPC/train.tsv" by adding a trailing newline character at the end of the file (see attached file).
Run
python run_classifier.py \
--task_name=MRPC \
--do_train=true \
--do_eval=true \
--data_dir=$GLUE_DIR/MRPC \
--vocab_file=$BERT_BASE_DIR/vocab.txt \
--bert_config_file=$BERT_BASE_DIR/bert_config.json \
--init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \
--max_seq_length=128 \
--train_batch_size=32 \
--learning_rate=2e-5 \
--num_train_epochs=3.0 \
--output_dir=/tmp/mrpc_output/

Traceback:
Traceback (most recent call last):
File "bert/run_classifier.py", line 981, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "bert/run_classifier.py", line 842, in main
train_examples = processor.get_train_examples(FLAGS.data_dir)
File "bert/run_classifier.py", line 302, in get_train_examples
self._read_tsv(os.path.join(data_dir, "train.tsv")), "train")
File "bert/run_classifier.py", line 325, in _create_examples
text_a = tokenization.convert_to_unicode(line[3])
IndexError: list index out of range

list_ioob2.zip

@himanshututeja1998

This comment has been minimized.

Copy link

@himanshututeja1998 himanshututeja1998 commented Jun 7, 2019

Same error please help out I am also struck on this please anyone help

@clayms

This comment has been minimized.

Copy link

@clayms clayms commented Jul 15, 2019

Supposedly you can change the following to return a list with the number of classes in your particular case.

bert/run_classifier.py

Lines 354 to 356 in 0fce551

def get_labels(self):
"""See base class."""
return ["0", "1"]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.