doubt in metric.py

[udion]

I think get_ner_BIO() in metric.py is wrong.

consider the example where label_list = [I-MISC, I-MISC, O, I-PER, I-PER, O, O, O, O, O I-ORG, O] according to current function the following will happen :

Since there is no tag involving B-, whole_tag and tag_index will always be [] and hence the output of the function is [] which is wrong?

[udion]

infact in the above case tag_list will have only one null string

[jiesutd]

The metric is correct. You use the wrong data format, only BIO/BMES format are supported.

You need do more homework before doubt others code.

[udion]

@jiesutd Yes, thank you for pointing that out, the above example is not even correct with respect to BIO or BIOES as first I-MISC is followed by another I-MISC, I just checked and the dataset I was using had other such sentences, I corrected it to BIOES format and It seems to be giving f-score (!=-1)

However, I have no reference to check if my converted dataset is correct, do you have such reference? Does CoNLL website provide BIO and BIOES format as well? (I don't have their dataset directly as it requires some manual authentication/NDA process)

comments?

[jiesutd]

You can refer this paper https://arxiv.org/pdf/1707.06799.pdf for the difference between BIO/BIOES/IOB.