Wrong NER classification for basic words

[ashwinbiyani]

I see apple is classfied as ORG and fruit is classfied as ORG too. Here is the code i am trying

import spacy

nlp = spacy.load('en_core_web_lg')
doc = nlp(u'Apple is a Fruit')

for ent in doc.ents:
print(ent.text, ent.start_char, ent.end_char, ent.label_)

OUTPUT
Apple 0 5 ORG
Fruit 11 16 ORG

Your Environment

• Operating System:
• Python Version Used:
• spaCy Version Used:
• Environment Information:

[ashwinbiyani]

both spacy and python version are latest.

[ines]

The process of recognising named entities is statistical – so the English model has no deeper knowledge of what a "fruit" is, or how an "organization" is defined. It only predicts what label is most likely in this context and which span to apply it to, based on the data the model was trained on.

In this case, "Fruit" is capitalised, which it normally isn't in general-purpose text. So to the model, this looks like a strong candidate for a named entity. If inconsistent capitalization is common in your data, you probably want to fine-tune the model to be less sensitive to that.

It's also important to keep in mind that the pre-trained models distributed with the library are baseline models that were tuned for the best possible compromise of speed, size, and accuracy and make it easy to get started building your own systems. You almost always want to adjust the model to your specific domain if extracting named entities is important to you. You can find more details on this in the documentation on training and updating models.

[ashwinbiyani]

ok, got it. thanks.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.