-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wrongly picking negations in negaspcy? #14
Comments
Hi @Raghu17s - I've run across this issue too, especially with scispacy as the language model. The NER is making the entity span "no headache" instead of just "headache". Check out the example here: https://github.com/jenojp/negspacy#negations-in-noun-chunks Basically just modify this line and it should work as you expect.
|
Thank you. I have observed one strange thing. If I add comma in between no and headache (no, headache instead of no headache) it picks as negation. But this cannot be done manually everytime. |
I see, that makes sense because the scispaCy NER model does not treat it as 1 entity because of the comma. Yes you should add other words that you're having the problem with based on the NER. You should be careful to not be too greedy there since in the biomedical domain, many entities could "start with" a negation-like word (e.g., non hodgkin's lymphoma). |
Hi @jenojp, if self.chunk_prefix: the check is only on the first word (i.e., doc[e.start]) and therefore 'free of' is not appropriately handled. |
@stefano-marchesin, that's a good catch. I hadn't had any composite terms pop up in my use cases but I can definitely see that being an issue. Could you share an example entity that this is happening on? I'm assuming you're using scispacy? |
Hi @jenojp! yes, I am using scispacy and within negex the language param is set to "en_clinical". As an example, consider the following: "resection margins free of dysplasia" In this case, free of dysplasia is considered as a single entity and Negex fails to catch "free of" as chunk_prefix. As a workaround I did the following (starting at line 307 of negation.py): if self.chunk_prefix: which solved the problem for me. |
On a side note, I've also noticed that for the same mention of the previous comment -- i.e., "resection margins free of dysplasia" -- negex finds also "resection margins" as a negative mention. I believe this is related to the "free" pattern included within following_negations. However, is there any chance to remove this behavior? It would create inconsistencies. |
Hey can you give me a de-id test block of text to work with? I want to try and recreate what you're seeing. |
Also, you're able to add and remove patterns on the fly in 0.1.8. So you can remove "free" simply by doing the following:
|
Issue #14 (compound chunk prefixes) and remove patterns bug fix
Closed with release 0.1.9 |
Hi,
I am trying to find negations in a sentence using negspacy. But, it's printing first negation (no headache) as False which is supposed to be True and picking the second negation correctly. Should I fine-tune any parameters to get the first negation correctly?
Here is my code.
Output:
The text was updated successfully, but these errors were encountered: