STOP_NER stops Locations from being found with rules

[maxaalexeeva]

While we were working on the migration branch, @zupon wrote a few backoff rules to get the locations that are not found by the neural net:

eidos/src/main/resources/org/clulab/wm/eidos/english/grammars/context.yml

Line 30 in 45dbf5f

- name: location-backoff

It looks like (at least) this is stopping these rules from working:

eidos/src/main/scala/org/clulab/wm/eidos/extraction/RuleBasedEntityFinder.scala

Line 77 in 45dbf5f

// Make sure there is a noun that isn't a named entity. We can also check for stop words with some re-architecting...

eidos/src/main/scala/org/clulab/wm/eidos/extraction/RuleBasedEntityFinder.scala

Line 80 in 45dbf5f

!StopwordManager.STOP_NER.contains(entities(i))

because 'LOCATION' is included in the list of the NER stop labels:

eidos/src/main/scala/org/clulab/wm/eidos/utils/StopwordManager.scala

Line 106 in f570783

val STOP_NER: Set[String] = Set("DATE", "DURATION", "LOCATION", "MONEY", "NUMBER", "ORDINAL", "ORGANIZATION", "PERSON", "PLACE", "SET", "TIME")

What would be the way to go about this? Taking locations out of the stop word list? Making a more fine-grained check for what entities to filter out and which ones to keep? Let it be and just keep missing some of the locations? Some other option?

Here's an example:

a) with entities filtered out:

b) with entities not filtered out (the locations are found with the location-backoff rule):

[BeckySharp]

Those methods were designed to make sure that causal relations weren't returned where one of the arguments (i.e., the cause or the effect) has no content *aside from* things like NEs... I think that they are only run over the final output... do we want to return all locations, whether they occur with an event/relation of interest or not? If so, we can do that, probably here: https://github.com/clulab/eidos/blob/master/src/main/scala/org/clulab/wm/eidos/utils/StopwordManager.scala#L59 by first separating out Locations, then running the rest on the leftovers, then returning Locations and the filter survivors... but again, IDK if we want to do that In the webapp, there's a checkbox that has "CAG relevant only" or something, does checking/unchecking affect the display of these? If not -- the problem isn't where I thought it was...

…

On Sun, Jan 26, 2020 at 12:07 PM Maria (Masha) Alexeeva < ***@***.***> wrote: While we were working on the migration branch, @zupon <https://github.com/zupon> wrote a few backoff rules to get the locations that are not found by the neural net: https://github.com/clulab/eidos/blob/45dbf5fffb6b58659644c543ec1142a641184d8b/src/main/resources/org/clulab/wm/eidos/english/grammars/context.yml#L30 It looks like (at least) this is stopping these rules from working: https://github.com/clulab/eidos/blob/45dbf5fffb6b58659644c543ec1142a641184d8b/src/main/scala/org/clulab/wm/eidos/extraction/RuleBasedEntityFinder.scala#L77 https://github.com/clulab/eidos/blob/45dbf5fffb6b58659644c543ec1142a641184d8b/src/main/scala/org/clulab/wm/eidos/extraction/RuleBasedEntityFinder.scala#L80 because 'LOCATION' is included in the list of the NER stop labels: https://github.com/clulab/eidos/blob/f570783ad51ee867cf54986f13908b5f0f76e50d/src/main/scala/org/clulab/wm/eidos/utils/StopwordManager.scala#L106 What would be the way to go about this? Taking locations out of the stop word list? Making a more fine-grained check for what entities to filter out and which ones to keep? Let it be and just keep missing some of the locations? Some other option? Here's an example: a) with entities filtered out: [image: Screenshot from 2020-01-26 13-01-08] <https://user-images.githubusercontent.com/31713912/73140976-0b4c8100-403c-11ea-8de6-c72cd8e8c208.png> b) with entities not filtered out (the locations are found with the location-backoff rule): [image: Screenshot from 2020-01-26 13-00-40] <https://user-images.githubusercontent.com/31713912/73140977-0daedb00-403c-11ea-8a7f-ffb0dded7e43.png> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#738?email_source=notifications&email_token=ABJCPCM6MGRBR4QXOUX7VCTQ7XUORA5CNFSM4KLY5WAKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IIY454Q>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJCPCLIZGUFZ6YSPGTIT2LQ7XUORANCNFSM4KLY5WAA> .