Evaluation of conflicting predictions.

[JohannesEschbach]

Consider the gold standard "Paris [B-PER] Hilton [I-PER] visits [O] Paris [B-LOC]".
UniNER retrieves "Paris" when asked for LOCATION. Is this label now applied to all occurences of "Paris" in the text? If the model now also retrieves "Paris Hilton" for the entity type PERSON, we no longer have a unique assignment of entity types to tokens. How did you handle such label conflicts in your evaluation when computing the recall?