Merge_noun_chunk trying to merge disjoint spans

[Fourthought]

How to reproduce the behaviour

` There appears to be a problem where overlapping noun_chunks are being created for a particular text. This seems to be a bug within in-built functionality. Reproduction of the problem as follows:

nlp = spacy.load("en_core_web_md")
nlp2 =  spacy.load("en_core_web_md"

merge_nps = nlp.create_pipe("merge_noun_chunks")
nlp.add_pipe(merge_nps)
  
merge_ents = nlp.create_pipe("merge_entities")
nlp.add_pipe(merge_ents)

text taken from: 
https://www.americanrhetoric.com/speeches/gwbushcubaindependence100th.htm
 
doc = nlp(text)

error message: ValueError: [E102] Can't merge non-disjoint spans. 'markets' is already part of 
tokens to merge. If you want to find the longest non-overlapping spans, you can use the 
util.filter_spans helper: https://spacy.io/api/top-level#util.filter_spans

for chunk in nlp(text).noun_chunks:
    if str(chunk).find("markets") != -1:
        print(chunk.start, '|', chunk)

Output: 
1232 | markets      # <= the start of this chunk overlaps with the next
1231 | where markets have brought prosperity`

I should be able to temporarily resolve the problem by creating a custom add-on using filterspans, but thought you would want to know this.

Your Environment

spaCy version: 2.2.4
Platform: Windows-10-10.0.18362-SP0
Python version: 3.7.6

[adrianeboyd]

I think this is more or less a duplicate of #5393. Thanks for the additional example!

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.