Merge highlighted text when neighbors

[payoto]

In the previous implementation dictionary inputs were not correctly
processed meaning: a None token was added on the empty string between
each entity. That None entry is removed letting neighbouring entities
be merged.

Description

Please include:

• relevant motivation
• a summary of the change
• which issue is fixed.
• any additional dependencies that are required for this change.

Closes: # (issue)

Checklist:

• I have performed a self-review of my own code
• I have added a short summary of my change to the CHANGELOG.md
• My code follows the style guidelines of this project
• I have commented my code in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

A note about the CHANGELOG

Hello 👋 and thank you for contributing to Gradio!

All pull requests must update the change log located in CHANGELOG.md, unless the pull request is labeled with the "no-changelog-update" label.

Please add a brief summary of the change to the Upcoming Release > Full Changelog section of the CHANGELOG.md file and include
a link to the PR (formatted in markdown) and a link to your github profile (if you like). For example, "* Added a cool new feature by [@myusername](link-to-your-github-profile) in [PR 11111](https://github.com/gradio-app/gradio/pull/11111)".

If you would like to elaborate on your change further, feel free to include a longer explanation in the other sections.
If you would like an image/gif/video showcasing your feature, it may be best to edit the CHANGELOG file using the
GitHub web UI since that lets you upload files directly via drag-and-drop.

[freddyaboulton]

Thanks for the PR @payoto !

This test is no longer passing with your change. Can you fix?

[payoto]

Yes I can, FYI the fix I can see would be to edit the reference result which includes some of the ('', None), is that an ok fix?

Entries are still only merged when the user passes the specific argument.

This change does include a change in the behaviour of the postprocess method. An alternative way to fix this bug which would not change the default behavior would be to ignore entries which are ('', None) when merging neighbors. Which solution do you prefer?

Personally I think the first one is the way to go but you certainly have more context as to how these features might be used than I do.

[payoto]

Ok in the end I went with the change that creates least changes of behaviour, so I do the merge by not considering the entries which say ('', None) in the merge strategy

[freddyaboulton]

Just to be clear - users would still need to set combine_adjacent=True right?

[payoto]

Yes, that's correct

[abidlabs]

Thanks so much for the PR @payoto! So that we can test it out, can you provide an example of an example that was being processed incorrectly previously?

[payoto]

Merging on dictionary inputs would simply not work:

from transformers import pipeline
import gradio as gr

nlp = pipeline("ner", model="dslim/bert-base-NER")
example = "My name is Sylvain and I live in Berlin"

ner_results = nlp(example)
print(ner_results)

gr.Interface(
    lambda p: dict(text=p, entities=nlp(p)),
    inputs=gr.Textbox(),
    outputs=gr.HighlightedText(combine_adjacent=True,),
    examples=[example],
).launch()

Raw input

[{'entity': 'B-PER', 'score': 0.9987564, 'index': 4, 'word': 'S', 'start': 11, 'end': 12}, {'entity': 'B-PER', 'score': 0.93149716, 'index': 5, 'word': '##yl', 'start': 12, 'end': 14}, {'entity': 'B-PER', 'score': 0.79625636, 'index': 6, 'word': '##va', 'start': 14, 'end': 16}, {'entity': 'B-PER', 'score': 0.8565875, 'index': 7, 'word': '##in', 'start': 16, 'end': 18}, {'entity': 'B-LOC', 'score': 0.9995517, 'index': 12, 'word': 'Berlin', 'start': 33, 'end': 39}]

Before the patch:

After the patch:

[abidlabs]

Thanks for making the PR @payoto! Was thinking through this PR and while this does fix the bug, it treats the None label differently than the labels for the other entities. It would be better if we could treat all of the entities the same, and I think we can do that by changing the condition from elif text == "" and category is None: to elif text == "" (or the more Pythonic elif not text).

Let me know what you think of the suggestion. I also added some suggestions for the tests.

If this looks good, please go ahead and make the suggestions, and happy to review again. cc @freddyaboulton

[freddyaboulton]

This looks good to me @payoto ! Thank you for addressing the changes and I can confirm it now properly merges adjacent entries. Please fix the linting bash scripts/format_backend.sh and I will merge 🚀

This branch

Main

[abidlabs]

Merging, thank you so much @payoto for this fix.