PromptNER based Chain-of-Thought prompting for span tasks

[kabirkhan]

Description

Implements Chain-of-Thought prompting for Span tasks

Based on https://arxiv.org/pdf/2305.15444.pdf

Example Usage:

import spacy

nlp = spacy.blank("en")
nlp.add_pipe(
    "llm",
    name="llm_ner",
    config={
        "task" : {
            "@llm_tasks": "spacy.NER.v3",
            "description": "Entities are the names of people and basketball teams. Names of sports are not entities. Names of soccer, baseball, rubgy, etc teams are not basketball team entities. adjectives are not entities. Pronouns are not persons. Man city is the name of a soccer team so not an entity in this case.",
            "examples": [
                {
                    "text": "The Golden State Warriors beat the LA Lakers by almost 20 points on the back of an amazing performance from Steph Curry.",
                    "entities": [
                         # In the final code these should be able to be provided in the prompt str format
                         # e.g. | Golden State Warriors | BASKETBALL_TEAM | True | is a basketball team in the NBA |
                        {
                            "text": "Golden State Warriors",
                            "label": "BASKETBALL_TEAM",
                            "is_entity": True,
                            "reason": "is a basketball team in the NBA"
                        },
                        {
                            "text": "LA Lakers",
                            "label": "BASKETBALL_TEAM",
                            "is_entity": True,
                            "reason": "is a basketball team in the NBA"
                        },
                        {
                            "text": "20",
                            "label": "==NONE==",
                            "is_entity": False,
                            "reason": "is a number"
                        },
                        {
                            "text": "amazing performance",
                            "label": "==NONE==",
                            "is_entity": False,
                            "reason": "is a description of an event"
                        },
                        {
                            "text": "Steph Curry",
                            "label": "PERSON",
                            "is_entity": True,
                            "reason": "is the name of a person"
                        },
                    ]
                }
            ],
            "labels": "PERSON,BASKETBALL_TEAM",
        },
        "model": {
            "@llm_models": "spacy.GPT-3-5.v1",
            "config": {"max_tokens": 200},
        },
        "save_io": True
    }
)

doc = nlp("The Denver Nuggets beat the Miami Heat 4 games to 1")
print([(ent.text, ent.label_) for ent in doc.ents])

# The names of these sports teams are specifically changed to view the model's reasoning behind labeling them or not.
doc2 = nlp("In other sports news Atletico Madrid was able to tie the Gonzaga bulldogs away from home with a stunning strike from Kevin De Bruyne.")
print([(ent.text, ent.label_) for ent in doc2.ents])

Types of change

Feature

Checklist

• I confirm that I have the right to submit this contribution under the project's MIT license.
• I ran all tests in tests and usage_examples/tests, and all new and existing tests passed. This includes
  • all external tests (i. e. pytest ran with --external)
  • all tests requiring a GPU
• My changes don't require a change to the documentation, or if they do, I've added all required information.

[rmitsch]

We're getting there 🙂 The templates for the other span-based tasks need to be updated still.

[rmitsch]

I think this is ready for a final once-over and running the external tests in the CI.

[svlandeg]

There's still two unresolved conversations, can you have a look at those?

I'd also like us to run a few more tests on how v3 can be used as a substitute for v2, and which changes (if any) would be needed in e.g. the Dolly NER example.

[svlandeg]

This is looking really good, nice team effort all! 🙏

Had a few nits + one more important question about a phrasing in the NER template.