gold_span object is undefined when training EL pipe

[Joozty]

How to reproduce the behaviour

I bootstraped project from explosion/projects/tutorials/nel_emerson and when I slightly changed create_kb script - diff here I get this error.

⚠ Aborting and saving the final best model. Encountered exception:
AttributeError("'NoneType' object has no attribute 'label_'")
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/cli/_util.py", line 68, in setup_cli
    command(prog_name=COMMAND)
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/cli/train.py", line 59, in train_cli
    train(nlp, output_path, use_gpu=use_gpu, stdout=sys.stdout, stderr=sys.stderr)
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/training/loop.py", line 114, in train
    raise e
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/training/loop.py", line 98, in train
    for batch, info, is_best_checkpoint in training_step_iterator:
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/training/loop.py", line 212, in train_while_improving
    score, other_scores = evaluate()
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/training/loop.py", line 264, in evaluate
    scores = nlp.evaluate(dev_corpus(nlp))
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/language.py", line 1358, in evaluate
    results = scorer.score(examples)
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/scorer.py", line 129, in score
    scores.update(component.score(examples, **self.cfg))
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/pipeline/entity_linker.py", line 405, in score
    return Scorer.score_links(examples, negative_labels=[self.NIL])
  File "/path/Documents/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy/scorer.py", line 534, in score_links
    label = gold_span.label_
AttributeError: 'NoneType' object has no attribute 'label_'

Your Environment

spaCy version 3.0.3
Location /path/open_source/projects/tutorials/nel_emerson/venv/lib/python3.8/site-packages/spacy
Platform Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.29
Python version 3.8.5
Pipelines en_core_web_md (3.0.0)

[svlandeg]

I'll move this to the coding section on the discussions thread and follow up there!

[svlandeg]

The code fails at the point where it tries to retrieve the NER label of a gold span, because the scorer calculates the performance of NEL per NER label type.

Can you share a sample of the gold data you're using to run this project with?

[Joozty]

Can you share a sample of the gold data you're using to run this project with?

Everything is the same in the project except I changed the create_kb script

---

If I get you correctly does it mean if a new entity is created in entity ruler it needs to be also specified here (if it is present in text)?

[svlandeg]

I looked into this some more, and found a small bug in the scoring function for Named Entity Linking. I was able to reproduce the trouble you're having, and it should be fixed by #7074. Thanks for the report!

(it's not always easy to tell whether something is a bug or not before really diving in. In retrospect, I should have kept this thread on the original issue tracker, but either way the result is the same and it's getting fixed :-))

[svlandeg]

FYI - the fix is now merged and will be available in the next 3.0.4 release!