How to train an NER model with different beam parameters?

[bburns632]

I'm attempting to update a pre-trained spacy model en_core_web_md with a few rounds of a beam objective other than beam_width = 1, and I cannot seem to find the right way to pass the different parameters into the **cfg such that the model uses them for training (at THIS point).

This was my latest attempt:

pipe_exceptions = ["ner", "trf_wordpiecer", "trf_tok2vec"]
other_pipes = [pipe for pipe in nlp.pipe_names if pipe not in pipe_exceptions]
# only train NER
with nlp.disable_pipes(*other_pipes), warnings.catch_warnings():
    # show warnings for misaligned entity spans once
    warnings.filterwarnings("once", category=UserWarning, module='spacy')

    # TRY TO FORCE BEAM TRAINING INSTEAD OF GREEDY METHOD
    nlp.use_params({'ner':{'beam_width':16, 'beam_density':0.0001}})
    print(nlp.meta) 

    sizes = compounding(1.0, 4.0, 1.001)
    # batch up the examples using spaCy's minibatch
    for itn in range(n_iter):
        random.shuffle(TRAIN_DATA_2)
        batches = minibatch(TRAIN_DATA_2, size=sizes)
        losses = {}
        for batch in batches:
            texts, annotations = zip(*batch)
            nlp.update(texts, 
            annotations, 
            sgd=optimizer, 
            drop=0.35, 
            losses=losses
            )
        print("Losses", losses)

However, after training, the model/ner/cfg file still lists:

{
"beam_width":1,
"beam_density":0.0,
"beam_update_prob":1.0,
...

I have also tried nlp.resume_training( beam_width=16, beam_density=0.0001) and spacy.load(model, beam_width=16, beam_density=0.0001) . Both complete a model run, but the cfg file in the results show.

I also tried nlp.update(texts, annotations, sgd=optimizer, drop=0.35, losses=losses, component_cfg = {'ner':{'beam_width':16, 'beam_density':0.0001}}) which fails to run, because 'ner.update' does not allow additional arguments (src code)

So, I have a few questions:

1. Am I able to update an existing greedy trained model with a new beam objective?
2. If true, how can I properly changed these training parameters (and confirm they are changed)?
3. If false, for a new from scratch model, how can I properly changed these training parameters (and confirm they are changed)?

Why do this?
I am attempting to train a model that provides probabilities for NER decisions that I can surface to my users. THIS post and a few others show how to use beam_parse to obtain probabilities after the fact from a greedy model. However, they all mention that the greedy model hasn't been trained with a global objective, so these scores aren't especially meaningful unless you also perform some iterations of beam training as well. (link to github issue)

Note
This was first posted to stackoverflow.

Your Environment

• spaCy version: 2.3.1
• Platform: Darwin-19.5.0-x86_64-i386-64bit
• Python version: 3.6.10

[bburns632]

I found that the syntax to change config parameters is:

nlp.entity.cfg['beam_width'] = 16
nlp.entity.cfg['beam_density'] = 0.0001

See this stackoverflow post as well.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.