Reproduce spaCy training results

[ykeuter]

It would be nice if there is a way to reproduce training result in spaCy. Below snippet trains a trivial NER component, but shows different results in separate runs, even while using fix_random_seed. Is this expected behaviour?

import spacy

spacy.util.fix_random_seed()

nlp = spacy.blank("en")

ner = nlp.create_pipe("ner")
ner.add_label("TEST")
nlp.add_pipe(ner)

losses = {}
nlp.begin_training()
nlp.update(
    ["test"],  # batch of texts
    [{"entities": [(0, 4, "TEST")]}],  # batch of annotations
    losses=losses,
)
print(losses)

Environment:

* **spaCy version:** 2.0.18
* **Platform:** Linux-4.9.125-linuxkit-x86_64-with-debian-buster-sid
* **Python version:** 3.6.7
* **Models:** nl_core_news_sm

[ellenir]

I am having the same issue with spacy.util.fix_random_seed() I am trying to train the model for text categorization however (using textcat).

  `  import spacy
    spacy.util.fix_random_seed()
    nlp = spacy.load('en_core_web_sm')
    textclass = nlp.create_pipe('textcat')
    nlp.add_pipe(textclass, last=True)
    textclass.add_label('NotAtFault')
    
   
    optimizer = nlp.begin_training()`

I have experimented with the location of the seed setting but to no avail.

For me:

spaCy version: 2.0.18
Platform: Windows 10
Python version: 3.6.8
Models: en_core_web_sm

[honnibal]

The three sources of randomization should be:

numpy.random.seed(i)
cupy.random.seed(i)

I think currently cupy.random.seed isn't being set, so if you're using GPU, that would be the problem.

[ines]

Closing this, since this is already fixed on develop!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.