Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when saving new untrained model: 'bool' object is not subscriptable #2482

Closed
wpm opened this issue Jun 26, 2018 · 4 comments
Closed

Error when saving new untrained model: 'bool' object is not subscriptable #2482

wpm opened this issue Jun 26, 2018 · 4 comments

Comments

@wpm
Copy link

@wpm wpm commented Jun 26, 2018

Following the example in Updating the Named Entity Recognizer, I created a blank model and added an named entity recognizer to the pipeline.

import spacy
nlp = spacy.blank('en')
ner = nlp.create_pipe('ner')
nlp.add_pipe(ner, last=True)
nlp.to_disk('model')

I then tried to save the model and got an error.

TypeError                                 Traceback (most recent call last)
<ipython-input-5-a56721e00b6b> in <module>()
----> 1 nlp.to_disk('model')

/anaconda3/envs/keras-spacy/lib/python3.6/site-packages/spacy/language.py in to_disk(self, path, disable)
    619             serializers[name] = lambda p, proc=proc: proc.to_disk(p, vocab=False)
    620         serializers['vocab'] = lambda p: self.vocab.to_disk(p)
--> 621         util.to_disk(path, serializers, {p: False for p in disable})
    622 
    623     def from_disk(self, path, disable=tuple()):

/anaconda3/envs/keras-spacy/lib/python3.6/site-packages/spacy/util.py in to_disk(path, writers, exclude)
    501     for key, writer in writers.items():
    502         if key not in exclude:
--> 503             writer(path / key)
    504     return path
    505 

/anaconda3/envs/keras-spacy/lib/python3.6/site-packages/spacy/language.py in <lambda>(p, proc)
    617             if not hasattr(proc, 'to_disk'):
    618                 continue
--> 619             serializers[name] = lambda p, proc=proc: proc.to_disk(p, vocab=False)
    620         serializers['vocab'] = lambda p: self.vocab.to_disk(p)
    621         util.to_disk(path, serializers, {p: False for p in disable})

nn_parser.pyx in spacy.syntax.nn_parser.Parser.to_disk()

/anaconda3/envs/keras-spacy/lib/python3.6/site-packages/spacy/util.py in to_disk(path, writers, exclude)
    501     for key, writer in writers.items():
    502         if key not in exclude:
--> 503             writer(path / key)
    504     return path
    505 

nn_parser.pyx in spacy.syntax.nn_parser.Parser.to_disk.lambda3()

TypeError: 'bool' object is not subscriptable

If I run update on the model I can save it to disk. Even with an empty training batch like so.

optimizer = self.nlp.begin_training()
nlp.update([], [], sgd=self.nlp.begin_training())

(If I don't add the sgd argument I get the warning "Unnamed vectors -- this won't allow multiple vectors models to be loaded. (Shape: (0, 0))".)

Is this the correct behavior? I would expect that I would be able to save an empty model.

Info about spaCy

  • spaCy version: 2.0.11
  • Platform: Darwin-17.6.0-x86_64-i386-64bit
  • Python version: 3.6.5
  • Models: en_core_web_lg, en
@honnibal honnibal added the bug label Jul 5, 2018
@honnibal

This comment has been minimized.

Copy link
Member

@honnibal honnibal commented Jul 5, 2018

Thanks, yes this is a bug. Before calling begin_training() and without loading a model, spaCy doesn't immediately initialize one. I hadn't tested saving the model in this condition, and it seems we're missing a check.

@ProxyCausal

This comment has been minimized.

Copy link

@ProxyCausal ProxyCausal commented Jul 20, 2018

I don't understand why sometimes when i run train_ner.py under examples, the loss function minimizes, but then it reaches a point then the shell restarts. it is almost like a global minimum couldn't be reached and so the trainer gives up.

Some other details:
I successfully saved to folder before using the same code, but I wanted to update the model with new examples. I figured i didn't have that many so I trained from a new model.

honnibal added a commit that referenced this issue Sep 28, 2018
honnibal added a commit that referenced this issue Sep 28, 2018
@honnibal

This comment has been minimized.

Copy link
Member

@honnibal honnibal commented Sep 28, 2018

@enmatics That sounds like a completely different problem; perhaps related to memory usage?

@wpm Thanks -- fixed on develop now.

@honnibal honnibal closed this Sep 28, 2018
@lock

This comment has been minimized.

Copy link

@lock lock bot commented Oct 28, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Oct 28, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
3 participants
You can’t perform that action at this time.