Trouble downloading Spacy models in ubuntu 14.04 Amazon EC2

[jesushd12]

Hi, I'm trying to download spacy and the spanish models in ubuntu 14.04 Amazon EC2, I'm getting errors while downloading:
Following the instructions on the web:

I activated my virtualenv and then execute the following command:
pip install -U spacy
After 5 minutes
I get a lot of
:#warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION"

afterwards it said:

Successfully installed spacy numpy murmurhash cymem preshed thinc plac pip six pathlib ujson dill requests regex ftfy wrapt tqdm cytoolz termcolor certifi idna chardet urllib3 html5lib wcwidth toolz setuptools webencodings
Cleaning up...

so I proceeded to download the spanish model and ran the following command ( In this step I have 1.2 gb available):

python -m spacy download es

It started to download the es_core_web_md-1.0.0.tar.gz from github (395mb) and get the following errors:

  Running setup.py install for es-core-web-md ... error
    Complete output from command /home/ubuntu/development/backend_qas_jesus/env_qa/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-41HcDP-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-fE_EOd-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/ubuntu/development/backend_qas_jesus/env_qa/include/site/python2.7/es-core-web-md:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/es_core_web_md
    copying es_core_web_md/__init__.py -> build/lib.linux-x86_64-2.7/es_core_web_md
    creating build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0
    copying es_core_web_md/es_core_web_md-1.0.0/meta.json -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0
    creating build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/ner
    copying es_core_web_md/es_core_web_md-1.0.0/ner/config.json -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/ner
    copying es_core_web_md/es_core_web_md-1.0.0/ner/model -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/ner
    creating build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/pos
    copying es_core_web_md/es_core_web_md-1.0.0/pos/config.json -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/pos
    copying es_core_web_md/es_core_web_md-1.0.0/pos/model -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/pos
    creating build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/vocab
    copying es_core_web_md/es_core_web_md-1.0.0/vocab/oov_prob -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/vocab
    copying es_core_web_md/es_core_web_md-1.0.0/vocab/lexemes.bin -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/vocab
    copying es_core_web_md/es_core_web_md-1.0.0/vocab/strings.json -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/vocab
    copying es_core_web_md/es_core_web_md-1.0.0/vocab/vec.bin -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/vocab
    creating build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/deps
    copying es_core_web_md/es_core_web_md-1.0.0/deps/config.json -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/deps
    copying es_core_web_md/es_core_web_md-1.0.0/deps/model -> build/lib.linux-x86_64-2.7/es_core_web_md/es_core_web_md-1.0.0/deps
    error: [Errno 28] No space left on device
    
    ----------------------------------------
Command "/home/ubuntu/development/backend_qas_jesus/env_qa/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-41HcDP-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-fE_EOd-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/ubuntu/development/backend_qas_jesus/env_qa/include/site/python2.7/es-core-web-md" failed with error code 1 in /tmp/pip-41HcDP-build/
/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/lib/python2.7/site-packages/spacy/__main__.py", line 133, in <module>
    plac.Interpreter.call(CLI)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/plac_ext.py", line 1142, in call
    print(out)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/plac_ext.py", line 914, in __exit__
    self.close(exctype, exc, tb)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/plac_ext.py", line 952, in close
    self._interpreter.throw(exctype, exc, tb)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/plac_ext.py", line 964, in _make_interpreter
    arglist = yield task
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/plac_ext.py", line 1139, in call
    raise_(task.etype, task.exc, task.tb)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/plac_ext.py", line 380, in _wrap
    for value in genobj:
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/plac_ext.py", line 95, in gen_exc
    raise_(etype, exc, tb)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/plac_ext.py", line 966, in _make_interpreter
    cmd, result = self.parser.consume(arglist)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/plac_core.py", line 207, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/lib/python2.7/site-packages/spacy/__main__.py", line 33, in download
    cli_download(model, direct)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/spacy/cli/download.py", line 24, in download
    link_package(model_name, model, force=True)
  File "/home/ubuntu/development/backend_qas_jesus/env_qa/local/lib/python2.7/site-packages/spacy/cli/link.py", line 22, in link_package
    pkg = importlib.import_module(package_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named es_core_web_md

In my local machine works well but when triying to deploy on the amazon server it happen this. Could it be a space disk problem?.

I would appreciate any help.
Thanks in advance.

My Environment

## Info about spaCy

* **Python version:** 2.7.6
* **Platform:** Linux-3.13.0-91-generic-x86_64-with-Ubuntu-14.04-trusty
* **spaCy version:** 1.9.0
* **Installed models:** 

[ines]

Could it be a space disk problem?

Possibly – in the error message shown after copying the files, it says:

error: [Errno 28] No space left on device

The good news is, this hopefully won't be a problem anymore with spaCy v2.0, as the models are much smaller. You can already install the new version (currently in alpha, but very close to a release candidate) via pip install spacy-nightly. For more info on the new version, check out the alpha documentation.

The Spanish neural network model for v2.0, es_core_web_sm is only 34 MB. See here for details: http://alpha.spacy.io/models/es

[jesushd12]

Hi Ines, thanks for answering.
I tried install spacy-nightly. But I get the following error:

----------------------------------------
Command "/home/ubuntu/development/backend_qas_jesus/env_qa/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-8loaJT/spacy-nightly/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-QUcHrQ-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/ubuntu/development/backend_qas_jesus/env_qa/include/site/python2.7/spacy-nightly" failed with error code -9 in /tmp/pip-build-8loaJT/spacy-nightly/

[jesushd12]

I also tried to install it in a new enviroment and I got the following:

$ pip install spacy-nightly
Downloading/unpacking spacy-nightly
  Could not find a version that satisfies the requirement spacy-nightly (from versions: 2.0.0a8, 2.0.0a4, 2.0.0a2, 2.0.0a13, 2.0.0a6, 2.0.0a10, 2.0.0a5, 2.0.0a3, 2.0.0a16, 2.0.0a7, 2.0.0a17, 2.0.0a11, 2.0.0a12, 2.0.0a15, 2.0.0a0, 2.0.0a14, 2.0.0a9)
Cleaning up...
No distributions matching the version for spacy-nightly
Storing debug log for failure in /home/ubuntu/.pip/pip.log

[jesushd12]

Hi again @ines ,
I already installed spacy-nightly. I had to enable install pre-release using the command --pre:
pip install spacy-nightly --pre .
However, the similarity function is not working as in previus version.

>>> doc1 = nlp(u'El hombre pasea al perro')
>>> doc2 = nlp(u'El hombre pasea al gato')
>>> doc1.similarity(doc2)
0.0

Also if I use the token instead sentence I get:
IndexError: index 0 is out of bounds for axis 0 with size 0
Is there a bug with the similarity function in this alpha??

grettings.

[ines]

@jesushd12 Thanks for updating with your solution! (I didn't know about the --pre setting in pip and it hasn't come up before. So this is good to know.)

And yes, the vectors aren't wired up yet in the current version of the models (see #1457). But they will be included in the stable release. So if Spanish vectors are important to you, you might have to go back to using the v1.x model or wait until we have the new models trained. (Sorry about that, I should have asked about the features you need!)

[jesushd12]

ohh I see, well I will try to increase the disk space on the server and try to install the previous version until the stable version is available. Thank you for your time @ines .

[jhogan]

I was having many issues installing spaCy on a T2.nano. I noticed that MySQL was using up a lot of memory and that the spaCy installation process was consuming the remainder of physical memory. I also noticed that there is no swap drive on the T2.nano that I was using.

I freed up some memory by shutting down the the MySQL daemon. I also used the --no-cache-dir option to pip3 (not sure if this did anything).

I kept an eye on the free memory I had when installing spaCy, and it seems I had just enough to get it installed.

However, it did finish with the below error:

Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-hwwe9ha5/spacy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-__74dozt-record/install-record.txt --single-version-externally-managed --compile" failed with error code -9 in /tmp/pip-build-hwwe9ha5/spacy/

I'm not sure what the above means, but I am able to import spacy in the python3 interactive console.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.