It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.

[bozzmob]

First of all, thank you so much for creating one of the most useful repos. Your time and work is highly appreciated.

Coming to the issue,
I am trying to use the latest Spacy 2.2.3 using the ARN provided-

{
        "package_version": "2.2.3",
        "package": "spacy",
        "layer_version_arn": "arn:aws:lambda:us-west-2:113088814899:layer:Klayers-python37-spacy:13",
        "deployed_region": "us-west-2"
    }

I am getting the following error-

[ERROR] OSError: [E050] Can't find model '/opt/en_core_web_sm-2.2.3'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 5, in lambda_handler
    nlp = spacy.load('/opt/en_core_web_sm-2.2.3')
  File "/opt/python/spacy/__init__.py", line 30, in load
    return util.load_model(name, **overrides)
  File "/opt/python/spacy/util.py", line 169, in load_model
    raise IOError(Errors.E050.format(name=name))

I did read your comment in another issue-

There’s two options, the best would be to download the model, zip it up into a layer and include that layer in your function. You can then load the model via spacy.load(‘/opt/model_name’)

I am having tough times getting this to work. I referred to this blog but by using the latest ARN, but, it doesn't work. What I mean is, when I create a layer out of the model, it still throws the above error.

You have suggested to package the model as a layer, please can you give more information on it? How do we do it? Is there a layer for the model already? I checked the json, its not available.

Basically I have downloaded the en_core_web_sm-2.2.5.tar.gz from here, how do I get this to a layer in lambda so that I can access the lambda from /opt/en_core_web_sm-2.2.5

Any advice or assistance will be highly appreciated. I can write a brief documentation of the solution which can eventually help lot more people.

[keithrozario]

@bozzmob sounds like you're almost there.

You need both the spacy layer and the model layer (you can have up to 5 layers at a time on a single function).

I suspect your problem is that you're not pulling in the model at the right directory level. Try the following in your code.

/opt/en_core_web_sm-2.2.5/en_core_web_sm or
/opt/en_core_web_sm-2.2.5/en_core_web_sm/en_core_web_sm-2.2.5

One of the above should work just fine. Let me know.

[bozzmob]

Thank you so much, I got it to work. Basically the model loaded. But, the code as such is not working for some reason. With no errors in cloudwatch console as well.
Does 2.2.3 spacy base package work with 2.2.5 model? maybe that's the issue?

[bozzmob]

Ok, it actually works now. The problem was - I had less RAM allocated. Now increased it to 512MB and it works.
Thank you so much for your time and efforts.

[keithrozario]

Awesome!

[keithrozario]

Closing this as it looks like the problem is solved.