Don't exclude *.egg-info from lambda package

[paulina-mudano]

Context

Excluding *.egg-info from the package deployed to lambda, when slim_handler: true causes random "DistributionNotFound" errors for packages that do not have dist-info but have egg-info.

Related issues:

• slim_handler does not allow to import AWS SageMaker #1555
• "DistributionNotFound" error after deployment #1601

I had exactly the problem described in #1555 and after investigation it turned out that pkg_resources ignores packages that do not have dist-info or egg-info directories. After removing the *.egg-info' from ZIP_EXCLUDES in Zappa core.py the code executes without issues!

Expected Behavior

All required files from venv should be included in the lambda package

Actual Behavior

.egg-info directories are missing which is causing DistributionNotFound errors.

Possible Fix

Remove '*.egg-info' from ZIP_EXCLUDES.

Steps to Reproduce

1. docker run -it amazonlinux:1 bash
2. export LANG=en_US.utf8
3. yum install -y python36-pip python36-devel vim
4. pip-3.6 install zappa awscli

ln -fs /usr/bin/python3 /usr/bin/python
ln -fs /usr/bin/pip-3.6 /usr/bin/pip

6. mkdir /tmp/myproject && cd /tmp/myproject
7. python -m venv .env
8. . .env/bin/activate
9. pip install sagemaker

export AWS_ACCESS_KEY_ID=xxx
export AWS_SECRET_ACCESS_KEY=xxx
export AWS_DEFAULT_REGION=eu-west-1

11. Create lambda_function.py with:

import sagemaker

def lambda_handler():
  print("Hello")

12. Create zappa_settings.json (see below)
13. zappa deploy dev
14. Execute Lambda in AWS and get this error:

The 'sagemaker' distribution was not found and is required by the application: DistributionNotFound
Traceback (most recent call last):
File "/var/task/handler.py", line 567, in lambda_handler
return LambdaHandler.lambda_handler(event, context)
File "/var/task/handler.py", line 237, in lambda_handler
handler = cls()
File "/var/task/handler.py", line 129, in __init__
self.app_module = importlib.import_module(self.settings.APP_MODULE)
File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "/tmp/test-zappa-egginfo/lambda_function.py", line 1, in <module>
import sagemaker
File "/tmp/test-zappa-egginfo/sagemaker/__init__.py", line 15, in <module>
from sagemaker import estimator # noqa: F401
File "/tmp/test-zappa-egginfo/sagemaker/estimator.py", line 23, in <module>
from sagemaker.analytics import TrainingJobAnalytics
File "/tmp/test-zappa-egginfo/sagemaker/analytics.py", line 22, in <module>
from sagemaker.session import Session
File "/tmp/test-zappa-egginfo/sagemaker/session.py", line 28, in <module>
from sagemaker.user_agent import prepend_user_agent
File "/tmp/test-zappa-egginfo/sagemaker/user_agent.py", line 22, in <module>
SDK_VERSION = pkg_resources.require('sagemaker')[0].version
File "/tmp/test-zappa-egginfo/pkg_resources/__init__.py", line 892, in require
needed = self.resolve(parse_requirements(requirements))
File "/tmp/test-zappa-egginfo/pkg_resources/__init__.py", line 778, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'sagemaker' distribution was not found and is required by the application

15. Modify /usr/local/lib/python3.6/site-packages/zappa/core.py: remove '*.egg-info' from ZIP_EXCLUDES in line 201
16. zappa update dev
17. Execute Lambda again, get successful execution.

Your Environment

• Zappa version used: 0.46.2
• Operating System and Python version: Amazon Linux, Python 3.6.5.
  Similar issue happens on MacOS 10.12.6 with Python 3.6.5 for other packages.
• The output of pip freeze:

boto3==1.8.6
botocore==1.11.6
docutils==0.14
jmespath==0.9.3
numpy==1.15.1
protobuf==3.6.1
protobuf3-to-dict==0.1.5
python-dateutil==2.7.3
PyYAML==3.13
s3transfer==0.1.13
sagemaker==1.9.2
scipy==1.1.0
six==1.11.0
urllib3==1.23

• Link to your project (optional): N/A
• Your zappa_settings.py:

{
    "dev": {
        "app_function": "lambda_function.lambda_handler",
        "aws_region": "eu-west-1",
        "project_name": "test-zappa-egginfo",
        "runtime": "python3.6",
        "s3_bucket": "some-bucket-here",
        "manage_roles": false,
        "role_arn": "role-arn-here",
        "keep_warm": false,
        "slim_handler": true,
        "log_level": "DEBUG",
        "debug": true,
        "touch": false,
        "num_retained_versions": 1
    }
}

[4lph4-Ph4un]

Good find! Is it possible to get a patch that would include this as an option?