slim_handler does not allow to import AWS SageMaker

[aloschilov]

Context

The general scenario of using slim_handler does work for me, but when it comes to importing AWS SageMaker using import sagemaker the issue takes place.

Expected Behavior

AWS SageMaker imports normally.

Actual Behavior

AWS SageMaker import raising the following exception:

import sagemaker
File "/tmp/lds-topic-modeling/sagemaker/__init__.py", line 15, in <module>
from sagemaker import estimator # noqa: F401
File "/tmp/lds-topic-modeling/sagemaker/estimator.py", line 22, in <module>
from sagemaker.analytics import TrainingJobAnalytics
File "/tmp/lds-topic-modeling/sagemaker/analytics.py", line 22, in <module>
from sagemaker.session import Session
File "/tmp/lds-topic-modeling/sagemaker/session.py", line 28, in <module>
from sagemaker.user_agent import prepend_user_agent
File "/tmp/lds-topic-modeling/sagemaker/user_agent.py", line 22, in <module>
SDK_VERSION = pkg_resources.require('sagemaker')[0].version
File "/var/task/pkg_resources/__init__.py", line 891, in require
needed = self.resolve(parse_requirements(requirements))
File "/var/task/pkg_resources/__init__.py", line 777, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'sagemaker' distribution was not found and is required by the application

Possible Fix

I could not say about direct resolution of this issue. Whatever would allow to trick pkg_resources to provide proper results. Here is user_agent.py file:

from __future__ import absolute_import

import pkg_resources
import platform
import sys

import boto3
import botocore

SDK_VERSION = pkg_resources.require('sagemaker')[0].version
OS_NAME = platform.system() or 'UnresolvedOS'
OS_VERSION = platform.release() or 'UnresolvedOSVersion'
PYTHON_VERSION = '{}.{}.{}'.format(sys.version_info.major, sys.version_info.minor, sys.version_info.micro)


def prepend_user_agent(client):
    prefix = 'AWS-SageMaker-Python-SDK/{} Python/{} {}/{} Boto3/{} Botocore/{}'\
        .format(SDK_VERSION, PYTHON_VERSION, OS_NAME, OS_VERSION, boto3.__version__, botocore.__version__)
    if client._client_config.user_agent is None:
        client._client_config.user_agent = prefix
    else:
        client._client_config.user_agent = '{} {}'.format(prefix, client._client_config.user_agent)

Steps to Reproduce

It should be possible to reproduce after using single file with the only statement import sagemaker in file. For example, sample app.py for Flask application.

Environment

requirements.txt

appnope==0.1.0
argcomplete==1.9.3
base58==1.0.0
bleach==2.1.3
boto3==1.7.52
botocore==1.10.52
certifi==2018.4.16
cfn-flip==1.0.3
chardet==3.0.4
click==6.7
cycler==0.10.0
decorator==4.3.0
docutils==0.14
durationpy==0.5
entrypoints==0.2.3
Flask==1.0.2
future==0.16.0
hjson==3.0.1
html5lib==1.0.1
idna==2.7
ipykernel==4.8.2
ipython==5.0.0
ipython-genutils==0.2.0
ipywidgets==7.2.1
itsdangerous==0.24
Jinja2==2.10
jmespath==0.9.3
jsonschema==2.6.0
jupyter-client==5.2.3
jupyter-console==5.2.0
jupyter-core==4.4.0
kappa==0.6.0
kiwisolver==1.0.1
lambda-packages==0.20.0
MarkupSafe==1.0
mistune==0.8.3
nbconvert==5.3.1
nbformat==4.4.0
nltk==3.3
numpy==1.14.5
pandas==0.23.1
pandas-redshift==1.1.2
pandocfilters==1.4.2
pexpect==4.6.0
pickleshare==0.7.4
placebo==0.8.1
prompt-toolkit==1.0.15
protobuf==3.6.0
psycopg2==2.7.4
psycopg2-binary==2.7.4
ptyprocess==0.6.0
Pygments==2.2.0
pyparsing==2.2.0
python-dateutil==2.6.1
python-slugify==1.2.4
pytz==2018.4
PyYAML==3.12
pyzmq==17.0.0
qtconsole==4.3.1
requests==2.19.1
s3transfer==0.1.13
sagemaker==1.4.2
scikit-learn==0.19.1
scipy==1.1.0
Send2Trash==1.5.0
simplegeneric==0.8.1
six==1.11.0
sklearn==0.0
SQLAlchemy==1.2.8
sqlalchemy-redshift==0.7.1
terminado==0.8.1
testpath==0.3.1
toml==0.9.4
tornado==5.0.2
tqdm==4.19.1
traitlets==4.3.2
troposphere==2.3.1
Unidecode==1.0.22
urllib3==1.23
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.14.1
widgetsnbextension==3.2.1
wsgi-request-logger==0.4.6
zappa==0.46.1

[paulina-mudano]

We are encountering exactly the same error. Did you manage to get it to work?

[aloschilov]

@paulina-mudano

We are encountering exactly the same error. Did you manage to get it to work?

No, I have launched EC2 instance as temporary workaround. Time allows not to address this issue immediately my side.