Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
`import sagemaker.tensorflow` is broken after upgrading TF serving #139
So I recently upgraded my local tensorflow serving version to uh, 1.5:
After this upgrade, I realized I can't run
The exact issue I get is as follows:
It seems to me like somehow it's trying to load that
At the moment this is entirely preventing me from using SageMaker through the Python SDK which is certainly a problem...
Quick update: I tried this out a few times and can only replicate it with
sorry for a long delay. Indeed I observe the same behavior. The reason behind this is that Python SDK has copies of the tensorflow-serving protobuf files (under src/sagemaker/tensorflow/tensorflow_serving). Version 1.6.0 has changed these files potentially making them back-incompatible from previous versions.
When you try to import from from tensorflow_serving.apis outside of the SDK it goes to the installed packages and works.
This will be analyzed and addressed by the team. Thank you for reporting this!
This was referenced
May 1, 2018
referenced this issue
May 10, 2018
I investigated the issue today and got to the following conclusions:
1 - tensorflow-serving-api is only available in python 2 and cannot be installed using python 3. TensorFlow serving is not planning to create a python 3 version (tensorflow/serving#700) although the content of these versions would be the same. That is the main reason for us to maintain a copy of the protobuf messages inside SageMaker Python SDK.
2 - The issue occurs because Python 2 will prioritize loading the system installed module instead of the relative module version. That issue is solved adding future absolute import in any file that loads the reference. I enforced absolute imports in SageMaker to avoid it happening again: #180
3 - I have an additional PR updating the TF serving protobuf messages #181
I will merge these important changes as soon as possible to unblock you.
I appreciate you alerting us the issue and thanks for the patience.
The release https://github.com/aws/sagemaker-python-sdk/releases/tag/v1.2.5 includes the fixes for the issue.
Thanks again for reporting this issue.