Not able to deploy pretrained Pytorch Model

[mukeshyadav]

I have a pre-trained model, now trying to create an endpoint using Sagemaker, my folder structure like this
"model.tar.gz" looks like this:

model
|- config.json
|- pytorch_model.bin
|- special_tokens_map.json
|- spiece.model
|- tokenizer_config.json
|- training_args.bin
code
|- inference.py
| - requirements.txt

running following script to create endpoint:

pytorch_model = PyTorchModel( model_data='s3://mck-dl-ai-studio/answer_card/answercard.tar.gz', role=role, entry_point='inference.py', framework_version="1.3.1")

predictor = pytorch_model.deploy(instance_type='ml.t2.medium', initial_instance_count=1)

An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (500) from model with message "No module named 'transformers'". See https://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#logEventViewer:group=/aws/sagemaker/Endpoints/pytorch-inference-2020-07-20-16-45-51-564 in account xxxxxx for more information.

what I am missing here tried adding source_dir and py_version but no success

[chuyang-deng]

Hi @mukeshyadav , did you import transformers in your inference.py script? If so, it might be that the container failed to install modules listed in requirements.txt.

When you init the PyTorchModel, did you provide the source_dir with both inference.py and requirements.txt at the root level of source_dir? Here's documentation on how to include requirements.txt: https://sagemaker.readthedocs.io/en/stable/frameworks/pytorch/using_pytorch.html#using-third-party-libraries

[mukeshyadav]

Hi @ChuyangDeng Thanks for sharing the link, I resolved the issue but now during calling predict function getting below error:

An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (0) from the model with message "Your invocation timed out while waiting for a response from container model. Review the latency metrics for each container in Amazon CloudWatch,

[laurenyu]

Could you share your CloudWatch logs from the endpoint?