Elastic Inference for Pytorch 1.4.0

[raguiar2]

Is your feature request related to a problem? Please describe.
I am using the faster r-cnn network with pytorch for object detection in images. However, there is a bug in pytorch 1.3.1 (patched in 1.4.0 and torchvision 0.5.0) that blocks torch.script from encoding/decoding the model, and therefore prevents me from using Elastic Inference with sagemaker. I really need EI to work to produce results in realtime, and I really want this solution so I can have fully managed hosting with sagemaker.

Describe the solution you'd like
Add Elastic Inference support for pytorch 1.4.0 with sagemaker along with pytorch 1.3.1

Describe alternatives you've considered
None

Additional context
None

[knakad]

Thanks for reaching out! I've forwarded your request to the appropriate team :)

[raguiar2]

Thank you! It would be amazing if this was enabled :)

[warex03]

+1

[mohanasudhan]

@raguiar2 Thanks for reporting this issue. Please refer to Elastic Inference document for recommendation https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html

Q: We currently use Amazon Elastic Inference (EI) accelerators. Will we be able to continue using them after April 15, 2023?

Yes, you will be able use Amazon EI accelerators. We recommend that you migrate your current ML Inference workloads running on Amazon EI to other hardware accelerator options at your earliest convenience.

Recommend going over the migration steps https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html#ei-migration

Please reopen the issue if you see any issue with migration.