Support Lambda - Reduce Size

[KlausSchaefers]

Describe the bug
I want to use the SDK in a lambda function. However, when building a zip file the zip exceeds 50 MB. Please reduce the bundle size or split the sdk into several parts.

To reproduce

1. Build a lambda that uses the SagerMaker SDK
2. Try to build a zip file including the SDK
3. Size is over 50 MB

Expected behavior

• Size should be smaller, or SDK should be included in Lambda runtime

Screenshots or logs

System information

• Python3.10

Additional context
Add any other context about the problem here.

[lucafrost]

+1 for this — the inability to call the SDK within Lambda is certainly frustrating.

I have seen approaches that attempt to "debloat" the SageMaker SDK, but have not tried them personally. See here for an example.

[j-adamczyk]

+1, this is really irritating. Especially since this should a simple SDK, and heavy libraries like Numpy and Scipy can be easily moved to a package extra (plugin), like pip install sagemaker[numpy]

[flyingcircle]

@j-adamczyk I am coincidentally looking at this today for my own project as well. Disappointed that I have to include numpy and pandas in my docker image despite only using api's and models. A smaller package could easily cut my image size by 30-50%

[guimorg]

+1 To point out this is also an issue for my team.

We have to rely on Lambda using Docker Images instead of Deployment Packages due to the size of the package but this is not optimal and makes us have to worry about other things.

It would be really nice to specify which internal part of SageMaker we would like to install, like sagemaker[pipeline] or sagemaker[all]. It would definitely improve usability and impact a smaller footprint for packages.

[lucafrost]

Despite the convenience of the SageMaker package, it’s simply an abstraction of the API exposed through boto3. Our team made the switch to pure boto3, would highly recommend.On 11 Oct 2023, at 19:03, Guilherme de Amorim ***@***.***> wrote: To point out this is also an issue for my team. We have to rely on Lambda using Docker Images instead of Deployment Packages due to the size of the package but this is not optimal and makes us have to worry about other things. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[jjerphan]

Hi everyone,

For your information, this issue is currently being addressed by #4222.
I am closing this issue since it duplicates #1200.

Feel free to continue discussions over there. 🙂