Need to use swap memory for loading (sdxl turbo) model, But I can't set it in sagemaker

[Suprhimp]

Hi, I found that I can't load my model to sagemaker because inside the docker, my model cause OOM while loading in side the sagemaker deploy docker.

I checked that with My EC2 instance inf2.xlarge works well in this environement. (logged with free -m)

this environment the works very well with sdxl turbo

this environment gives me OOM error.

Is there any setting that I can use swap memory? especially I want to know How can I allocate swap memory (sagemaker docker env can't use --privilegeflag).

[jeffhataws]

Hi @Suprhimp, will you help provide more information:

• Version of docker image
• Are you using SageMaker Notebooks or SageMaker Studio?

[Suprhimp]

https://github.com/aws/deep-learning-containers/blob/master/huggingface/pytorch/inference/docker/1.13/py3/sdk2.15.0/Dockerfile.neuronx

I used this docker image for sagemaker endpoint

I finally give up to use sdxl with neuronx in sagemaker environment 😂

But I build my backend with ec2.

[aws-taylor]

Hello @Suprhimp,

Since this appears to be an issue with SageMaker itself, I suggest you reach out to https://repost.aws/tags/questions/TAT80swPyVRPKPcA0rsJYPuA?view=all.