scaled_dot_product_attention OOMed: switched to slice attention

[eniora]

Hello, when I create an image with SDXL that is 4k (~3840x2160), I get in console scaled_dot_product_attention OOMed: switched to slice attention
Afterwards the image is generated fine but just wondering why I get this on a 3090 24GB but not on a 1070 8GB on another PC with the exact same settings and resolution (the image is generated fine with the 1070 as well, I am trying with only 1 sampling step on both PCs to speed up the test). Thanks in advance!

[ltdrdata]

I guess because of 8GB ram act on --lowvram mode implicitly.
How about try with --medvram mode

[eniora]

@ltdrdata Thanks, I just tried with --lowvram on the 3090 and I still get the same warning (--medvram says unrecognized argument)

Here's the workflow I am using if you or someone is willing to test, just generate without changing any settings, should be a fast test:

{ "last_node_id": 190, "last_link_id": 284, "nodes": [ { "id": 73, "type": "Reroute", "pos": [ 380, -900 ], "size": [ 75, 26 ], "flags": {}, "order": 8, "mode": 0, "inputs": [ { "name": "", "type": "*", "link": 129 } ], "outputs": [ { "name": "", "type": "CLIP", "links": [ 127, 128 ], "slot_index": 0 } ], "properties": { "showOutputText": false, "horizontal": false } }, { "id": 40, "type": "Reroute", "pos": [ 1000, -580 ], "size": [ 75, 26 ], "flags": {}, "order": 15, "mode": 0, "inputs": [ { "name": "", "type": "*", "link": 130, "slot_index": 0 } ], "outputs": [ { "name": "", "type": "VAE", "links": [ 222 ], "slot_index": 0 } ], "properties": { "showOutputText": false, "horizontal": false } }, { "id": 39, "type": "Reroute", "pos": [ 480, -580 ], "size": [ 75, 26 ], "flags": {}, "order": 9, "mode": 0, "inputs": [ { "name": "", "type": "*", "link": 65 } ], "outputs": [ { "name": "", "type": "VAE", "links": [ 130 ], "slot_index": 0 } ], "properties": { "showOutputText": false, "horizontal": false } }, { "id": 71, "type": "CLIPTextEncodeSDXLRefiner", "pos": [ 480, -720 ], "size": { "0": 210, "1": 130 }, "flags": {}, "order": 14, "mode": 0, "inputs": [ { "name": "clip", "type": "CLIP", "link": 128 }, { "name": "text", "type": "STRING", "link": 120, "widget": { "name": "text", "config": [ "STRING", { "multiline": true } ] } } ], "outputs": [ { "name": "CONDITIONING", "type": "CONDITIONING", "links": [ 209 ], "shape": 3, "slot_index": 0 } ], "title": "CLIPTextEncodeRefiner", "properties": { "Node name for S&R": "CLIPTextEncodeSDXLRefiner" }, "widgets_values": [ 2.5, 4096, 4096, "3D render, cartoon, (low quality:1.2), blurry, ugly, tiling, poorly drawn" ] }, { "id": 70, "type": "CLIPTextEncodeSDXLRefiner", "pos": [ 480, -900 ], "size": { "0": 210, "1": 130 }, "flags": {}, "order": 13, "mode": 0, "inputs": [ { "name": "clip", "type": "CLIP", "link": 127 }, { "name": "text", "type": "STRING", "link": 116, "widget": { "name": "text", "config": [ "STRING", { "multiline": true } ] } } ], "outputs": [ { "name": "CONDITIONING", "type": "CONDITIONING", "links": [ 208 ], "shape": 3, "slot_index": 0 } ], "title": "CLIPTextEncodeRefiner", "properties": { "Node name for S&R": "CLIPTextEncodeSDXLRefiner" }, "widgets_values": [ 6, 4096, 4096, "galaxy inside gemstone, (sharp focus:1.2), extremely detailed, (photorealistic:1.4), (RAW image, 8k high resolution:1.2), RAW candid cinema, 16mm, color graded Portra 400 film, ultra realistic, cinematic film still, subsurface scattering, ray tracing, (volumetric lighting)" ] }, { "id": 65, "type": "CLIPTextEncodeSDXL", "pos": [ 480, -170 ], "size": { "0": 210, "1": 220 }, "flags": { "collapsed": false }, "order": 17, "mode": 0, "inputs": [ { "name": "clip", "type": "CLIP", "link": 126 }, { "name": "text_g", "type": "STRING", "link": 139, "widget": { "name": "text_g", "config": [ "STRING", { "multiline": true, "default": "CLIP_G" } ] } }, { "name": "text_l", "type": "STRING", "link": 146, "widget": { "name": "text_l", "config": [ "STRING", { "multiline": true, "default": "CLIP_L" } ] } } ], "outputs": [ { "name": "CONDITIONING", "type": "CONDITIONING", "links": [ 200 ], "shape": 3, "slot_index": 0 } ], "title": "CLIPTextEncode", "properties": { "Node name for S&R": "CLIPTextEncodeSDXL" }, "widgets_values": [ 4096, 4096, 0, 0, 4096, 4096, "3D render, cartoon, (low quality:1.2), blurry, ugly, tiling, poorly drawn", "3D render, cartoon, (low quality:1.2), blurry, ugly, tiling, poorly drawn" ] }, { "id": 146, "type": "VAEDecode", "pos": [ 1150, -890 ], "size": { "0": 140, "1": 50 }, "flags": { "collapsed": true }, "order": 22, "mode": 0, "inputs": [ { "name": "samples", "type": "LATENT", "link": 217 }, { "name": "vae", "type": "VAE", "link": 222 } ], "outputs": [ { "name": "IMAGE", "type": "IMAGE", "links": [ 264 ], "shape": 3, "slot_index": 0 } ], "properties": { "Node name for S&R": "VAEDecode" } }, { "id": 16, "type": "PrimitiveNode", "pos": [ -780, -230 ], "size": { "0": 430, "1": 150 }, "flags": {}, "order": 0, "mode": 0, "outputs": [ { "name": "STRING", "type": "STRING", "links": [ 120, 139, 146 ], "slot_index": 0, "widget": { "name": "text", "config": [ "STRING", { "multiline": true } ] } } ], "title": "Negative Prompt", "properties": {}, "widgets_values": [ "3D render, cartoon, (low quality:1.2), blurry, ugly, tiling, poorly drawn" ], "color": "#322", "bgcolor": "#533" }, { "id": 50, "type": "CLIPTextEncodeSDXL", "pos": [ 480, -430 ], "size": { "0": 210, "1": 220 }, "flags": { "collapsed": false }, "order": 16, "mode": 0, "inputs": [ { "name": "clip", "type": "CLIP", "link": 93 }, { "name": "text_g", "type": "STRING", "link": 279, "widget": { "name": "text_g", "config": [ "STRING", { "multiline": true, "default": "CLIP_G" } ] }, "slot_index": 1 }, { "name": "text_l", "type": "STRING", "link": 184, "widget": { "name": "text_l", "config": [ "STRING", { "multiline": true, "default": "CLIP_L" } ] } } ], "outputs": [ { "name": "CONDITIONING", "type": "CONDITIONING", "links": [ 199 ], "shape": 3, "slot_index": 0 } ], "title": "CLIPTextEncode", "properties": { "Node name for S&R": "CLIPTextEncodeSDXL" }, "widgets_values": [ 4096, 4096, 0, 0, 4096, 4096, "galaxy inside gemstone, (sharp focus:1.2), extremely detailed, (photorealistic:1.4), (RAW image, 8k high resolution:1.2), RAW candid cinema, 16mm, color graded Portra 400 film, ultra realistic, cinematic film still, subsurface scattering, ray tracing, (volumetric lighting)", "galaxy inside gemstone, (sharp focus:1.2), extremely detailed, (photorealistic:1.4), (RAW image, 8k high resolution:1.2), RAW candid cinema, 16mm, color graded Portra 400 film, ultra realistic, cinematic film still, subsurface scattering, ray tracing, (volumetric lighting)" ] }, { "id": 61, "type": "Reroute", "pos": [ 380, -430 ], "size": [ 75, 26 ], "flags": {}, "order": 11, "mode": 0, "inputs": [ { "name": "", "type": "*", "link": 281, "slot_index": 0 } ], "outputs": [ { "name": "", "type": "CLIP", "links": [ 93, 126 ], "slot_index": 0 } ], "properties": { "showOutputText": false, "horizontal": false } }, { "id": 142, "type": "KSamplerAdvanced", "pos": [ 790, -920 ], "size": { "0": 290, "1": 330 }, "flags": {}, "order": 21, "mode": 0, "inputs": [ { "name": "model", "type": "MODEL", "link": 275 }, { "name": "positive", "type": "CONDITIONING", "link": 208 }, { "name": "negative", "type": "CONDITIONING", "link": 209 }, { "name": "latent_image", "type": "LATENT", "link": 224 }, { "name": "start_at_step", "type": "INT", "link": 229, "widget": { "name": "start_at_step", "config": [ "INT", { "default": 0, "min": 0, "max": 10000 } ] } }, { "name": "steps", "type": "INT", "link": 266, "widget": { "name": "steps", "config": [ "INT", { "default": 20, "min": 1, "max": 10000 } ] } } ], "outputs": [ { "name": "LATENT", "type": "LATENT", "links": [ 217 ], "shape": 3, "slot_index": 0 } ], "properties": { "Node name for S&R": "KSamplerAdvanced" }, "widgets_values": [ "disable", 722721601948537, "randomize", 40, 6, "dpmpp_2m_sde_gpu", "karras", 32, 1000, "disable" ] }, { "id": 186, "type": "Reroute", "pos": [ 380, -920 ], "size": [ 75, 26 ], "flags": {}, "order": 7, "mode": 0, "inputs": [ { "name": "", "type": "*", "link": 274 } ], "outputs": [ { "name": "", "type": "MODEL", "links": [ 275 ], "slot_index": 0 } ], "properties": { "showOutputText": false, "horizontal": false } }, { "id": 11, "type": "CheckpointLoaderSimple", "pos": [ 70, -920 ], "size": { "0": 290, "1": 100 }, "flags": {}, "order": 1, "mode": 0, "outputs": [ { "name": "MODEL", "type": "MODEL", "links": [ 274 ], "shape": 3, "slot_index": 0 }, { "name": "CLIP", "type": "CLIP", "links": [ 129 ], "shape": 3, "slot_index": 1 }, { "name": "VAE", "type": "VAE", "links": [ 65 ], "shape": 3, "slot_index": 2 } ], "properties": { "Node name for S&R": "CheckpointLoaderSimple" }, "widgets_values": [ "SDXLRefiner.safetensors" ] }, { "id": 31, "type": "Reroute", "pos": [ 380, -450 ], "size": [ 75, 26 ], "flags": {}, "order": 10, "mode": 0, "inputs": [ { "name": "", "type": "*", "link": 272, "slot_index": 0 } ], "outputs": [ { "name": "", "type": "MODEL", "links": [ 227 ], "slot_index": 0 } ], "properties": { "showOutputText": false, "horizontal": false } }, { "id": 4, "type": "CheckpointLoaderSimple", "pos": [ 70, -450 ], "size": { "0": 290, "1": 100 }, "flags": {}, "order": 2, "mode": 0, "outputs": [ { "name": "MODEL", "type": "MODEL", "links": [ 272 ], "slot_index": 0 }, { "name": "CLIP", "type": "CLIP", "links": [ 281 ], "slot_index": 1 }, { "name": "VAE", "type": "VAE", "links": [], "slot_index": 2 } ], "properties": { "Node name for S&R": "CheckpointLoaderSimple" }, "widgets_values": [ "SDXL.safetensors" ] }, { "id": 41, "type": "Reroute", "pos": [ 653, -561 ], "size": [ 75, 26 ], "flags": {}, "order": 20, "mode": 0, "inputs": [ { "name": "", "type": "*", "link": 284 } ], "outputs": [ { "name": "", "type": "LATENT", "links": [ 224 ], "slot_index": 0 } ], "properties": { "showOutputText": false, "horizontal": false } }, { "id": 139, "type": "KSamplerAdvanced", "pos": [ 790, -450 ], "size": { "0": 290, "1": 330 }, "flags": {}, "order": 18, "mode": 0, "inputs": [ { "name": "model", "type": "MODEL", "link": 227 }, { "name": "positive", "type": "CONDITIONING", "link": 199 }, { "name": "negative", "type": "CONDITIONING", "link": 200 }, { "name": "latent_image", "type": "LATENT", "link": 232 }, { "name": "steps", "type": "INT", "link": 225, "widget": { "name": "steps", "config": [ "INT", { "default": 20, "min": 1, "max": 10000 } ] } }, { "name": "end_at_step", "type": "INT", "link": 228, "widget": { "name": "end_at_step", "config": [ "INT", { "default": 10000, "min": 0, "max": 10000 } ] } } ], "outputs": [ { "name": "LATENT", "type": "LATENT", "links": [ 283 ], "shape": 3, "slot_index": 0 } ], "properties": { "Node name for S&R": "KSamplerAdvanced" }, "widgets_values": [ "enable", 265931407709440, "randomize", 40, 6, "dpmpp_2m_sde_gpu", "karras", 0, 32, "enable" ] }, { "id": 190, "type": "Reroute", "pos": [ 1100, -560 ], "size": [ 75, 26 ], "flags": {}, "order": 19, "mode": 0, "inputs": [ { "name": "", "type": "*", "link": 283 } ], "outputs": [ { "name": "", "type": "LATENT", "links": [ 284 ], "slot_index": 0 } ], "properties": { "showOutputText": false, "horizontal": false } }, { "id": 15, "type": "PrimitiveNode", "pos": [ -780, -420 ], "size": { "0": 430, "1": 150 }, "flags": {}, "order": 3, "mode": 0, "outputs": [ { "name": "STRING", "type": "STRING", "links": [ 116, 184, 279 ], "widget": { "name": "text", "config": [ "STRING", { "multiline": true } ] }, "slot_index": 0 } ], "title": "Positive Prompt", "properties": {}, "widgets_values": [ "galaxy inside gemstone, (sharp focus:1.2), extremely detailed, (photorealistic:1.4), (RAW image, 8k high resolution:1.2), RAW candid cinema, 16mm, color graded Portra 400 film, ultra realistic, cinematic film still, subsurface scattering, ray tracing, (volumetric lighting)" ], "color": "#232", "bgcolor": "#353" }, { "id": 158, "type": "Reroute", "pos": [ 610, 80 ], "size": [ 75, 26 ], "flags": {}, "order": 12, "mode": 0, "inputs": [ { "name": "", "type": "*", "link": 280 } ], "outputs": [ { "name": "", "type": "LATENT", "links": [ 232 ], "slot_index": 0 } ], "properties": { "showOutputText": false, "horizontal": false } }, { "id": 152, "type": "PrimitiveNode", "pos": [ -560, -40 ], "size": { "0": 210, "1": 82 }, "flags": {}, "order": 4, "mode": 0, "outputs": [ { "name": "INT", "type": "INT", "links": [ 225, 266 ], "slot_index": 0, "widget": { "name": "steps", "config": [ "INT", { "default": 20, "min": 1, "max": 10000 } ] } } ], "title": "Total Steps", "properties": {}, "widgets_values": [ 40, "fixed" ] }, { "id": 155, "type": "PrimitiveNode", "pos": [ -780, -40 ], "size": { "0": 210, "1": 82 }, "flags": {}, "order": 5, "mode": 0, "outputs": [ { "name": "INT", "type": "INT", "links": [ 228, 229 ], "slot_index": 0, "widget": { "name": "end_at_step", "config": [ "INT", { "default": 10000, "min": 0, "max": 10000 } ] } } ], "title": "Steps On Base Model", "properties": {}, "widgets_values": [ 32, "fixed" ] }, { "id": 183, "type": "PreviewImage", "pos": [ 1320, -920 ], "size": { "0": 1920, "1": 1110 }, "flags": {}, "order": 23, "mode": 0, "inputs": [ { "name": "images", "type": "IMAGE", "link": 264 } ], "properties": { "Node name for S&R": "PreviewImage" } }, { "id": 5, "type": "EmptyLatentImage", "pos": [ -560, 80 ], "size": { "0": 210, "1": 110 }, "flags": {}, "order": 6, "mode": 0, "inputs": [], "outputs": [ { "name": "LATENT", "type": "LATENT", "links": [ 280 ], "slot_index": 0 } ], "title": "Image and batch size", "properties": { "Node name for S&R": "EmptyLatentImage" }, "widgets_values": [ 3840, 2160, 1 ] } ], "links": [ [ 65, 11, 2, 39, 0, "*" ], [ 93, 61, 0, 50, 0, "CLIP" ], [ 116, 15, 0, 70, 1, "STRING" ], [ 120, 16, 0, 71, 1, "STRING" ], [ 126, 61, 0, 65, 0, "CLIP" ], [ 127, 73, 0, 70, 0, "CLIP" ], [ 128, 73, 0, 71, 0, "CLIP" ], [ 129, 11, 1, 73, 0, "*" ], [ 130, 39, 0, 40, 0, "*" ], [ 139, 16, 0, 65, 1, "STRING" ], [ 146, 16, 0, 65, 2, "STRING" ], [ 184, 15, 0, 50, 2, "STRING" ], [ 199, 50, 0, 139, 1, "CONDITIONING" ], [ 200, 65, 0, 139, 2, "CONDITIONING" ], [ 208, 70, 0, 142, 1, "CONDITIONING" ], [ 209, 71, 0, 142, 2, "CONDITIONING" ], [ 217, 142, 0, 146, 0, "LATENT" ], [ 222, 40, 0, 146, 1, "VAE" ], [ 224, 41, 0, 142, 3, "LATENT" ], [ 225, 152, 0, 139, 4, "INT" ], [ 227, 31, 0, 139, 0, "MODEL" ], [ 228, 155, 0, 139, 5, "INT" ], [ 229, 155, 0, 142, 4, "INT" ], [ 232, 158, 0, 139, 3, "LATENT" ], [ 264, 146, 0, 183, 0, "IMAGE" ], [ 266, 152, 0, 142, 5, "INT" ], [ 272, 4, 0, 31, 0, "*" ], [ 274, 11, 0, 186, 0, "*" ], [ 275, 186, 0, 142, 0, "MODEL" ], [ 279, 15, 0, 50, 1, "STRING" ], [ 280, 5, 0, 158, 0, "*" ], [ 281, 4, 1, 61, 0, "*" ], [ 283, 139, 0, 190, 0, "*" ], [ 284, 190, 0, 41, 0, "*" ] ], "groups": [ { "title": "Input", "bounding": [ -800, -500, 470, 710 ], "color": "#8AA" } ], "config": {}, "extra": {}, "version": 0.4 }

[comfyanonymous]

That's normal it's because scaled_dot_product_attention aka pytorch attention doesn't support what the VAE is doing very well yet. This message is just for debug purposes and there will be no difference in the output image.

This is why the regular standalone package still ships with xformers.

[eniora]

Thanks a lot for the info :)
I just installed xformers with the command in the readme.md file and it's now at least 30% faster than pytorch attention at generating a full 4k image with 40 steps. (225 seconds with pytorch vs 170 seconds with xformers)