VRAM requirements?

[vgabbo]

Sorry, I was expecting it to work on a 20GB gpu but it goes OOM.
Could you please point out what are the expected minimum vram requirements?

Thank you very much.

[gleize]

Hi @vgabbo,

The smallest GPU we've been able to run the model reliably is a V100, which is about 32 Gb of VRAM.

[tanghaotommy]

The memory usage is closely tied to the size of the voxel output from stage 1. Generally, if the voxel size is in the hundreds or low thousands, your setups should handle it without issues. However, when the voxel size exceeds 10k, you may experience OOM, especially on GPUs with smaller VRAM.

To mitigate this, you can try setting the downsample_ss_dist parameter in pipeline.yaml to a more aggressive value, such as 2 or 4. This will downsample the stage 1 voxel size and should help reduce memory usage. Please note, though, that this may impact performance, and the results are not guaranteed.

[gleize]

GPU minimum requirement added to prerequisites section. Closing this now.

[gjtjx]

Step-by-step execution is possible. After model inference is completed, GPU memory is released. The RTX 4090 24GB can support the complete inference process.​

[wang-xilin]

Step-by-step execution is possible. After model inference is completed, GPU memory is released. The RTX 4090 24GB can support the complete inference process.​

Any modification to the code? It seems that demo.py not work for my local 4090 24GB. Thanks.

[shaonvpiaoyaodesilian]

可以分步执行。模型推理完成后，GPU 内存将被释放。RTX 4090 24GB 可以支持完整的推理过程。

Yes, I have the same question. Can you describe the implementation steps

[shaonvpiaoyaodesilian]

可以分步执行。模型推理完成后，GPU 内存将被释放。RTX 4090 24GB 可以支持完整的推理过程。

I tried to modify the YAML file in the checkpoints to reduce the parameters, but the author clearly failed because they fixed the pre trained PT. I also tried to analyze the error and it seems that there is not enough GPU memory left when decoding the slavdedoder_mesh
File "/home/ubuntu/project/NeRF/sam-3d-objects/notebook/inference.py", line 109, in call
    return self._pipeline.run(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/project/NeRF/sam-3d-objects/sam3d_objects/pipeline/inference_pipeline_pointmap.py", line 388, in run
    outputs = self.decode_slat(
              ^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/project/NeRF/sam-3d-objects/sam3d_objects/pipeline/inference_pipeline.py", line 608, in decode_slat
    ret["mesh"] = self.models"slat_decoder_mesh"
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/miniconda3/envs/sam3d-objects2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/miniconda3/envs/sam3d-objects2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/project/NeRF/sam-3d-objects/sam3d_objects/model/backbone/tdfy_dit/models/structured_latent_vae/decoder_mesh.py", line 184, in forward
    h = super().forward(x)
        ^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/project/NeRF/sam-3d-objects/sam3d_objects/model/backbone/tdfy_dit/models/structured_latent_vae/base.py", line 133, in forward
    h = block(h)
        ^^^^^^^^
  File "/data/miniconda3/envs/sam3d-objects2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/miniconda3/envs/sam3d-objects2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/project/NeRF/sam-3d-objects/sam3d_objects/model/backbone/tdfy_dit/modules/sparse/transformer/blocks.py", line 84, in forward
    return self._forward(x)
           ^^^^^^^^^^^^^^^^
  File "/home/ubuntu/project/NeRF/sam-3d-objects/sam3d_objects/model/backbone/tdfy_dit/modules/sparse/transformer/blocks.py", line 71, in _forward
    h = self.attn(h)
        ^^^^^^^^^^^^
  File "/data/miniconda3/envs/sam3d-objects2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/miniconda3/envs/sam3d-objects2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/project/NeRF/sam-3d-objects/sam3d_objects/model/backbone/tdfy_dit/modules/sparse/attention/modules.py", line 151, in forward
    h = sparse_windowed_scaled_dot_product_self_attention(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/project/NeRF/sam-3d-objects/sam3d_objects/model/backbone/tdfy_dit/modules/sparse/attention/windowed_attn.py", line 182, in sparse_windowed_scaled_dot_product_self_attention
    out = masked_sdpa(q, k, v, seq_lens, seq_lens)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/project/NeRF/sam-3d-objects/sam3d_objects/model/backbone/tdfy_dit/modules/sparse/attention/masked_sdpa.py", line 49, in masked_sdpa
    out = F.scaled_dot_product_attention(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/miniconda3/envs/sam3d-objects2/lib/python3.11/site-packages/torch/utils/_device.py", line 106, in torch_function
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.18 GiB. GPU 0 has a total capacity of 23.64 GiB of which 1.26 GiB is free. Including non-PyTorch memory, this process has 22.38 GiB memory in use. Of the allocated memory 16.75 GiB is allocated by PyTorch, and 5.13 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting

[DarK404]

Step-by-step execution is possible. After model inference is completed, GPU memory is released. The RTX 4090 24GB can support the complete inference process.​

Any modification to the code? It seems that demo.py not work for my local 4090 24GB. Thanks.

You can modify the code to skip mesh decoding and save GPU memory. Here's the relevant part from demo.py:

output = inference._pipeline.run(
    inference.merge_mask_to_rgba(image, mask),
    None,
    seed=10,
    decode_formats=["gaussian"],  # skip mesh decoding to save memory
)

[wang-xilin]

Step-by-step execution is possible. After model inference is completed, GPU memory is released. The RTX 4090 24GB can support the complete inference process.​

Any modification to the code? It seems that demo.py not work for my local 4090 24GB. Thanks.

You can modify the code to skip mesh decoding and save GPU memory. Here's the relevant part from demo.py:

output = inference._pipeline.run(
inference.merge_mask_to_rgba(image, mask),
None,
seed=10,
decode_formats=["gaussian"], # skip mesh decoding to save memory
)
It works! Thanks.

[shaonvpiaoyaodesilian]

Step-by-step execution is possible. After model inference is completed, GPU memory is released. The RTX 4090 24GB can support the complete inference process.​

Any modification to the code? It seems that demo.py not work for my local 4090 24GB. Thanks.

You can modify the code to skip mesh decoding and save GPU memory. Here's the relevant part from demo.py:

output = inference._pipeline.run(
inference.merge_mask_to_rgba(image, mask),
None,
seed=10,
decode_formats=["gaussian"], # skip mesh decoding to save memory
)

I seem to have set it up this way without success. I'm running demo_3db_mesh_alignment. Are there any other solutions? Thank you. I have already set the parameters in ppipeline.yaml to downsample_ss_dist: 8 and compile_model: true.

Error: At "Decoding sparse latent..."
Omitted content...
ret["gaussian"] = self.models"slat_decoder_gs"
Omitting...
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.15 GiB. GPU 0 has a total capacity of 23.64 GiB of which 3.10 GiB is free.