Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting cudaErrorMemoryAllocation out of memory #103

Open
quicktwit opened this issue Jun 30, 2022 · 6 comments
Open

Getting cudaErrorMemoryAllocation out of memory #103

quicktwit opened this issue Jun 30, 2022 · 6 comments

Comments

@quicktwit
Copy link

quicktwit commented Jun 30, 2022

Hi
This library is awesome! Thank you, author.
I am working on big-size point clouds (in ply format). I am trying this code (https://github.com/neka-nat/cupoch/blob/master/examples/python/advanced/pointcloud_outlier_removal.py) to filter point cloud noises. On a few MBs or around 1Gbs size point cloud file works like a charm.
But when I try on a bigger point cloud file(5GB - approx 97million points) then I after remove_statistical_outlier I am getting cudaErrorMemoryAllocation out of memory.

cl, ind = voxel_down_pcd.remove_statistical_outlier(nb_neighbors=20, std_ratio=2.0)
Memory Error: std::bad_alloc: CUDA error at /home/path/cupoch/cupoch/third_party/rmm/include/rmm/mr/device/cuda_memory_resource.hpp:69: cudaErrorMemoryAllocation out of memory

I really wanted to use this library as it is much faster than other alternatives.
Can you help me with how to solve this issue?
I really appreciate this. Thank you again in advance!!
P.S.
System:
Ubuntu 20.04
RTX3090 24GB
I installed cupoch using pip

@neka-nat
Copy link
Owner

neka-nat commented Jul 1, 2022

Hi,
Thanks for the report!
If possible, could you please share the pointcloud file with the error?

@quicktwit
Copy link
Author

Hi @neka-nat
Thank you for your reply. Some data I cannot put publicly. So I’m finding some workaround so that l can share here easily. I will update soon. Thank you so much again.

@quicktwit
Copy link
Author

Hi @neka-nat
I tested one point cloud data. You can use the file called L003.ply from here: https://1drv.ms/u/s!Amlc6yZnF87psX6hKS8VOQllVvj4?e=yWhrYX
This file L003.ply has more than 39million points.
Screen Shot 2022-07-03 at 19 29 11
Same error I am getting for this L003.ply also:

cl, ind = voxel_down_pcd.remove_statistical_outlier(nb_neighbors=20, std_ratio=2.0)
MemoryError: std::bad_alloc: CUDA error at: /home/runner/work/cupoch/cupoch/third_party/rmm/include/rmm/mr/device/cuda_memory_resource.hpp:69: cudaErrorMemoryAllocation out of memory

@quicktwit
Copy link
Author

Hi @neka-nat
Do you have any advice on how do I fix this? I tried too much downsampling but it's affecting the result. Let me know if you have any suggestions. Thank you in advance.

@neka-nat
Copy link
Owner

I am still investigating the problem, but I found that it was failing to allocate memory during flann tree construction.
It may take some time to solve the problem.

@quicktwit
Copy link
Author

Hi @neka-nat
Thank you for the update. I will wait for your update. Thank you once again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants