-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory error for large arrays #10
Comments
Thanks for letting me know about this @tglauch . I will look at it when time permits it. I have to admit I am not sure why it happens. Are you able to post code reproducing the problem? E.g. using random data and a seed? |
Seems to be related to the convolution in |
Hi, I'll try to provide an example code asap. Cheers. |
Here we go with a snippet
Changing slightly the binning brings you from 2GB
to 52GB
Let me add: |
Hi again @tglauch . I'm afraid I won't be of much help regarding this. Running the code snippet you provided grinds my desktop computer to a halt, so it's a difficult problem to debug. Again, I believe the problem is related to Ideas you could try for debugging.
An idea for working around this
|
Hi,
First: thanks you for the great package, it works really well! However I experience something that I don't quite understand. I'm fitting a pretty large 4D KDE and it all works fine if a evaluate on a grid of the size
nbins_x, nbins_y, nbins_z, nbins_z2 = 200, 30, 80, 500
but once I increase the resolution a bit to let's say
nbins_x, nbins_y, nbins_z, nbins_z2 = 200, 40, 80, 500
I get a memory error. I have also profiled the memory consumption in the first and second case
1.)
2.)
As you see the memory consumption of the evaluate function suddenly jumps up from 9GB to 47GB without any obvious reason. I unfortunately currently don't have time to investigate this in more detail. But you might have an idea?! If it helps: I experience the same behavior on different computers.
Here the traceback:
Thanks a lot,
Theo
The text was updated successfully, but these errors were encountered: