Process killed due to very large array

[IslamAAli]

Hello,
I am trying to get the distance correlation between two very large vectors (25k each), and the dcor function gets killed due to out of memory error. How can we fix that?

dcor.distance_correlation(np.array(x, dtype=np.float32), np.array(y, dtype=np.float32), exponent=0.5)

[vnmabus]

The order of the naive algorithm for computing distance covariance is $O(N^2)$, which is problematic for large inputs. However, by default a fast $O(NlogN)$ algorithm is used whenever possible. The conditions for using it are that the input random vectors are unidimensional, and that the exponent used is one.

In your case you have exponent=0.5, so the naive algorithm is used. Do you have a particular reason for that choice? Otherwise I recommend you to stick to the default value of exponent=1.

[IslamAAli]

Thanks for your reply. I don't have a specific reason for having exp=0.5 at the moment. I changed that to 1, and the process is running smoothly.