Problem with Cov3D

[ThinkJanice]

Describe the bug:
Estimate the noise of images using AnisotropicNoiseEstimator
0%| | 0/10 [00:05<?, ?it/s]
2024-10-10 19:27:31,441 CRITICAL [root] Implicit conversion to a NumPy array is not allowed. Please use .get() to construct a NumPy array explicitly.
Traceback:
File "cov3d_dimer_1k_volume.py", line 99, in
noise_estimator = AnisotropicNoiseEstimator(source, batchSize=batchsize) #check also
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/noise/noise.py", line 244, in init
self.filter = self._create_filter()
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/noise/noise.py", line 321, in _create_filter
noise_psd = self.estimate_noise_psd()
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/noise/noise.py", line 336, in estimate_noise_psd
images = self.src.images[i : i + self.batchSize].asnumpy()
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/source/image.py", line 102, in getitem
return self.fun(indices)
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/source/relion.py", line 275, in _images
return self.generation_pipeline.forward(Image(im), indices)
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/image/xform.py", line 68, in forward
return self._forward(im, indices=indices)
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/image/xform.py", line 438, in _forward
im = _apply_transform_cached(xform, im, indices, False)
File "/home/jingjing/anaconda3/envs/1115/lib/python3.8/site-packages/joblib/memory.py", line 312, in call
return self.func(*args, **kwargs)
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/image/xform.py", line 373, in _apply_xform
return xform.forward(im, indices=indices)
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/image/xform.py", line 68, in forward
return self._forward(im, indices=indices)
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/image/xform.py", line 207, in _forward
return im.downsample(self.resolution)
File "/home/jingjing/Newdisk3/fast-PCA/Cov3D-master/ASPIRE-Python_v2/code_folder/../src/aspire/image/image.py", line 362, in downsample
crop_fx = np.array([crop_pad_2d(fx[i], ds_res) for i in range(self.n_images)])
File "cupy/_core/core.pyx", line 1475, in cupy._core.core._ndarray_base.array

To Reproduce:
When I set:
common:
# numeric module to use - one of numpy/cupy
numeric: cupy
# fft backend to use - one of pyfftw/scipy/cupy/mkl
fft: cupy
the error appear.

If I set:
common:
# numeric module to use - one of numpy/cupy
numeric: numpy
# fft backend to use - one of pyfftw/scipy/cupy/mkl
fft: pyfftw
it works well.

Expected behavior:
I hope to know the reason of the bug

Environment (please complete the following information):

• Python: [e.g. Python 3.8]
• ASPIRE Version: [12.0, develop]
• CPU Type: [Ubuntu 22.04]
• GPU Extensions: [yes]

[garrettwrong]

Hi, unfortunately without the actual code you attempted to run it makes things rather difficult to nail down the exact issue you have.

Reading the report, it looks like you are running a very old version of ASPIRE which probably does not support CuPy in the component you are using. Even today, not all components support CuPy. It appears you may have had an error in a preprocessing component that I patched in the last few releases...

I just attempted the line you posted and it appears to work with the current release.

# install current release
pip install  "aspire[dev,gpu_12x]"

Configure CuPy mode

diff --git a/src/aspire/config_default.yaml b/src/aspire/config_default.yaml
index 9af8732d..6e32ead3 100644
--- a/src/aspire/config_default.yaml
+++ b/src/aspire/config_default.yaml
@@ -1,9 +1,9 @@
 version: 0.13.0
 common:
     # numeric module to use - one of numpy/cupy
-    numeric: numpy
+    numeric: cupy
     # fft backend to use - one of pyfftw/scipy/cupy/mkl
-    fft: scipy
+    fft: cupy
 
     # Set cache directory for ASPIRE example data.
     # By default the cache location will be set by pooch.os_cache(),
(END)

Simplest code to estimate the noise of images using AnisotropicNoiseEstimator:

import aspire

src = aspire.source.Simulation(L=32,n=100,C=1,seed=123)
nest = aspire.noise.AnisotropicNoiseEstimator(src)
x = nest.estimate()
print(x)

I have not intentionally accelerated the area you appear to be working in... so it is only marginally faster in practice (about 2x, see below).

Changing the repro problem to n=10000 images, L=128 pixels on my machine:
Estimating noise with CuPy: 11s
Estimating noise with default config: 20s

Changing the repro problem to n=10000 images, L=256 pixels on my machine:
Estimating noise with CuPy: 52s
Estimating noise with default config: 104s

Overall I'd say try installing the latest release code to see if that solves this issue. You should also find that the new default config and code is much faster everywhere...

Thanks

[garrettwrong]

Hi @ThinkJanice , did updating the software resolve the problem you encountered? Thanks!

[ThinkJanice]

Hi, yes, I was using version 0.12 before, which caused issues with the conversion between CuPy and NumPy. After downloading the new version 0.13, the CuPy issue was resolved, and I can now use CuPy directly. Thank you for your help.

However, I am not using a powerful machine like the A100, so I performed all cov3d calculations using float32. Do you know if there is a significant impact on the results between float32 and float64?

[garrettwrong]

Glad that worked out.

Hard to say regarding accuracy of results. You should find they are not radically different times (I'd expect no more than 2x) if you'd like to try both.

Thanks!