ERROR multiprocessing.pool.RemoteTraceback

[Chrisvanhoorn]

Hi all,

I am running into an issue during the refinement stage. I have all the dependencies installed with the right version according to the tensorflow table:

Python 3.7.11
Tensor flow 2.6.2
cuDNN 8.1.0.77-h90431f1_0
CUDA 11.2

Would you know what is causing the following error?
many thanks!

isonet.py refine ./subtomo.star  --gpuID 0,1,2,3

######Isonet done extracting subtomograms######

11-10 20:12:55, INFO     
######Isonet starts refining######

11-10 20:13:11, INFO     Done preperation for the first iteration!
11-10 20:13:11, INFO     Start Iteration1!
11-10 20:13:27, INFO     Noise Level:0.0
11-10 20:13:28, ERROR    multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/lmb/home/cvhoorn/miniconda3/envs/isonet/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/lmb/home/cvhoorn/miniconda3/envs/isonet/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/lmb/home/cvhoorn/IsoNet/preprocessing/prepare.py", line 147, in get_cubes
    with mrcfile.open(current_mrc) as mrcData:
  File "/lmb/home/cvhoorn/miniconda3/envs/isonet/lib/python3.7/site-packages/mrcfile/load_functions.py", line 139, in open
    header_only=header_only)
  File "/lmb/home/cvhoorn/miniconda3/envs/isonet/lib/python3.7/site-packages/mrcfile/mrcfile.py", line 108, in __init__
    self._open_file(name)
  File "/lmb/home/cvhoorn/miniconda3/envs/isonet/lib/python3.7/site-packages/mrcfile/mrcfile.py", line 125, in _open_file
    self._iostream = open(name, self._mode + 'b')
FileNotFoundError: [Errno 2] No such file or directory: 'results/TS_010_iter00.mrc'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/lmb/home/cvhoorn/IsoNet/bin/refine.py", line 25, in run
    run_whole(args)
  File "/lmb/home/cvhoorn/IsoNet/bin/refine.py", line 143, in run_whole
    get_cubes_list(args)
  File "/lmb/home/cvhoorn/IsoNet/preprocessing/prepare.py", line 189, in get_cubes_list
    res = p.map(func,inp)
  File "/lmb/home/cvhoorn/miniconda3/envs/isonet/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/lmb/home/cvhoorn/miniconda3/envs/isonet/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
FileNotFoundError: [Errno 2] No such file or directory: 'results/TS_010_iter00.mrc'

[procyontao]

Hi,

We ran into this problem before, it was because the tomogram file name contains a dot sign.
Please tell me whether this is also your case.

[Chrisvanhoorn]

That fixed it! thank you

[alisterburt]

quick script which I used to fix this for my subtomo filenames which included two leading '.' characters, might save someone some time

import starfile
import os

star = starfile.read('subtomo.star')
for src, dest in zip (star['rlnImageName'], star['rlnImageName'].str.replace('.','_', 2)):
       	os.symlink(src, dest)
star['rlnImageName'] = star['rlnImageName'].str.replace('.','_', 2)
starfile.write(star, 'subtomo_nodot.star')

[alisterburt]

I'm still running into this at the refinement step despite the subtomo names having no dots.

To be clear:

• my tomogram names have dots
• deconvolution and extraction work as expected
• I modify my subtomo.star file and symlink the files with the script above
  -I run isonet.py refine subtomo_nodot.star --gpuID 0,1,2,3 --iterations 30 --noise_start_iter 10,15,20,25 --noise_level 0.05,0.1,0.15,0.2
• I get the following traceback

Traceback (most recent call last):
  File "/lmb/home/aburt/isonet/IsoNet/bin/refine.py", line 25, in run
    run_whole(args)
  File "/lmb/home/aburt/isonet/IsoNet/bin/refine.py", line 109, in run_whole
    args = prepare_first_iter(args)
  File "/lmb/home/aburt/isonet/IsoNet/preprocessing/prepare.py", line 115, in prepare_first_iter
    res = p.map(func, settings.mrc_list)
  File "/lmb/home/aburt/mambaforge/envs/isonet/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/lmb/home/aburt/mambaforge/envs/isonet/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
FileNotFoundError: [Errno 2] No such file or directory: 'subtomo/TS_009_mrc_8_80Apx_000000.mrc'

• I confirm that the file does indeed exist

(isonet) hal.lmb.internal ➜  isonet  ls subtomo/TS_009_mrc_8_80Apx_000000.mrc
subtomo/TS_009_mrc_8_80Apx_000000.mrc

[procyontao]

Hi alisterburt,

I sorry that I just noticed you message have not been replied. People recently are still complaining about similar file not found error. I wonder whether this problem still exists in your case?

And thank you for your suggestion for fixing the issue related to '.' characters.

Best,
Yuntao