StopIteration error

[Tordelap]

What happened?

Hello,

I was trying to preprocess a first subject within a simple data set that just includes T1 and resting state images. I ran the first subject and it reported an error while preprocessing the functional images, json files seemed to have typos and I have corrected them, but now every time I intent to run any subject I am getting the error shown bellow. Am I doing something wrong? Can you help me please?

Thank you very much

What command did you use?

fmriprep-docker --fs-license-file /media/usuari/Datos/freesurfer/license.txt /media/usuari/Datos/MBCT_OCD /media/usuari/Datos/MBCT_OCD/out -w /media/usuari/Datos/trash --participant-label sub-050101602B --use-aroma --output-spaces MNI152NLin2009cAsym
RUNNING: docker run --rm -e DOCKER_VERSION_8395080871=20.10.17 -it -v /media/usuari/Datos/freesurfer/license.txt:/opt/freesurfer/license.txt:ro -v /media/usuari/Datos/MBCT_OCD:/data:ro -v /media/usuari/Datos/MBCT_OCD/out:/out -v /media/usuari/Datos/trash:/scratch nipreps/fmriprep:22.0.2 /data /out participant --participant-label sub-050101602B --use-aroma -w /scratch --output-spaces MNI152NLin2009cAsym
Traceback (most recent call last):
  File "/opt/conda/bin/fmriprep", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/cli/run.py", line 39, in main
    parse_args()
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/cli/parser.py", line 764, in parse_args
    config.from_dict({})
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 671, in from_dict
    execution.load(settings, init=initialize('execution'), ignore=ignore)
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 232, in load
    cls.init()
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 472, in init
    cls._layout = BIDSLayout(
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/layout.py", line 152, in __init__
    indexer(self)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 109, in __call__
    self._index_dir(self._layout._root, self._config)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 193, in _index_dir
    self._index_dir(d, list(config), default_action=default)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 193, in _index_dir
    self._index_dir(d, list(config), default_action=default)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 173, in _index_dir
    _, dirnames, filenames = next(os.walk(path))
StopIteration
fMRIPrep: Please report errors to https://github.com/nipreps/fmriprep/issues

What version of fMRIPrep are you running?

fmriprep-docker-22.0.2

How are you running fMRIPrep?

Docker

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

fmriprep-docker --fs-license-file /media/usuari/Datos/freesurfer/license.txt /media/usuari/Datos/MBCT_OCD /media/usuari/Datos/MBCT_OCD/out -w /media/usuari/Datos/trash --participant-label sub-050101602B --use-aroma --output-spaces MNI152NLin2009cAsym
RUNNING: docker run --rm -e DOCKER_VERSION_8395080871=20.10.17 -it -v /media/usuari/Datos/freesurfer/license.txt:/opt/freesurfer/license.txt:ro -v /media/usuari/Datos/MBCT_OCD:/data:ro -v /media/usuari/Datos/MBCT_OCD/out:/out -v /media/usuari/Datos/trash:/scratch nipreps/fmriprep:22.0.2 /data /out participant --participant-label sub-050101602B --use-aroma -w /scratch --output-spaces MNI152NLin2009cAsym
Traceback (most recent call last):
  File "/opt/conda/bin/fmriprep", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/cli/run.py", line 39, in main
    parse_args()
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/cli/parser.py", line 764, in parse_args
    config.from_dict({})
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 671, in from_dict
    execution.load(settings, init=initialize('execution'), ignore=ignore)
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 232, in load
    cls.init()
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 472, in init
    cls._layout = BIDSLayout(
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/layout.py", line 152, in __init__
    indexer(self)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 109, in __call__
    self._index_dir(self._layout._root, self._config)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 193, in _index_dir
    self._index_dir(d, list(config), default_action=default)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 193, in _index_dir
    self._index_dir(d, list(config), default_action=default)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 173, in _index_dir
    _, dirnames, filenames = next(os.walk(path))
StopIteration
fMRIPrep: Please report errors to https://github.com/nipreps/fmriprep/issues

Additional information / screenshots

No response

[effigies]

Thanks for this. Could you add --shell to your command to open up a bash prompt inside the container? Inside it, run:

python -m pdb -m fmriprep /data /out participant --participant-label sub-050101602B --use-aroma -w /scratch --output-spaces MNI152NLin2009cAsym

This will open a debug shell. It will stop immediately, and you should continue by typing c and hitting enter. Then when you get to the error, you will again have an opportunity to inspect the current state of the program. When you're at this line:

  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 173, in _index_dir
    _, dirnames, filenames = next(os.walk(path))

You can type print(path) to see what path is failing. It would also be helpful to go up in the stack by typing up to here:

  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 193, in _index_dir
    self._index_dir(d, list(config), default_action=default)

See what d is with print(d). Go up one more time and do that again. Then exit with q. Paste all of this here.

[Tordelap]

fmriprep-docker --fs-license-file /media/usuari/Datos/freesurfer/license.txt /media/usuari/Datos/MBCT_OCD /media/usuari/Datos/MBCT_OCD/out -w /media/usuari/Datos/trash --participant-label sub-050101602B --use-aroma --output-spaces MNI152NLin2009cAsym --shell
RUNNING: docker run --rm -e DOCKER_VERSION_8395080871=20.10.17 -it -v /media/usuari/Datos/freesurfer/license.txt:/opt/freesurfer/license.txt:ro -v /media/usuari/Datos/MBCT_OCD:/data:ro -v /media/usuari/Datos/MBCT_OCD/out:/out -v /media/usuari/Datos/trash:/scratch --entrypoint=bash nipreps/fmriprep:22.0.2
(base) root@76bfcf3338ea:/tmp# python -m pdb -m fmriprep /data /out participant --participant-label sub-050101602B --use-aroma -w /scratch --output-spaces MNI152NLin2009cAsym
> /opt/conda/lib/python3.9/site-packages/fmriprep/__main__.py(23)<module>()
-> from .cli.run import main
(Pdb) c
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/pdb.py", line 1724, in main
    pdb._runmodule(mainpyfile)
  File "/opt/conda/lib/python3.9/pdb.py", line 1560, in _runmodule
    self.run(code)
  File "/opt/conda/lib/python3.9/bdb.py", line 580, in run
    exec(cmd, globals, locals)
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/__main__.py", line 23, in <module>
    from .cli.run import main
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/cli/run.py", line 39, in main
    parse_args()
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/cli/parser.py", line 764, in parse_args
    config.from_dict({})
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 671, in from_dict
    execution.load(settings, init=initialize('execution'), ignore=ignore)
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 232, in load
    cls.init()
  File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 472, in init
    cls._layout = BIDSLayout(
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/layout.py", line 152, in __init__
    indexer(self)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 109, in __call__
    self._index_dir(self._layout._root, self._config)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 193, in _index_dir
    self._index_dir(d, list(config), default_action=default)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 193, in _index_dir
    self._index_dir(d, list(config), default_action=default)
  File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 173, in _index_dir
    _, dirnames, filenames = next(os.walk(path))
StopIteration
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /opt/conda/lib/python3.9/site-packages/bids/layout/index.py(173)_index_dir()
-> _, dirnames, filenames = next(os.walk(path))
(Pdb) print(path)
/data/sub-050202002B/anat
(Pdb) up
> /opt/conda/lib/python3.9/site-packages/bids/layout/index.py(193)_index_dir()
-> self._index_dir(d, list(config), default_action=default)
(Pdb) print(d)
/data/sub-050202002B/anat
(Pdb) up
> /opt/conda/lib/python3.9/site-packages/bids/layout/index.py(193)_index_dir()
-> self._index_dir(d, list(config), default_action=default)
(Pdb) print(d)
/data/sub-050202002B
(Pdb) q
Post mortem debugger finished. The fmriprep will be restarted
> /opt/conda/lib/python3.9/site-packages/fmriprep/__main__.py(23)<module>()
-> from .cli.run import main
(Pdb) 

[Tordelap]

Thank you very much, I have done what you said and pasted it above

[effigies]

Is the sub-050202002B/anat directory empty?

[Tordelap]

It is now, but I do not know how that happened, its the only missing files. It had the NIFTI and json files as all others

[Tordelap]

The directory seems to be corrupted

[effigies]

Okay. This should be a PyBIDS issue to handle this case more gracefully, but there's nothing to do in fMRIPrep.

[effigies]

I've opened an issue at bids-standard/pybids#925.

[Tordelap]

Yes, I just performed BIDS validation again and something in the functional json files is not quite right. This error was not shown before, sorry. I will fix this and rerun fMRIPrep. Thank you very much!