bricks.stat.spm.EstimateModel writes the pyscript.m file to populse_mia package

[servoz]

For now, I have no idea of the exact cause of the issue and where it comes from (mia_processes, populse_mia, capsul, nipype, etc.)?. I'm opening this ticket in mia_processes but maybe it concerns another package ...

The mia_processes.bricks.stat.spm.EstimateModel brick writes the pyscript.m file to the root of the populse_mia package.
It seems that only this brick of mia_processes produces this bug.

• In developer mode (python main.py):
  $ pwd
  /home/econdami/Data/Git_Projects/populse_mia/populse_mia
  $ ls -l
  total 540
  drwxrwxr-x. 3 econdami econdami 4096 Sep 6 15:17 data_manager
  -rw-rw-r--. 1 econdami econdami 4097 Jul 3 11:51 info.py
  -rw-rw-r--. 1 econdami econdami 2291 Jun 29 09:50 __init__.py
  -rwxrwxr-x. 1 econdami econdami 614 Jun 29 09:50 __main__.py
  -rw-rw-r--. 1 econdami econdami 59549 Jun 29 09:50 main.py
  drwxrwxr-x. 2 econdami econdami 4096 Sep 13 15:44 __pycache__
  -rw-rw-r--. 1 econdami econdami 733 Sep 19 15:40 pyscript.m
  -rw-rw-r--. 1 econdami econdami 55810 Aug 30 16:00 software_properties.py
  drwxrwxr-x. 3 econdami econdami 4096 Jul 10 12:18 sources_images
  -rw-rw-r--. 1 econdami econdami 389220 Sep 6 15:17 test.py
  drwxrwxr-x. 6 econdami econdami 4096 Aug 30 16:00 user_interface
  drwxrwxr-x. 3 econdami econdami 4096 Jul 10 12:18 utils

• In user mode (python -m populse_mia):
  $ pwd
  /home/econdami/.local/lib/python3.9/site-packages/populse_mia-2.4.1.dev0+8e6ba826-py3.9.egg/populse_mia
  $ ls -l
  total 540
  drwxrwxr-x. 3 econdami econdami 4096 Sep 19 14:55 data_manager
  -rw-rw-r--. 1 econdami econdami 4097 Sep 19 14:55 info.py
  -rw-rw-r--. 1 econdami econdami 2291 Sep 19 14:55 __init__.py
  -rw-rw-r--. 1 econdami econdami 614 Sep 19 14:55 __main__.py
  -rw-rw-r--. 1 econdami econdami 59549 Sep 19 14:55 main.py
  drwxrwxr-x. 2 econdami econdami 4096 Sep 19 14:55 __pycache__
  -rw-rw-r--. 1 econdami econdami 733 Sep 19 15:20 pyscript.m
  -rw-rw-r--. 1 econdami econdami 55810 Sep 19 14:55 software_properties.py
  drwxrwxr-x. 3 econdami econdami 4096 Sep 19 14:55 sources_images
  -rw-rw-r--. 1 econdami econdami 389220 Sep 19 14:55 test.py
  drwxrwxr-x. 6 econdami econdami 4096 Sep 19 14:55 user_interface
  drwxrwxr-x. 3 econdami econdami 4096 Sep 19 14:55 utils

It's a pollution we can't accept (we don't have to write in the site-packages!!!!).

I've already spent several hours trying to figure out what's wrong, but I haven't yet managed to understand exactly what's going wrong...

[servoz]

The script is saved in the directory where populse_mia is launched (main.py or _main.py), exactly here, when process.spm_mat_file is defined (due to the capsul sync_process_output_traits() call back).
I have the feeling that something is sub-optimal between capsul and nipype in some cases ... I still haven't managed to work out what and how ...!

[servoz]

Is this ticket linked with the one in capsul?

I'm beginning to understand what's going on ... maybe it will be possible to fix 2 tickets in one go ...?
I'll make a summary ASAP.

[denisri]

Yes it's linked, sure !
The problem is that nipype writes this file in the current working directory, and this is not what we normally do in capsul and populse_mia. A possible solution would be to run nipype interfaces in a temp directory (as I think we already do for SPM outputs wrapping), this can be handled in the nipype wrapper.

[servoz]

Yes, you're right, we're already running nipype interfaces in a temporary directory, but the problem is that nipype makes system calls with (or without writing) pyscript files in some cases, during the module initialisation steps and at this stage we haven't yet changed the current directory to a temporary one.

I've spent some time trying to understand what's going on between populse and nipype (which has given me a better understanding of the machinery involved in running a process!) and it seems to me that we could perhaps simply change the current directory to a temporary directory just before running the process in capsul.process.process.Process.run_from_commandline(), then return to the initial current directory after execution?
It's simple (and therefore generally robust!) and can be effective in all (or many) cases.
Maybe I've forgotten something, a side effect?

I make a PR with a proposal along these lines, and that'll get us started for discussion if needed. From my point of view, it fix the #52 ticket.

[servoz]

Fixed.

[servoz]

I'm reopening this ticket because I just see again a pyscript generated with the CVR pipeline ... just to keep in mind, I take again this ASAP ticket ... Arggghh ...