[Todo for V2] Reconnecting the iteration

[servoz]

It is pressing need for the clinical application, but also for the research activity, to reconnect the iteration.

At least as it existed in V1 (selection of inputs automatically from the database with filters).

There were imperfections in the V1 iteration (the main one being that there was no initialisation step separated from the run).

The initialisation step, as it currently exits, could be made to disappear.
Indeed, at present, it make inappropriate operations (e.g. outputs are entered into the database when they do not already exist, etc.) in addition to verifying that the run step should run without trouble.
There is yet a lot of thinking to be done on this initialisation step and decisions to be made on exactly how it will work as it may have impact in the code for iteration (and not only!).

But maybe we are getting out of the scope of this ticket . Getting it back would be nice for now :-)

[denisri]

Databasing needs a complete check / rework (independently of iterations), that's sure (and discussed in #181).
For iterations, for me they still work in the branch populse_db2_capsulengine, in a different way as they used to work

• the iteration button puts the pipeline into an iterative node, replacing all parameters with lists
• it is possible to fill in parameters using the "+" / "filter" buttons of parameters: to iterate over 10 data, just select 10 data in the list for the main input parameters. For processes/pipelines with a working completion system, other parameters will be filled in automatically when the init is done. When processes inside the iteration were already lists (like for the mia_processes/spm/smooth brick), we end up with lists of lists, and currently the GUI to enter them is not user friendly at all - but at least it works. A workaround for that is to use a "Files to list" brick and connect it before the smooth brick (in our example), before iterating. This way the main input of the iterated pipeline is a File, and in the iteration we have a list of File - which can be filled using "Filter" and a single database query.
• the iteration filter buttons (top left of the main GUI) have just been [partly at least ?] fixed. When iteration is on, they filter the database_scans main input. We can use an "input filter" connected to it to add a filter and plug it to the main input of the iteration node. When the pipeline is initialized, the input filter will generate a filtered list of inputs, then completion will happen in the iteration pipeline.

Thus the whole chain of operations to build the iteration pipeline is not straightforward, and a real work is needed to make it user friendly, but once the pipeline is built, it works for the user just like a regular pipeline. I dare to say that the former way of using iterations (in v1) was not really easier and intuitive to setup.

[denisri]

To me the "double filtering", one in the iteration GUI, and the other through "input filers" is strange, non-intuitive, and difficult to use for an inexperienced user who has to select filtering at several places.

The iteration GUI is not very clear to me: it splits the whole database according to a given tag values, and produces lists of lists of scans. But I don't understand how/where they are used. The database_scans input is a single list, thus receiving the re-union of the above split (for selected values if "Filter" has been used).

The "Filter" button in the iteration GUI opens a dialog which should be entirely redesigned: all tag values are displayed in a single page, with checkbox buttons for each value. Every single one has to be clicked to actually select/unselect values. In our databases we have thousands, sometimes tenths of thousands, and soon hundreds of thousands of subjects: they don't fit in a single page, and it's not possible to select every one by hand.
Even when we have only, say, 30 values, [un]selecting them is already a pain. We need a better GUI for that, at least a table or listview widget with extended selection via ranges of items.
But for very large databases every single value should probably not be displayed at all. This is a deeper discussion as scans lists are stored in python variables (scans_list etc) which already negates the advantages of using a database. Well we will look at this later, it's not the time for that. The same remark applies for the data browser: the entire database is exposed and displayed in a single table, which is probably very inefficient when using really large databases. This is not an hypothetical use case: we already manipulate "in real life" databases with 30000 subjects (which means millions of files in the database), not in MIA yet, but it just has to work for this size, otherwise it will be useless. Well, not in v2, OK.

[denisri]

To improve the GUI and simplify user experience, maybe we could do this:

• when the user clicks the "iteration" button, a dialog displays the list of all parameters, and allows to select for each, if it will not be iterated at all (fixed value for all iterations), or iterated the "standard way" (transformed into a list), or connected to the database_scans (via an input filter node). By default all will be iterated the standard way.
• Then the new iterative pipeline is automatically built from it, and appropriate input filter nodes inserted and connected.
• The user then "just" has to edit filter(s), then init and run.

[denisri]

The improvements above have been implemented.
There is still a database indexation issue in iterations since the same process instances will get different UUIDs and different parameters for each iteration, and this is not taken into account in the system.

[servoz]

I just started again to try to do something with the iteration, but I crash mia in less than 30s ...

Please, can you give a minimum step to run, say, mia_processes.pipelines.preprocessing.Spatial_preprocessing_1, with 3 patients?

[denisri]

I admit I didn't try this pipeline after reworking the iteration GUI, and there are a few bugs left that show up on it. I'm looking at it.

[denisri]

I have a problem when iterating a pipeline without exporting its plugs, in this case all nodes inside the iterated pipeline get disabled (the pipeline inside the iteration is actually a pipeline, with everything connected, but all nodes disabled). I have to find what is going on. When exporting the plugs before iterating, things seem to go better (I have not run it yet), but there is something strange there. I'm investigating...

[denisri]

There are also parameters setting problems in nipype nodes (or Mia processes containing nipype nodes, I don't know) - some parameters get the same value for all iterations, which is wrong.

[servoz]

I don't understand.
If I want to iterate a smooth on 5 patients and I want the same smoothing for all, I have to take the same value of Gaussian smoothing kernel? You must be talking about something else I think?

[denisri]

If you iterate on a process, each parameter can become a list, so that you can use either the same value for all iterations, or a different value for each. This is true for all parameters (input images, output files, numeric parameters etc).
But right now in some processes inside the spatial_preprocessing pipeline, some processes with several input images get the same value for all iterations even if we specify different values from the user side. This is a bug, and I guess it's due to exceptions during traits setting while setting nipype interface parameters (when we set several parameters, one after another, there are moments when all are not consistent, and parameters checks can trigger some exceptions, I guess). I did not have time to figure that out, and am quite busy today so I don't know if I can find it out today.

[denisri]

I have reworked several problems showing up in iterations (not all directly linked to iterations), in the MIA parameters completion engine, in the node controllers which were not correctly destroyed and were firing notifications long after they were dead, in populse_mia, in capsul and in soma-base. I also had problems of parameters checking which were different depending on nipype versions. I have pushed a "more stable" version now which seems to work for me on 2 different machines installed differently (one in a containerized build, the other direclty on the system).
I have reworked the part of the docs of the pipeline manager to explain the new iteration system (and several ways to use it).
I hope it's getting better...

[servoz]

I finally have some time to test the new iteration.
Thank you very much for the documentation you have written about it.
I have started to follow it.
I have a problem from the start:

Direct iteration

Starting with a new, empty pipeline tab in the Pipeline Manager:

• Add the pipeline mia_processes > pipelines > preprocess > spatial Preprocessing 1 to the pipeline editor => done, OK
• Use the “capsul > pipeline > custom_nodes > reduce_node > ReduceNode” brick. validate the default node parameters. The reduce node appears in the pipeline editor => done, OK
• connect the outputs plug of the reduce node to the func_files plug of the spatial_preprocessing node => done, not working
  impossible to link outputs of reducenode to func_files of spatial_preprocessing_1.
  The stdout:
  source to destination types are not compatible

[servoz]

Ok I succeeded using "mia_processes > bricks > tools> Files_To_List" brick instead of "capsul > pipeline > custom_nodes > reduce_node > ReduceNode" brick (it may be that the problem is caused because the default value of ReduceNode is [''] and func_files expects either a list of existing paths or []?)
After initialisation step the run is now available.
While looking at the data browser before launching the run, I am very surprised to observe that only the outputs of the second patient are indexed. Is there a problem with the initialisation that indexes only the last patient (I have iterated on two patients).
Anyway, I'm running it to see what happens!

[servoz]

Ok, after the run I get all the expected results in the derived_data directory but as expected only the last patient of the iteration is indexed in the database.

So this is a great advance, the direct mode iteration seems to work !!

It remains to be seen:

• Could the ReduceNode process be initialised to [] rather than ['']?
• index all data in the database and not only the last iteration cycle (I will open a ticket for this).

After these two remarks, I find this mode very interesting. The only reservation I see is the possibility of an easy error if we have a lot of data to manage because the lists must be in the same patient order.
I think that this mode is therefore reserved for a small number of patients and/or an extremely concentrated user :-))

[denisri]

ReduceNode:
Aha I had run into this problem and thought I had fixed it in Capsul, but... as usual I forgot to push into the origin repository ;)
It's done now, you need to pull capsul/master.

The database issue is more difficult and will probably need some work: in MIA process runs are indexed using a process UUID which is associated with a process instance, not to a "run". In iterations the same instance will be reused several times (for each iteration), so we have to make changes in this infrastructure. So yes it deserves a separate ticket.

[servoz]

Oh Yeah !!!!
After pulling the latest version of capsul there are no more problems with the ReduceNode process!
So there is just the indexing issue to be fixed (and that certainly requires a bit more work!). I opened a ticket about it !

[servoz]

I'm still testing:

Via input filters - Quickly

• It works perfectly well.
• We observe the same indexing problem as for the Direct iteration mode (only the last patient of the list is indexed in the database) which is natural because the same causes produce the same effects. On the other hand, as for the Direct iteration mode, all the expected data can be indeed found in the derived_data directory of the project.
• Again, the only reservation I see, is the possibility of an easy error if we have a lot of data to manage because the various input lists must be in the same patient order (as indicated in the documentation by the TODO: check that anats and corresponding functional files are in the same order… The database filters do not ensure that and do not allow to specify any order….). I have no idea how to make this check automatically (and above all, that it works every time!!)
• Again, I think this mode is very interesting but may be to keep for a small number of patients and/or an extremely concentrated user :-))

[servoz]

I'm still testing:

Via input filters - Manually

The documentation for this part is a bit more complex to follow and may be partly obsolete (there are several links, some of which seem to correspond to the V1 iteration), which may lead to confusion. I will rewrite this part when I have managed to run this mode, which at first sight is the closest to the iteration in V1 (which had the advantage for the user not to have to worry about the order of the patients). As I had a bit of trouble understanding the documentation for this part (maybe a leftover influence from the V1 iteration method?), maybe I did it wrong, but the initialisation fails and I observe the following exceptions:

Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casaTraceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
- Pipeline initialising ...

Completion ...
Exception occurred in traits notification handler for object: <populse_mia.user_interface.pipeline_manager.process_mia.MIAProcessCompletionEngine object at 0x7f82775dadb0>, trait: completion_progress, old value: 0.0, new value: 1.05
Traceback (most recent call last):
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 524, in _dispatch_change_event
    self.dispatch(handler, *args)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 619, in dispatch
    handler(*args)
  File "/casa/host/src/capsul/master/capsul/qt_gui/widgets/attributed_process_widget.py", line 430, in _completion_progress_changed
    if not hasattr(self, 'progressdialog'):
RuntimeError: wrapped C/C++ object of type AttributedProcessWidget has been deleted

Error during initialisation of the "NoName" pipeline ...!
Traceback:
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_manager_tab.py", line 1204, in initialize
    self.test_init = self.init_pipeline()
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_manager_tab.py", line 1268, in init_pipeline
    self.complete_pipeline_parameters(pipeline)
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_manager_tab.py", line 985, in complete_pipeline_parameters
    completion.complete_parameters()
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/process_mia.py", line 310, in complete_parameters
    self.completion_progress = self.fallback_engine.completion_progress
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 478, in __call__
    self.notify_listener(self, object, trait_name, old, new)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 553, in _notify_method_listener
    object, trait_name, old, new, listener
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 532, in _dispatch_change_event
    handle_exception(object, trait_name, old, new)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 149, in _handle_exception
    raise excp
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 524, in _dispatch_change_event
    self.dispatch(handler, *args)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 619, in dispatch
    handler(*args)
  File "/casa/host/src/capsul/master/capsul/qt_gui/widgets/attributed_process_widget.py", line 430, in _completion_progress_changed
    if not hasattr(self, 'progressdialog'):
RuntimeError: wrapped C/C++ object of type AttributedProcessWidget has been deleted
/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
Traceback (most recent call last):
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_editor.py", line 1789, in update_scans_list
    setattr(pipeline, 'database_scans', self.scan_list)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2515, in validate
    return TraitListObject(self, object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 585, in __init__
    notifiers=[self.notifier],
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_list_object.py", line 865, in _item_validator
    return trait_validator(object, self.name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_types.py", line 2517, in validate
    self.error(object, name, value)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/base_trait_handler.py", line 75, in error
    object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'database_scans' trait of a Pipeline instance must be a list of items which are a filename or object implementing the os.PathLike interface, but a value of 'data/raw_data/coug250816_test25032019-IRMFonct_+perfusion-2016-08-25094516-04-T13DSENSE-T1TFE-000424_000.nii' <class 'str'> was specified.
- Pipeline initialising ...

Completion ...
Exception occurred in traits notification handler for object: <populse_mia.user_interface.pipeline_manager.process_mia.MIAProcessCompletionEngine object at 0x7f82775dadb0>, trait: completion_progress, old value: 0.0, new value: 1.05
Traceback (most recent call last):
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 524, in _dispatch_change_event
    self.dispatch(handler, *args)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 619, in dispatch
    handler(*args)
  File "/casa/host/src/capsul/master/capsul/qt_gui/widgets/attributed_process_widget.py", line 430, in _completion_progress_changed
    if not hasattr(self, 'progressdialog'):
RuntimeError: wrapped C/C++ object of type AttributedProcessWidget has been deleted

Error during initialisation of the "NoName" pipeline ...!
Traceback:
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_manager_tab.py", line 1204, in initialize
    self.test_init = self.init_pipeline()
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_manager_tab.py", line 1268, in init_pipeline
    self.complete_pipeline_parameters(pipeline)
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_manager_tab.py", line 985, in complete_pipeline_parameters
    completion.complete_parameters()
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/process_mia.py", line 310, in complete_parameters
    self.completion_progress = self.fallback_engine.completion_progress
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 478, in __call__
    self.notify_listener(self, object, trait_name, old, new)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 553, in _notify_method_listener
    object, trait_name, old, new, listener
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 532, in _dispatch_change_event
    handle_exception(object, trait_name, old, new)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 149, in _handle_exception
    raise excp
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 524, in _dispatch_change_event
    self.dispatch(handler, *args)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 619, in dispatch
    handler(*args)
  File "/casa/host/src/capsul/master/capsul/qt_gui/widgets/attributed_process_widget.py", line 430, in _completion_progress_changed
    if not hasattr(self, 'progressdialog'):
RuntimeError: wrapped C/C++ object of type AttributedProcessWidget has been deleted

So, there is a traits.trait_errors.TraitError issue.
After lunch, I will go over this part carefully.

[denisri]

Well I don't know how you have built your pipeline here, but I agree the doc for this use case is not clear enough since it actually groups several cases.

[servoz]

Iteration Via input filters - Manually

OK, I did it, but the documentation is obsolete and it is not worth following for this part. I will rewrite it.
Which I did:
Starting from a project where the Exp Type and PatientName tags are correctly filled in

From now, it is quite simple. We have just to follow the Via input filters - Quickly steps.

• right-click on the anat_file_filter node, and select “Open filter”. In the filter pop-up, modify the filter to apply to select the 'anatomical' data
  
• similarly, right-click on the “func_files_filter” node, and select “Open filter”. In the filter pop-up, modify the filter to apply to select 'functional' data
  
• then, in the iteration section (top right of the pipeline manager tab), click on the "iterate over" feature ("Select" button).
  • a dialog pops up and displays all the tags in the current project. Check the tags on which the pipeline will be iterated, a good choice could be here the PatientName tag.
  • there is now under "Iterate over", the name of the previously chosen tag: PatientName.
  • next to it there is a Combobox where it is possible to see all the values of the selected tags used to iterate.
    
• it is possible to change the values used for the iteration by clicking on the next "Filter" button. In the example below we only want to iterate on the alej" and "rocmi" patients
  
• click on “Initialize pipeline”. The Run button becomes enabled.
  • as for the previous modes described above only the last patient is indexed in the database but all the expected data can be found in the derived_data directory of the project.
• click on “Run pipeline”.
• We keep here the idea of iteration as it was in V1 (no need to take care of the order of the input parameters and also to use the power of the database to select the necessary data) and I think this new version seems even better :-)

There are certainly still some issues to be solved. For example after the initialisation if we go to see the data browser
For example after the initialisation if we go to see the data browser, then returning to the pipeline manager the run button is no longer available (you have to initialise the pipeline again). Similarly if we go to see a parameter (without changing it) we have to initialise the pipeline again before to run. This is not the case with a normal pipeline and it is not advisable. As the first observed side effect of this behaviour:

1. we do an initialisation
2. we go to see the data browser: we can see that only the last patient is indexed (ok it's a ticket already opened)
3. we come back to the pipeline manager: run is no longer available. We redo an initialisation.
4. we go to see the data browser: we see that there is no more output data indexed at all (everything has disappeared only the initial data is still indexed)
5. we come back to the pipeline manager: run is no longer available.
6. go to step 1., etc.

It looks like a bug. I think we can open a ticket for that (loss of the run button and loss of all indexing if a new initialisation is done, in fact this makes 2 tickets ..)

[servoz]

In order to close this ticket and to propose a new documentation as clear as possible, for the iteration part, I start again to test from the beginning the different possibilities of use.
I am really surprised to see that what should be a simple formality (the first case I tested recently and that worked very well) does not seem to work every time...

• Direct Iteration (current documentation)
  • Add the pipeline mia_processes > pipelines > preprocess > spatial Preprocessing 1 to the pipeline editor => Done, OK
  • Use the “capsul > pipeline > custom_nodes > reduce_node > ReduceNode” brick (validate the default node parameters. The reduce node appears in the pipeline editor). => Done, OK
  • connect the outputs plug of the reduce node to the func_files plug of the spatial_preprocessing node => Done, OK
  • export the input_0 plug of the reduce node, renamed as func_files (for clarity) => Done, OK
  • export all unconnected plugs of the spatial preprocessing node (right click on the node then select “export all unconnected plugs”) => Done, OK
  • check on the “iterate pipeline” button (just click “OK” on the dialog pop-up) => Done, OK
  • select the inputs node => Done, OK
  • click the “Filter” button for the anat files parameter, and select the files (anatomical MRIs) you wish to extract the brain from. => Done, OK
  • similarly, click on the “Filter” button for the func_files parameter and select the same number of functional files.=> Done, OK
  • Click on “Initialize pipeline”. => Done, OK
  • The Run button becomes enabled. => OK

-> Note to add to ticket #207: Only the last patient in the iteration is indexed in the database. If we go to the data browser and return to the pipeline manager, the run button remains enabled.

• Run the pipeline => Done, the run fall when calculating for the second patient (the first patient worked well; test iteration on 2 patients).

Stdout:

- Pipeline running ...


. spatial_preprocessing_1_1.list_duplicate1 (mia_processes.bricks.tools.tools.List_Duplicate) MIA node ...

During the <mia_processes.bricks.tools.tools.List_Duplicate object at 0x7fe63d6792b0> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...


. spatial_preprocessing_1_1.realign1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Realign) MIA node ...

. spatial_preprocessing_1_1.newsegment1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.NewSegment) MIA node ...

. spatial_preprocessing_1_1.coregister1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Coregister) MIA node ...

. spatial_preprocessing_1_1.normalize12_1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.normalize12_2 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.smooth1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Smooth) MIA node ...

. spatial_preprocessing_1_1.list_duplicate1 (mia_processes.bricks.tools.tools.List_Duplicate) MIA node ...

During the <mia_processes.bricks.tools.tools.List_Duplicate object at 0x7fe63d6792b0> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...


. spatial_preprocessing_1_1.realign1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Realign) MIA node ...

. spatial_preprocessing_1_1.newsegment1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.NewSegment) MIA node ...

. spatial_preprocessing_1_1.coregister1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Coregister) MIA node ...

. spatial_preprocessing_1_1.normalize12_1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.normalize12_2 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.smooth1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Smooth) MIA node ...
soma-workflow starting in light mode
Workflow controller initialised
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
show_status
connect to computing resource: irmage-pteric
Lauching workflow controller
soma-workflow starting in light mode
Workflow controller initialised
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
show_status
connect to computing resource: irmage-pteric
Lauching workflow controller
soma-workflow starting in light mode
Workflow controller initialised
ComputingResourcePool delete_connection: irmage-pteric
del WFC
del WorkflowController
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
error while importing outputs in ProcessIteration
outputs: {}
postprocess pipeline: <capsul.pipeline.pipeline.Pipeline object at 0x7fe63d827938>
set exec status on: 597670ff-8e74-493a-9f39-52fb1c595eb9 2021-07-07 11:54:05
set exec status on: c314057f-7415-4bae-86ad-005b51e3bac4 2021-07-07 11:54:02
update file history for: data/raw_data/alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii : ['a0bd0ca7-2cf1-4187-ba0a-d26fcc604ea2', 'c314057f-7415-4bae-86ad-005b51e3bac4'] -> ['c314057f-7415-4bae-86ad-005b51e3bac4']
obsolete bricks: set()
really orphan: set()
  =================================================
environ({'SINGULARITY_COMMAND': 'run', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'LD_LIBRARY_PATH': '/casa/host/build/lib:/casa/host/lib:/usr/local/lib/mesa:/.singularity.d/libs:/usr/local/lib', 'CASA_SYSTEM': 'ubuntu-18.04', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'BRAINVISA_BVMAKER_CFG': '/casa/host/conf/bv_maker.cfg', 'LANG': 'C.UTF-8', 'CASA_INSTALL': '/casa/host/install', 'DISPLAY': ':0', 'CASA_DISTRO': 'opensource', 'CASA_SRC': '/casa/host/src', 'CASA_CONF': '/casa/host/conf', 'CASA_PACK': '/casa/host/pack', 'CASA_ENVIRONMENT': 'opensource-master-5.0', 'SOFTWARE_OPENGL': '1', 'BRAINVISA_HOME': '/casa/host/build', 'SSH_AUTH_SOCK': '/casa/ssh_auth_sock', 'CASA_HOST_DIR': '/home/econdami/casa_distro/brainvisa-opensource-master', 'BRAINVISA_UNENV_PATH': '/casa/host/build/bin:/casa/casa-distro/cbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/casa/host/bootstrap/brainvisa-cmake/bin:/casa/bootstrap/brainvisa-cmake', 'SINGULARITY_CONTAINER': '/home/econdami/casa_distro/casa-dev-5.0-1_wr', 'PWD': '/casa/home/Git_projects/populse_mia/python/populse_mia', 'HOME': '/casa/home', 'QT_API': 'pyqt5', 'BRAINVISA_SHARE': '/casa/host/build/share', 'CASA_BRANCH': 'master', 'BRAINVISA_UNENV_PYTHONPATH': '', 'LC_NUMERIC': 'C', 'CASA_HOST_HOME': '/home/econdami', 'SINGULARITY_ENVIRONMENT': '/.singularity.d/env/91-environment.sh', 'TERM': 'xterm-256color', 'SINGULARITY_NAME': 'casa-dev-5.0-1_wr', 'BRAINVISA_UNENV_LD_LIBRARY_PATH': '/casa/host/lib:/usr/local/lib/mesa:/.singularity.d/libs:/usr/local/lib', 'BRAINVISA_UNENV_DCMDICTPATH': '', 'SHLVL': '1', 'PYTHONPATH': '/casa/host/build/python', 'CASA_TESTS': '/casa/host/tests', 'CASA_TYPE': 'dev', 'XDG_RUNTIME_DIR': '/tmp/runtime-econdami', 'XAUTHORITY': '/casa/Xauthority', 'PATH': '/casa/host/src/development/casa-distro/master/cbin:/casa/host/build/bin:/casa/host/build/bin:/casa/casa-distro/cbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/casa/host/bootstrap/brainvisa-cmake/bin:/casa/bootstrap/brainvisa-cmake', 'PS1': '\\[\\e]0;(opensource-master-5.0)\\u@\\h: \\w\\a\\]${CASA_ENVIRONMENT:+($CASA_ENVIRONMENT)}\\[\\033[01;32m\\]\\u@\\h\\[\\033[00m\\]:\\[\\033[01;34m\\]\\w\\[\\033[00m\\]\\$ ', 'XLIB_SKIP_ARGB_VISUALS': '1', 'SINGULARITY_BIND': '/home/econdami/casa_distro/brainvisa-opensource-master:/casa/host,/:/host,/mnt:/mnt,/media:/media,/srv:/srv,/home:/home,/data:/data,/usr:/usr_host,/run/media:/run_media,/home/econdami/casa_distro/brainvisa-opensource-master/home:/casa/home,/tmp/casa-distro-0_p90564.Xauthority:/casa/Xauthority,/run/user/1000/keyring/ssh:/casa/ssh_auth_sock,/home/econdami', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'CASA_BUILD': '/casa/host/build', 'OLDPWD': '/casa/home', '_': '/usr/bin/python3', 'NO_ET': '1', 'ETS_TOOLKIT': 'qt4'})
  ===========================================herm======
environ({'SINGULARITY_COMMAND': 'run', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'LD_LIBRARY_PATH': '/casa/host/build/lib:/casa/host/lib:/usr/local/lib/mesa:/.singularity.d/libs:/usr/local/lib', 'CASA_SYSTEM': 'ubuntu-18.04', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'BRAINVISA_BVMAKER_CFG': '/casa/host/conf/bv_maker.cfg', 'LANG': 'C.UTF-8', 'CASA_INSTALL': '/casa/host/install', 'DISPLAY': ':0', 'CASA_DISTRO': 'opensource', 'CASA_SRC': '/casa/host/src', 'CASA_CONF': '/casa/host/conf', 'CASA_PACK': '/casa/host/pack', 'CASA_ENVIRONMENT': 'opensource-master-5.0', 'SOFTWARE_OPENGL': '1', 'BRAINVISA_HOME': '/casa/host/build', 'SSH_AUTH_SOCK': '/casa/ssh_auth_sock', 'CASA_HOST_DIR': '/home/econdami/casa_distro/brainvisa-opensource-master', 'BRAINVISA_UNENV_PATH': '/casa/host/build/bin:/casa/casa-distro/cbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/casa/host/bootstrap/brainvisa-cmake/bin:/casa/bootstrap/brainvisa-cmake', 'SINGULARITY_CONTAINER': '/home/econdami/casa_distro/casa-dev-5.0-1_wr', 'PWD': '/casa/home/Git_projects/populse_mia/python/populse_mia', 'HOME': '/casa/home', 'QT_API': 'pyqt5', 'BRAINVISA_SHARE': '/casa/host/build/share', 'CASA_BRANCH': 'master', 'BRAINVISA_UNENV_PYTHONPATH': '', 'LC_NUMERIC': 'C', 'CASA_HOST_HOME': '/home/econdami', 'SINGULARITY_ENVIRONMENT': '/.singularity.d/env/91-environment.sh', 'TERM': 'xterm-256color', 'SINGULARITY_NAME': 'casa-dev-5.0-1_wr', 'BRAINVISA_UNENV_LD_LIBRARY_PATH': '/casa/host/lib:/usr/local/lib/mesa:/.singularity.d/libs:/usr/local/lib', 'BRAINVISA_UNENV_DCMDICTPATH': '', 'SHLVL': '1', 'PYTHONPATH': '/casa/host/build/python', 'CASA_TESTS': '/casa/host/tests', 'CASA_TYPE': 'dev', 'XDG_RUNTIME_DIR': '/tmp/runtime-econdami', 'XAUTHORITY': '/casa/Xauthority', 'PATH': '/casa/host/src/development/casa-distro/master/cbin:/casa/host/build/bin:/casa/host/build/bin:/casa/casa-distro/cbin:/usr/local/sbhermin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/casa/host/bootstrap/brainvisa-cmake/bin:/casa/bootstrap/brainvisa-cmake', 'PS1': '\\[\\e]0;(opensource-master-5.0)\\u@\\h: \\w\\a\\]${CASA_ENVIRONMENT:+($CASA_ENVIRONMENT)}\\[\\033[01;32m\\]\\u@\\h\\[\\033[00m\\]:\\[\\033[01;34m\\]\\w\\[\\033[00m\\]\\$ ', 'XLIB_SKIP_ARGB_VISUALS': '1', 'SINGULARITY_BIND': '/home/econdami/casa_distro/brainvisa-opensource-master:/casa/host,/:/host,/mnt:/mnt,/media:/media,/srv:/srv,/home:/home,/data:/data,/usr:/usr_host,/run/media:/run_media,/home/econdami/casa_distro/brainvisa-opensource-master/home:/casa/home,/tmp/casa-distro-0_p90564.Xauthority:/casa/Xauthority,/run/user/1000/keyring/ssh:/casa/ssh_auth_sock,/home/econdami', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'CASA_BUILD': '/casa/host/build', 'OLDPWD': '/casa/home', '_': '/usr/bin/python3', 'NO_ET': '1', 'ETS_TOOLKIT': 'qt4'})

 When the pipeline was launched, the following exception was raised: Error during workflow execution. Status=workflow_failed.
The workflow has not been removed from soma_workflow and must be deleted manually. 
Failed jobs: [417]
Aborted/killed jobs: []
** Workflow status OK
** Jobs failure, the following jobs ended with failed status:
+ job: realign1 , status: done , exit: finished_regularly , value: 1
  commandline:
'/usr/bin/python3' '-c' 'import sys; sys.path = [\'/casa/home/Git_projects/populse_mia/python\', \'/casa/home/Git_projects/mia_processes/python\', \'/casa/home/Git_projects/populse_mia/processes\'] + sys.path; from capsul.api import Process; Process.run_from_commandline("mia_processes.bricks.preprocess.spm.spatial_preprocessing.Realign")'

** Failed jobs outputs:

+ job 417: realign1 , status: done , exit: finished_regularly , value: 1
  commandline:
  ------------:
'/usr/bin/python3' '-c' 'import sys; sys.path = [\'/casa/home/Git_projects/populse_mia/python\', \'/casa/home/Git_projects/mia_processes/python\', \'/casa/home/Git_projects/populse_mia/processes\'] + sys.path; from capsul.api import Process; Process.run_from_commandline("mia_processes.bricks.preprocess.spm.spatial_preprocessing.Realign")'

  input parameters:
  -----------------
{'in_files': ['/data/IRM/projects_mia/rel2_test/data/raw_data/rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-01-BOLD_CVR_7_53sl_ModeratePNS-FEEPI-001212_000.nii'], 'jobtype': 'estwrite', 'quality': 0.9, 'separation': 4.0, 'fwhm': 5.0, 'register_to_mean': True, 'interp': 2, 'wrap': [0, 0, 0], 'write_which': [2, 1], 'write_interp': 4, 'write_wrap': [0, 0, 0], 'write_mask': True, 'out_prefix': 'r', 'realigned_files': '/data/IRM/projects_mia/rel2_test/data/derived_data/rrocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-01-BOLD_CVR_7_53sl_ModeratePNS-FEEPI-001212_000.nii', 'modified_in_files': '/data/IRM/projects_mia/rel2_test/data/derived_data/rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-01-BOLD_CVR_7_53sl_ModeratePNS-FEEPI-001212_000.nii', 'mean_image': '/data/IRM/projects_mia/rel2_test/data/derived_data/meanrocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-01-BOLD_CVR_7_53sl_ModeratePNS-FEEPI-001212_000.nii', 'realignment_parameters': '/data/IRM/projects_mia/rel2_test/data/derived_data/rp_rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-01-BOLD_CVR_7_53sl_ModeratePNS-FEEPI-001212_000.txt', 'output_directory': '/data/IRM/projects_mia/rel2_test/data/derived_data', 'use_mcr': True, 'paths': ['/usr_host/local/SPM/spm12_standalone'], 'matlab_cmd': '/usr_host/local/SPM/spm12_standalone/run_spm12.sh /usr_host/local/MATLAB/MATLAB_Runtime/v95 script', 'mfile': True, 'spm_script_file': '/data/IRM/projects_mia/rel2_test/scripts/pyscript_realign_8dc372e9-bcbd-4290-84dd-f652eee75047.m'}

  output parameters:
  ------------------
None

  environment:
  ------------
{'SOMAWF_INPUT_PARAMS': '/casa/home/.soma-workflow/transfered_files/econdami_1/25992', 'SOMAWF_OUTPUT_PARAMS': '/casa/home/.soma-workflow/transfered_files/econdami_1/25993'}

  standard output:
  ----------------



  standard error:
  ---------------

EXCEPTION: '_Undefined' object is not iterable
while syncing nipype parameter output_directory on Realign
Traceback (most recent call last):
  File "/casa/host/src/capsul/master/capsul/process/nipype_process.py", line 170, in sync_process_output_traits
    nipype_outputs = (process_instance.
  File "/casa/home/.local/lib/python3.6/site-packages/nipype-1.6.0-py3.6.egg/nipype/interfaces/spm/preprocess.py", line 502, in _list_outputs
    for imgf in self.inputs.in_files:
TypeError: '_Undefined' object is not iterable
EXCEPTION: '_Undefined' object is not iterable
while syncing nipype parameter use_mcr on Realign
Traceback (most recent call last):
  File "/casa/host/src/capsul/master/capsul/process/nipype_process.py", line 170, in sync_process_output_traits
    nipype_outputs = (process_instance.
  File "/casa/home/.local/lib/python3.6/site-packages/nipype-1.6.0-py3.6.egg/nipype/interfaces/spm/preprocess.py", line 502, in _list_outputs
    for imgf in self.inputs.in_files:
TypeError: '_Undefined' object is not iterable
EXCEPTION: '_Undefined' object is not iterable
while syncing nipype parameter paths on Realign
Traceback (most recent call last):
  File "/casa/host/src/capsul/master/capsul/process/nipype_process.py", line 170, in sync_process_output_traits
    nipype_outputs = (process_instance.
  File "/casa/home/.local/lib/python3.6/site-packages/nipype-1.6.0-py3.6.egg/nipype/interfaces/spm/preprocess.py", line 502, in _list_outputs
    for imgf in self.inputs.in_files:
TypeError: '_Undefined' object is not iterable
EXCEPTION: '_Undefined' object is not iterable
while syncing nipype parameter matlab_cmd on Realign
Traceback (most recent call last):
  File "/casa/host/src/capsul/master/capsul/process/nipype_process.py", line 170, in sync_process_output_traits
    nipype_outputs = (process_instance.
  File "/casa/home/.local/lib/python3.6/site-packages/nipype-1.6.0-py3.6.egg/nipype/interfaces/spm/preprocess.py", line 502, in _list_outputs
    for imgf in self.inputs.in_files:
TypeError: '_Undefined' object is not iterable
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/casa/host/src/capsul/master/capsul/process/process.py", line 731, in run_from_commandline
    result = ce.study_config.run(process, configuration_dict=configuration)
  File "/casa/host/src/capsul/master/capsul/study_config/study_config.py", line 428, in run
    configuration_dict=configuration_dict)
  File "/casa/host/src/capsul/master/capsul/study_config/run.py", line 158, in run_process
    returncode = process_instance._run_process()
  File "/casa/home/Git_projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/process_mia.py", line 499, in _run_process
    self.run_process_mia()
  File "/casa/home/Git_projects/mia_processes/python/mia_processes/bricks/preprocess/spm/spatial_preprocessing.py", line 1770, in run_process_mia
    return self.process.run(configuration_dict={})
  File "/casa/host/src/capsul/master/capsul/process/process.py", line 310, in run
    return self.__call__(**kwargs)
  File "/casa/host/src/capsul/master/capsul/process/process.py", line 302, in __call__
    returncode = self.get_study_config().run(self, **kwargs)
  File "/casa/host/src/capsul/master/capsul/study_config/study_config.py", line 428, in run
    configuration_dict=configuration_dict)
  File "/casa/host/src/capsul/master/capsul/study_config/run.py", line 76, in run_process
    os.makedirs(base)
  File "/usr/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/data/IRM/projects_mia/rel2_test/scripts'


---- full host env ----
 ...

• If we run the same procedure a second time (the only difference is that I switched directly from initialise pipeline to run pipeline without going to the data browser in between) the pipeline work ... we may get an erratic result depending on when the scripts/pyscript file is written (I haven't checked yet, it's just a guess) ????

- Pipeline initialising ...

Completion ...

. spatial_preprocessing_1_1.list_duplicate1 (mia_processes.bricks.tools.tools.List_Duplicate) MIA node ...

During the <mia_processes.bricks.tools.tools.List_Duplicate object at 0x7f64e08d5f68> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...


. spatial_preprocessing_1_1.realign1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Realign) MIA node ...

. spatial_preprocessing_1_1.newsegment1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.NewSegment) MIA node ...

. spatial_preprocessing_1_1.coregister1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Coregister) MIA node ...

. spatial_preprocessing_1_1.normalize12_1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...```

. spatial_preprocessing_1_1.normalize12_2 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.smooth1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Smooth) MIA node ...

. spatial_preprocessing_1_1.list_duplicate1 (mia_processes.bricks.tools.tools.List_Duplicate) MIA node ...

During the <mia_processes.bricks.tools.tools.List_Duplicate object at 0x7f64e08d5f68> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...


. spatial_preprocessing_1_1.realign1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Realign) MIA node ...

. spatial_preprocessing_1_1.newsegment1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.NewSegment) MIA node ...

. spatial_preprocessing_1_1.coregister1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Coregister) MIA node ...

. spatial_preprocessing_1_1.normalize12_1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.normalize12_2 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.smooth1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Smooth) MIA node ...

Completion done.

Path data/raw_data/alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/derived_data/meanalej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/derived_data/ralej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/raw_data/alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/derived_data/ralej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/derived_data/alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/derived_data/meanalej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/derived_data/rp_alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.txt already in database.
Path data/derived_data/malej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii already in database.
Path data/derived_data/c1alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii already in database.
Path data/derived_data/c2alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii already in database.
Path data/derived_data/c3alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii already in database.
Path data/derived_data/c4alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii already in database.
Path data/derived_data/c5alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii already in database.
Path data/derived_data/c6alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii already in database.
Path data/derived_data/y_alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii already in database.
Path data/derived_data/alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000_seg8.mat already in database.
Path data/derived_data/meanalej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/derived_data/ralej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/derived_data/walej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii already in database.
Path data/derived_data/wralej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
Path data/derived_data/swralej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii already in database.
init time: 3.105550765991211
- Pipeline running ...


. spatial_preprocessing_1_1.list_duplicate1 (mia_processes.bricks.tools.tools.List_Duplicate) MIA node ...

During the <mia_processes.bricks.tools.tools.List_Duplicate object at 0x7f64e08d5f68> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...


. spatial_preprocessing_1_1.realign1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Realign) MIA node ...

. spatial_preprocessing_1_1.newsegment1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.NewSegment) MIA node ...

. spatial_preprocessing_1_1.coregister1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Coregister) MIA node ...

. spatial_preprocessing_1_1.normalize12_1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.normalize12_2 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.smooth1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Smooth) MIA node ...

. spatial_preprocessing_1_1.list_duplicate1 (mia_processes.bricks.tools.tools.List_Duplicate) MIA node ...

During the <mia_processes.bricks.tools.tools.List_Duplicate object at 0x7f64e08d5f68> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...


. spatial_preprocessing_1_1.realign1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Realign) MIA node ...

. spatial_preprocessing_1_1.newsegment1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.NewSegment) MIA node ...

. spatial_preprocessing_1_1.coregister1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Coregister) MIA node ...

. spatial_preprocessing_1_1.normalize12_1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.normalize12_2 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.smooth1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Smooth) MIA node ...
soma-workflow starting in light mode
Workflow controller initialised
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
Could not remove our own lock file "/tmp/lock_file_populse_mia.lock" maybe permissions changed meanwhile?
error while importing outputs in ProcessIteration
outputs: {}
error while importing outputs in ProcessIteration
outputs: {'native_class_images': [[['/data/IRM/projects_mia/rel2_test/data/derived_data/c1rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-05-T13DSENSE-T1TFE-000424_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c2rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-05-T13DSENSE-T1TFE-000424_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c3rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-05-T13DSENSE-T1TFE-000424_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c4rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-05-T13DSENSE-T1TFE-000424_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c5rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-05-T13DSENSE-T1TFE-000424_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c6rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-05-T13DSENSE-T1TFE-000424_000.nii']], [['/data/IRM/projects_mia/rel2_test/data/derived_data/c1alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c2alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c3alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c4alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c5alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii'], ['/data/IRM/projects_mia/rel2_test/data/derived_data/c6alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii']]], 'normalized_anat': ['/data/IRM/projects_mia/rel2_test/data/derived_data/wrocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-05-T13DSENSE-T1TFE-000424_000.nii', '/data/IRM/projects_mia/rel2_test/data/derived_data/walej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii'], 'forward_deformation_field': ['/data/IRM/projects_mia/rel2_test/data/derived_data/y_rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-05-T13DSENSE-T1TFE-000424_000.nii', '/data/IRM/projects_mia/rel2_test/data/derived_data/y_alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii'], 'bias_corrected_images': ['/data/IRM/projects_mia/rel2_test/data/derived_data/mrocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-05-T13DSENSE-T1TFE-000424_000.nii', '/data/IRM/projects_mia/rel2_test/data/derived_data/malej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-00-T13DSENSE-T1TFE-000425_000.nii'], 'realignment_parameters': ['/data/IRM/projects_mia/rel2_test/data/derived_data/rp_rocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-01-BOLD_CVR_7_53sl_ModeratePNS-FEEPI-001212_000.txt', '/data/IRM/projects_mia/rel2_test/data/derived_data/rp_alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.txt'], 'smoothed_func': ['/data/IRM/projects_mia/rel2_test/data/derived_data/swrrocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-01-BOLD_CVR_7_53sl_ModeratePNS-FEEPI-001212_000.nii', '/data/IRM/projects_mia/rel2_test/data/derived_data/swralej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii'], 'bias_field_images': [<undefined>, <undefined>], 'coregistered_source': ['/data/IRM/projects_mia/rel2_test/data/derived_data/meanrocmi_120618test1-IRMFonct_+perfusion-2018-06-12060428-01-BOLD_CVR_7_53sl_ModeratePNS-FEEPI-001212_000.nii', '/data/IRM/projects_mia/rel2_test/data/derived_data/meanalej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii']}
postprocess pipeline: <capsul.pipeline.pipeline.Pipeline object at 0x7f64e0aa7990>
set exec status on: 4ef51d9c-0954-4168-a678-3bf0e49d16e5 2021-07-07 12:23:43
set exec status on: c1c722c0-8c97-4102-903c-8b6bb55d8078 2021-07-07 12:23:41
update file history for: data/raw_data/alej170316_test24042018-IRMFonct_+perfusion-2016-03-17083444-03-BOLD_CVR_7_53sl_ModeratePNSSENSE-FEEPI-001212_000.nii : ['144153b0-986c-4893-9c77-02045180176f', 'c1c722c0-8c97-4102-903c-8b6bb55d8078'] -> ['c1c722c0-8c97-4102-903c-8b6bb55d8078']
obsolete bricks: set()
really orphan: set()

-> Note: it seems then that the initialisation is again done at run time in the case of an iteration.

[servoz]

Ok I think the iteration produces a race condition for the creation of directories. I will propose ASAP a small PR in capsul to fix this issue.

[servoz]

I have just reproduced exactly the same minimum steps that produced a run fall in the previous post and this time the run worked perfectly... It seems that the problem occurs erratically and it could be a race condition.
I propose a small modification in capsul to fix a possible race condition.

[servoz]

We can therefore conclude that the iteration works in Direct Iteration mode (current documentation).
However, a few issues were found during this test (to add to #212 and #207 tickets if necessary and open new tickets if necessary):

• Only the last patient in the iteration is indexed in the database.
• After initalisation:
  • If we go to the data browser and return to the pipeline manager, the run button remains enabled. This behaviour is not an error, it is even a desired behaviour, but I specify it because it is not the case in all the tests. See the following tests to come)
  • If we change tab and come back on the tab of the pipeline previously initialised, the run is not available anymore. EDIT (10/07/2021): this issue is now fixed with 15d073e)
• After run:
  • In the Bricks tag (history of the pipeline), the Exec tag remains at "Not done" and the Exec Time tag has no value at the end of the run.
  • Initialisation is performed again at runtime (so it is performed twice), but indexing in the database is only performed at initialisation time (in the Bricks tag, Init time is the initailisation time, not the runtime).
  • An anomaly in the indexing of the database is observed: the Bricks tag contains n times THE SAME BRICK (n equals the number of iterated patients )
    

[servoz]

The Via Input_Filter brick (without use of the iteration table) mode:

• This mode is now working.
• As for the previous mode there are still some issues to solve (here again I note it to complete or open other tickets if needed at the end of these tests):
  • We observe the same indexing problem as for the Direct iteration mode (only the last patient of the list is indexed in the database) which is natural because the same causes produce the same effects. On the other hand, as for the Direct iteration mode, all the expected data can be indeed found in the derived_data directory of the project.

    • After initalisation:

      • If we go to the data browser and return to the pipeline manager, the run button remains enabled. This behaviour is not an error, it is even a desired behaviour, but I specify it because it is not the case in all the tests. See the following tests to come).

    • After run:

      • In the Bricks tag (history of the pipeline), the Exec tag remains at "Not done" and the Exec Time tag has no value at the end of the run.
      • Initialisation is performed again at runtime (so it is performed twice), but indexing in the database is only performed at initialisation time (in the Bricks tag, Init time is the initailisation time, not the runtime).
      • An anomaly in the indexing of the database is observed: the Bricks tag contains n times THE SAME BRICK (n equals the number of iterated patients )

[servoz]

The Via Input_Filter brick (with use of the iteration table) mode:

• This mode is now working.

• As for the previous modes there are still some issues to fix (here again I note it to complete or open other tickets if needed at the end of these tests):

  • Before initialisation:

    • If we change tabs (e.g. by going to the Data Browser or staying in the Pipeline Manager but opening a new editor or going to another editor and then returning to the one with the pipeline to iterate) it seems that the data filter in the iteration table is lost. For example, I define the iteration as described in the documentation (exclusion of one patient out of the 3 in the database), I go to the Data Browser then I come back in the Pipeline Manager to the editor with the iteration: in this case there is no more filtration on the patients. Same observation, staying in the piepline manager, if I change of editor and then go back. The remedy is to redefine the filter each time we return to a tab, but we would like to be able to keep the data when we change tabs (and this is prone to errors).

  • After initialisation:

    • We observe the same indexing problem as with the 2 other iteration modes (only the last patient of the list is indexed in the database). On the other hand, as for the 2 other iteration modes, all the expected data can be indeed found in the derived_data directory of the project, after a run step.
    • There are a few times (not every time and I have not been able to determine the exact reason, for the moment it seems eratic ... but I do not believe in magic things...) where exceptions are printed in the stdout:

     - Pipeline initialising ...

Completion ...

. Iteration_pipeline.func_files_filter (mia_processes.bricks.tools.tools.Input_Filter) MIA node ...

During the <mia_processes.bricks.tools.tools.Input_Filter object at 0x7ff4725931a8> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

Exception occurred in traits notification handler for object: <populse_mia.user_interface.pipeline_manager.process_mia.MIAProcessCompletionEngine object at 0x7ff4725a4f10>, trait: completion_progress, old value: 0.0, new value: 1.0
Traceback (most recent call last):
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 524, in _dispatch_change_event
    self.dispatch(handler, *args)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 619, in dispatch
    handler(*args)
  File "/casa/host/src/capsul/master/capsul/qt_gui/widgets/attributed_process_widget.py", line 430, in _completion_progress_changed
    if not hasattr(self, 'progressdialog'):
RuntimeError: wrapped C/C++ object of type AttributedProcessWidget has been deleted

. Iteration_pipeline.anat_file_filter (mia_processes.bricks.tools.tools.Input_Filter) MIA node ...

During the <mia_processes.bricks.tools.tools.Input_Filter object at 0x7ff4725f0e60> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

Exception occurred in traits notification handler for object: <populse_mia.user_interface.pipeline_manager.process_mia.MIAProcessCompletionEngine object at 0x7ff4725e0678>, trait: completion_progress, old value: 0.0, new value: 1.0
Traceback (most recent call last):
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 524, in _dispatch_change_event
    self.dispatch(handler, *args)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 619, in dispatch
    handler(*args)
  File "/casa/host/src/capsul/master/capsul/qt_gui/widgets/attributed_process_widget.py", line 430, in _completion_progress_changed
    if not hasattr(self, 'progressdialog'):
RuntimeError: wrapped C/C++ object of type AttributedProcessWidget has been deleted

. spatial_preprocessing_1_1.list_duplicate1 (mia_processes.bricks.tools.tools.List_Duplicate) MIA node ...

During the <mia_processes.bricks.tools.tools.List_Duplicate object at 0x7ff4c0f6f570> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...


. spatial_preprocessing_1_1.realign1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Realign) MIA node ...

. spatial_preprocessing_1_1.newsegment1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.NewSegment) MIA node ...

. spatial_preprocessing_1_1.coregister1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Coregister) MIA node ...

. spatial_preprocessing_1_1.normalize12_1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.normalize12_2 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Normalize12) MIA node ...

. spatial_preprocessing_1_1.smooth1 (mia_processes.bricks.preprocess.spm.spatial_preprocessing.Smooth) MIA node ...
Exception occurred in traits notification handler for object: <capsul.attributes.completion_engine_iteration.ProcessCompletionEngineIteration object at 0x7ff4984b1780>, trait: completion_progress, old value: 0.0, new value: 1.0
Traceback (most recent call last):
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 524, in _dispatch_change_event
    self.dispatch(handler, *args)
  File "/casa/home/.local/lib/python3.6/site-packages/traits/trait_notifiers.py", line 619, in dispatch
    handler(*args)
  File "/casa/host/src/capsul/master/capsul/qt_gui/widgets/attributed_process_widget.py", line 430, in _completion_progress_changed
    if not hasattr(self, 'progressdialog'):
RuntimeError: wrapped C/C++ object of type AttributedProcessWidget has been deleted
if we change tabs (e.g. by going to the data browser or staying in the pipeline manager but opening a new editor or going to another editor and then returning to the one with the pipeline to iterate) it seems that the filter data in the iteration table is lost.
Completion done.

• After run
  • After run step, the Run pipeline button remains available.
  • In the Bricks tag (history of the pipeline), the Exec tag remains at "Not done" and the Exec Time tag has no value at the end of the run.
  • Initialisation is performed again at runtime (so it is performed twice), but indexing in the database is only performed at initialisation time (in the Bricks tag, Init time is the initailisation time, not the runtime).
  • An anomaly in the indexing of the database is observed: the Bricks tag contains n times THE SAME BRICK (n equals the number of iterated patients )

[servoz]

OK, I think it's time to close this long ticket!
I've just completed the tickets already opened (#217) and opened new tickets (#218 #219 #220 #221) to summarize the few issues still to be fixed.