Exclude datasets containing images linked to wells

[sbesson]

This PR fixes https://trac.openmicroscopy.org.uk/ome/ticket/11848

For each found dataset, a query is submitted to find all contained images linked to wells and exclude this dataset. The number of excluded datasets is returned to the client output and the ID of the excluded datasets is stored in the script log.

$ bin/omero script launch 183 IDs=-2,2,51 Remove_From_Dataset=False
Using session 9a8aa5af-d148-46e8-9d07-bdc74f37e37b (root@localhost:4064). Idle timeout: 10.0 min. Current group: system
Job 142 ready
Waiting....
Callback received: FINISHED

    *** start stdout (id=186)***
    * {'Row_Names': 'letter', 'Remove_From_Dataset': False, 'Column_Names': 'number', 'Data_Type': 'Dataset', 'First_Axis_Count': 12L, 'IDs': [-2L, 2L, 51L], 'First_Axis': 'column'}
    * Dataset 2 contains images linked to wells.
    * Dataset 51 contains images linked to wells.
    * 
    *** end stdout ***


    *** out parameters ***
    * Message=Found 2 out of 3 dataset(s). Excluded 2 out of 2 dataset(s). 
    ***  done ***

To test this PR:

• try to run the Dataset To Plate multiple times on the same dataset with the Remove from Dataset option turned off
• try to run Dataset To Plate on a mixture of freshly import datasets and datasets already converted to plate using the Dataset To Plate script with the Remove from Dataset option turned off
• try to run Dataset To Plate on a dataset containing a mixture of freshly imported images and images linked to wells (e.g. run Dataset To Plate with the Remove from Dataset option turned off on a dataset and cut and paste a subset of its images to another dataset)

/cc @will-moore, @chris-allan

[manics]

Good to merge

[joshmoore]

You could return count(well) rather than wells to reduce overhead by using getQueryService().projection()

[bpindelski]

Test scenarios:

1. Initial run of the script produces a new plate. Consecutive runs return:

   *** start stdout (id=54)***
   * {'Row_Names': 'letter', 'Remove_From_Dataset': False, 'Column_Names': 'number', 'Data_Type': 'Dataset', 'First_Axis_Count': 12, 'IDs': [1L], 'First_Axis': 'column'}
   * Dataset 1 contains images linked to wells.
   *
   *** end stdout ***
   
   
   *** out parameters ***
   * Message=Excluded 1 out of 1 dataset(s).
   ***  done ***
   

2. Initial run outputs:

   {'Row_Names': 'letter', 'Remove_From_Dataset': False, 'Column_Names': 'number', 'Data_Type': 'Dataset',    'First_Axis_Count': 12, 'IDs': [1L, 2L], 'First_Axis': 'column'}
   Dataset 1 contains images linked to wells.
   Moving images from Dataset: 2 Bar to Plate: 2 Bar
   moving image: 10 CFPNEAT01_R3D.dv to row: 0, column: 0
   moving image: 9 IAGFP-Noc01_R3D.dv to row: 0, column: 1
   moving image: 16 lung-lentiCMtriple-4_R3D.dv to row: 0, column: 2
   moving image: 14 mt1_R3D_D3D.dv to row: 0, column: 3
   moving image: 13 off_by_1_a.dv to row: 0, column: 4
   moving image: 11 SMN10ul03_R3D_D3D.dv to row: 0, column: 5
   moving image: 15 test11_R3D.dv to row: 0, column: 6
   moving image: 12 very_small.d3d.dv to row: 0, column: 7
   

   Consecutive runs output:

   *** start stdout (id=83)***
   * {'Row_Names': 'letter', 'Remove_From_Dataset': False, 'Column_Names': 'number', 'Data_Type': 'Dataset', 'First_Axis_Count': 12, 'IDs': [1L, 2L], 'First_Axis': 'column'}
   * Dataset 1 contains images linked to wells.
   * Dataset 2 contains images linked to wells.
   *
   *** end stdout ***
   
   
   *** out parameters ***
   * Message=Excluded 2 out of 2 dataset(s).
   ***  done ***
   

3. Ran the script first to link images from a dataset to a plate. Then copied a subset of the dataset to a new dataset and imported a new image into the newly created dataset (ID=4). Ran the script:

   *** start stdout (id=118)***
   * {'Row_Names': 'letter', 'Remove_From_Dataset': False, 'Column_Names': 'number', 'Data_Type': 'Dataset', 'First_Axis_Count': 12, 'IDs': [4L], 'First_Axis': 'column'}
   * Dataset 4 contains images linked to wells.
   *
   *** end stdout ***
   
   
   *** out parameters ***
   * Message=Excluded 1 out of 1 dataset(s).
   ***  done ***
   

Good to merge.

[joshmoore]

Thanks, @bpindelski. @chris-allan : any thing needed for the worst-case fix? If not, merging tomorrow AM.

[chris-allan]

Just one thing to query which is the use of omero.sys.Parameters rather than omero.sys.ParametersI. omero.sys.ParametersI is both much safer and easier. Perpetuating the use of the bare omero.sys.Parameters is not something we should be doing.

[sbesson]

Good catch @chris-allan.

[sbesson]

Done

[chris-allan]

With omero.sys.ParametersI the map does not need to be initialised and methods can be used to populate all the relevant data using the relevant rtype instances. All you should need is:

...
params.addIds([i.getId() for i in dataset.listChildren()])
...

[chris-allan]

Looking good.

[sbesson]

--rebased-to #61