Delete and create bulk-map-annotations by namespace

[manics]

What this PR does

Adds infrastructure and tests for creating and deleting bulk-map-annotations with a namespace limit. At the moment I've added an extensible way of passing additional parameters, but I haven't added a dedicated CLI flag.

--depends-on #5068

Todo:

• Check if a73689b works as expected

Testing this PR

Run the integration tests

Run omero metadata populate ... --localcfg <cfg> where <cfg> is something that can be read by omero.utils.pydict_text_utils. For example, to limit by namespace any of the following should work:

--localcfg localcfg.yml or --localcfg OriginalFile:123:

ns: openmicroscopy/mapr/gene

--localcfg localcfg.json or --localcfg OriginalFile:123:

{
  "ns": "openmicroscopy/mapr/gene"
}

--localcfg '{"ns":"openmicroscopy/mapr/gene"}'

Related reading

• https://trello.com/c/e1OTY56t/96-annotate-delete-per-namespace

[atarkowska]

./components/tools/OmeroPy/test/integration/metadata/test_populate.py:58:1: F401 'skip' imported but unused

[atarkowska]

could you please fix travis? I will test it tomorrow then

[atarkowska]

INFO:omero.util.populate_metadata:Total: [{'Image'... 8385632L])}] MapAnnotationLink(s) in [u'openmicroscopy.org/mapr/gene']

could this say also size please?
maybe size in info and actual dict in --report?

[atarkowska]

This may never reach the logger because:

rsp <class 'omero.cmd.Delete2Response'> object #0 (::omero::cmd::Delete2Response)
{   
    deletedObjects =
    {   
        key = ome.model.annotations.WellAnnotationLink
        value =
        {   
            [0] = 8371438
        }
    }
}

[joshmoore]

https://github.com/openmicroscopy/openmicroscopy/blob/develop/components/blitz/resources/omero/cmd/Graphs.ice#L210

Delete2Response subclasses OK

[atarkowska]

Yes the problem is in

ndal = len(rsp.deletedObjects.get(
     "ome.model.annotations.AnnotationLink", []))

there is no ome.model.annotations.AnnotationLink

[atarkowska]

Tested with snoopycrimecop/idr-metadata@5978294

diff after upgrade

   id    |        name         |                    value                    
---------+---------------------+---------------------------------------------
 6777389 | Gene Identifier     | FBgn0026361
 6777389 | Gene Identifier URL | http://flybase.org/reports/FBgn0026361.html
 6777389 | Gene Symbol         | Sep-05
 6784588 | Gene Identifier     | FBgn0014029
 6784588 | Gene Identifier URL | http://flybase.org/reports/FBgn0014029.html
 6784588 | Gene Symbol         | Sep-02
 6787011 | Gene Identifier     | FBgn0259923
 6787011 | Gene Identifier URL | http://flybase.org/reports/FBgn0259923.html
 6787011 | Gene Symbol         | Sep-04
 6796010 | Gene Identifier     | FBgn0000427
 6796010 | Gene Identifier URL | http://flybase.org/reports/FBgn0000427.html
 6796010 | Gene Symbol         | Dec-01
 6805811 | Gene Identifier     | FBgn0011710
 6805811 | Gene Identifier URL | http://flybase.org/reports/FBgn0011710.html
 6805811 | Gene Symbol         | Sep-01
(15 rows)

to be deleted manually otherwise mapr will pick them up in metadata panel.

[mtbc]

FYI if it helps with testing the latest commits, in logback.xml if you bump omero.cmd.graphs up to DEBUG then Blitz-0.log will note everything that was deleted by Delete2.

[joshmoore]

Now run on demo33 (next production). The performance issue will be addressed in follow-ups.

[atarkowska]

--rebased-to #5246