omero.server.nodedescriptors allows components to be enabled/disabled

[manics]

This is a modified copy of https://github.com/ome/omero-grid-docker/blob/fba17b12891a971e3c4bfc90ba59485607e28863/omero-grid/process_defaultxml.py that can be used to define Ice master and worker nodes with custom combinations of enabled services

Adds a new property omero.server.nodedescriptors. E.g. setting it to master:Blitz-0,Tables-0 will disable Indexer-0, DropBox ,MonitorServer, Processor-0

To run the new test:
OMERODIR=/path/to/OMERO.server pytest test/unit/clitest/test_admin.py -k testNodeDescriptors
Note several of the other test_admin.py tests have been broken for a while #61

[joshmoore]

Few misc. thoughts while reading the code.

[joshmoore]

Thinking about the current indexer use case, it's interesting to think about how you might just turn off a single server.

[manics]

This is quite advanced functionality, so I think we should try and keep it simple for now and require the full list of servers to be specified. In future we could add + - prefixes such as +Processor-1 and -Indexer-0 to indicate the defaults should be modified.

[joshmoore]

Would it simplify things to just delete default.xml?

[manics]

It would, seeing as the only changes to https://github.com/ome/openmicroscopy/commits/develop/etc/templates/grid/default.xml have been to enable configuration from the admin plugin

[joshmoore]

Comments all acknowledged. Let's save any reworkings for future follow-ups (e.g. extracting admin.py out and moving the template there) You ready to drop "Draft" from this PR?

@sbesson, I assume you're interested in seeing this in IDR ASAP?

[sbesson]

@manics for functional testing of this PR, is it possible to create an IDR deployment that disables Indexer but also Dropbox and PixelData?

[manics]

I haven't actually tested this yet, which is why it's still marked as draft. For the IDR I'd think something like

omero.server.nodedescriptors=master:Blitz-0,Tables-0

might be enough?

[sbesson]

omero.server.nodedescriptors=master:Blitz-0,Tables-0

are certainly the two main services used by IDR that I know of. I don't know whether anything else in the template will be required otherwise starting with this minimal list and adding services as necessary should be fine.

[manics]

I've tested this using https://github.com/ome/docker-example-omero with these changes:

$ git diff

diff --git a/docker-compose.yml b/docker-compose.yml
index 60fd75c..23fec7a 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -14,12 +14,13 @@ services:
       - "database:/var/lib/postgresql/data"

   omeroserver:
-    image: "openmicroscopy/omero-server:5.6"
+    build: ./server
     environment:
       CONFIG_omero_db_host: database
       CONFIG_omero_db_user: omero
       CONFIG_omero_db_pass: omero
       CONFIG_omero_db_name: omero
+      CONFIG_omero_server_nodedescriptors: "master:Blitz-0,Tables-0"
       ROOTPASS: omero
     networks:
       - omero
diff --git a/server/Dockerfile b/server/Dockerfile
new file mode 100644
index 0000000..31974ac
--- /dev/null
+++ b/server/Dockerfile
@@ -0,0 +1,5 @@
+FROM openmicroscopy/omero-server:5.6.3
+
+USER root
+RUN /opt/omero/server/venv3/bin/pip install --upgrade --upgrade-strategy only-if-needed https://github.com/manics/omero-py/archive/process_defaultxml.zip
+USER omero-server

$ docker-compose exec omeroserver /opt/omero/server/OMERO.server/bin/omero admin ice server list
Blitz-0
OMERO.Glacier2
Tables-0

$ docker-compose exec omeroserver ps auwwx
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
omero-s+     1  0.0  0.0    212     4 ?        Ss   16:30   0:00 /usr/local/bin/dumb-init /bin/bash /usr/local/bin/entrypoint.sh
omero-s+     6  0.0  0.0  11704  2604 ?        Ss   16:30   0:00 /bin/bash /usr/local/bin/entrypoint.sh
omero-s+   153  0.2  0.9 239944 81328 ?        S    16:31   0:01 /opt/omero/server/venv3/bin/python3 /opt/omero/server/venv3/bin/omero admin start --foreground
omero-s+   783  0.0  0.2 1641848 18408 ?       Sl   16:31   0:00 icegridnode --nochdir --Ice.Config=/opt/omero/server/OMERO.server/etc/internal.cfg,/opt/omero/server/OMERO.server/etc/master.cfg --deploy /opt/omero/server/OMERO.server/etc/grid/default.xml
omero-s+   810 15.3  8.8 5077584 718584 ?      Sl   16:31   1:18 java -Xmx1252m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions -Djava.awt.headless=true -Dlogback.configurationFile=etc/logback.xml -Domero.logfile=var/log/${omero.name}.log -Domero.name=Blitz-0 ome.services.blitz.Entry --Ice.Config=/opt/omero/server/OMERO.server-5.6.1-ice36-b225/var/master/servers/Blitz-0/config/config
omero-s+   811  0.1  0.1 263396 10760 ?        Sl   16:31   0:00 glacier2router --Ice.Config=/opt/omero/server/OMERO.server-5.6.1-ice36-b225/var/master/servers/OMERO.Glacier2/config/config
omero-s+   832  0.3  1.0 913508 87372 ?        Sl   16:31   0:02 python -m runTables --Ice.Config=/opt/omero/server/OMERO.server-5.6.1-ice36-b225/var/master/servers/Tables-0/config/config
omero-s+  1058  0.0  0.0  51768  3128 pts/0    Rs+  16:40   0:00 ps auwwx

[manics]

See ome/docker-example-omero-grid#11 for another example

[sbesson]

Briefly discussed today with @manics who suggested an omero-py pre-release with this PR included might be the simplest way to include it in an IDR deployment. No objection to this progression from my side, which version number should be used?

[sbesson]

Discussed this morning. Merging and tagging as prerelease for consumption in IDR deployment.