enable web and nginx slaves

[atarkowska]

This PR improves existing containers infrastructure

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                   NAMES
2036d9641125        devspace_web        "/usr/sbin/init"         2 minutes ago       Up 2 minutes        22/tcp, 0.0.0.0:4080->4080/tcp          devspace_web_1
5f94f815fe7b        devspace_slave      "/tmp/jenkins-slave.s"   2 minutes ago       Up 2 minutes                                                devspace_slave_1
a773bc06f903        devspace_nginx      "/usr/sbin/init"         2 minutes ago       Up 2 minutes        22/tcp, 443/tcp, 0.0.0.0:8888->80/tcp   devspace_nginx_1
41121d7114bd        devspace_jenkins    "/bin/tini -- /usr/lo"   2 minutes ago       Up 2 minutes        0.0.0.0:8080->8080/tcp, 50000/tcp       devspace_jenkins_1
89b0a8f675b1        postgres            "/docker-entrypoint.s"   2 minutes ago       Up 2 minutes        5432/tcp                                devspace_pg_1
0d64ddcb5ce7        redis               "/entrypoint.sh redis"   2 minutes ago       Up 2 minutes        0.0.0.0:6379->6379/tcp                  devspace_redis_1

• use swarm plugin 2.0
• use latest omero-install and fix Dockerfile
• make nginx a jenkins node
• allow systemd to start services on server, web and nginx and robot
• allow linking nginx config and statics
• add redis node to test ome/omero-install@0ab09c1#diff-1337d8f4ef8c1d8af22cc375046c102c

jenkins_1 | Mar 26, 2016 9:41:49 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
jenkins_1 | INFO: Accepted connection #1 from /172.17.0.7:58020
jenkins_1 | Mar 26, 2016 9:41:49 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
jenkins_1 | INFO: Accepted connection #2 from /172.17.0.6:38142
jenkins_1 | Mar 26, 2016 9:41:49 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
jenkins_1 | INFO: Accepted connection #3 from /172.17.0.5:42540

NOTE:

• allow custom STATIC_ROOT openmicroscopy#4544 is required

MORE TODO:

• jenkins ip in web job https://github.com/joshmoore/devspace/pull/1/files#diff-f9a547ad226d5c0d8302f247b9ff5c39R91
• redis host in web job https://github.com/joshmoore/devspace/pull/1/files#diff-f9a547ad226d5c0d8302f247b9ff5c39R89
• ports in overwrite file (e.g. docker-compose.osx.yml) https://github.com/joshmoore/devspace/pull/1/files#r57531628
• improve env vars https://github.com/joshmoore/devspace/pull/1/files#r57534213
• platform independent jobs https://github.com/joshmoore/devspace/pull/1/files#r57531658
• robot framework using selenium hub
• robot job
• robot jenkins plugin

[joshmoore]

@sbesson had a good idea: down the road, we should likely turn this into a build-arg, but I think that will require a newer version of docker-compose.

[atarkowska]

args are in V 2.0 https://docs.docker.com/compose/compose-file/#args

[joshmoore]

Few comments added, but generally 👍

Discussing with @aleksandra-tarkowska just now, the general next steps for this week look to be:

• focus on Redis omero-install#95 and this PR (making it dependent on Redis omero-install#95)
• possibly open a PR against https://github.com/openmicroscopy/devslave-c7-docker if needed (this can alternatively be checked out locally and the image: command changed to build:)
• deploy them on beluga to get around web and nginx slaves joshmoore/devspace#1 (comment)
• remove bin/omero web start from OMERO-start in favor of OMERO-web
• look into getting some integration tests running, optimally robotframework

[atarkowska]

@joshmoore unfortunately I had no success on deployment. All my new systemd containers do not start correctly via compose (although they do start via docker run). I think this is due to compose version 1.3 being too old. On Mac I use 1.6 and have no problem. As compose is installed in your homedir I wasn't sure how to upgrade

[atarkowska]

I followed your strategy with miniconda and installed compose 1.6.2

$ export PATH=$PATH:/lifesci/groups/jrs/atarkowska/miniconda/bin/
[atarkowska@ome-c6220-1-4 ~]$ docker-compose -v
docker-compose version 1.6.2, build 4d72027

looks like there is no difference, systemd is not starting up. I am not sure what is wrong

$ docker exec -it develop_nginx_1 bash
[root@ff3066d8b921 nginx]# ps -ef
UID         PID   PPID  C STIME TTY          TIME CMD
root          1      0  0 14:26 ?        00:00:00 /usr/sbin/init
root         23      0  3 14:27 ?        00:00:00 bash
root         49     23  0 14:27 ?        00:00:00 ps -ef

[manics]

Can you try a minimal docker-compose file that just starts up omero-ssh-systemd, and play around with the parameters?

[atarkowska]

@manics thank you for you suggestions looks like we were missing

--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -38,6 +38,8 @@ web:
         - jenkins
         - redis
     volumes:
+        - /sys/fs/cgroup:/sys/fs/cgroup:ro
+        - /run
         - ./web:/home/omero
         - ./nginx/conf.d:/home/omero/nginx
     volumes_from:
@@ -52,6 +54,8 @@ nginx:
     links:
         - jenkins
     volumes:
+        - /sys/fs/cgroup:/sys/fs/cgroup:ro
+        - /run
         - ./nginx/conf.d:/etc/nginx/conf.d
         - ./web/OMERO.server/lib/python/omeroweb/static:/home/omero/nginx/web/static

@joshmoore do we want centos7 compose file to override default ?

[joshmoore]

@joshmoore do we need slack bit in the jenkins jobs? At this moment, when you save manually a job this is removed. Shall I remove them ?

Ah, I wonder if that's happening because the plugin is not installed. We can clean all of that up after the fact. In general, if you don't know why a change shows up in the diff, probably best not to commit it.

[atarkowska]

I removed previous commits and will play with user id in another PR

[sbesson]

Tested deployment on an existing devspace. As the existing customization created large conflicts, I went for the following approach (after backing up the whole devspace directory):

• tag the current Git repository
• $ git reset --hard origin/master
• merge this PR and Cap the number of OMERO-build artifacts to keep to 5 #21
• fix the permissions for each Dockerfile
• run rename.py regions (after small modifications to prevent modifications of XML files not under VCS - PR to follow)
• mv home/jobs/OMERO-start/* home/jobs/OMERO-test-integration/ to account for the job renaming
• ./ds build NB: this took almost an hour from scratch due to all the dependencies installation via omero-install. The ongoing effort of moving base images to hub should largely lower this barrier
• ./ds up -d
• fix the web container permissions
• fix various container issues with ./ds stop && ./ds rm && ./ds up -d

Got Jenkins up and running with the expected set of jobs. OMERO-test-integration has retained its history as expected. Started a Trigger job to test the entire workflow.

At this point, a couple of documentations PRs should follow:

• describe the various permissions steps listed in enable web and nginx slaves  #20 (comment) in the README
• describe the new job workflow in README

[atarkowska]

fix the permissions for each Dockerfile

I think it should be possible to do it on live container, runing

docker exec -it [container_name] /bin/bash -c "usermod -u NEW_UID omero"

All files which are located in the user’s home directory will have the file UID changed automatically. Files outside user’s home directory need to be changed manually.

docker exec -it [container_name] /bin/bash -c "find / -user OLD_UID -exec chown -h omero {} \;"

and then ./ds stop && ./ds up -d

[atarkowska]

The above is only needed for new systemd containers, jenkins will pick it up after restart. @manics @joshmoore how do you think?

[sbesson]

Tested a full Trigger workflow. Everything worked as expected (barring failing robot tests which are also happening on the mainline devspace). Also hooked an OMERO-import job to the server Jenkins slave and checked the imported data via the Web client. Everything worked as expected. Given the size of this PR but also the improvements it brings for development teams, merging this epic PR. We can record and carry out improvements and bug fixes in follow up PRs. Thanks @aleksandra-tarkowska.

[atarkowska]

thanks everyone for helping and testing

[joshmoore]

Notes on my process for upgrading:

• merge (single conflict in docker-compose.yml)
• mv OMERO-start/*
• merge Cap the number of OMERO-build artifacts to keep to 5 #21 (handle slack conflict)
• merge Production deployment #22 (no issues)
• slight issue of needing a docker-compose.unix.yml + docker-compose.me.yml (handling locally for the moment; solution is likely code-gen)
• build went faster than expected (orca-3)
• currently pulling new images...