configuration options for omero docker

[pr4deepr]

Hi
Thanks for making it possible to use OMERO via docker. It has simplified the process a lot.
I can get the omero server running on a virtual machine using:

docker-compose up -d
docker-compose logs -f

I can also login to OMERO.web and access the webclient.

I was running out of storage as the server has limited space. I have attached an external volume and mounted it. I would like to change the OMERO data directory to this external volume. This is under the assumption that all the image data will be stored in OMERO data directory.
If I've used the above 2 commands to setup the server, how do I access the configuration options within docker to change the data directory? Also, do I have to stop it, set it up and restart it? I only have a basic understanding of docker.

Thanks a lot.

Pradeep

[joshmoore]

Hi @pr4deepr

how do I access the configuration options within docker to change the data directory?

Probably the easiest way is to change what type of mount you are using in
https://github.com/ome/docker-example-omero/blob/master/docker-compose.yml#L30

Rather than using a volume within the docker-compose.yml (here called "omero") you can mount an external directory:

    volumes:
      - "/my/big/directory:/OMERO"

Also, do I have to stop it, set it up and restart it?

Definitely. docker-compose up -d should detect the change though and only restart the omeroserver container leaving the web and postgres running.

[pr4deepr]

Hi Josh
Thanks heaps for this.

So i did what you said:

volumes:
- "/pvol:/OMERO"

ran the command: docker-compose up -d

It recreated only the omeroserver container, BUT, when I tried to login via the omerowebclient it showed the error:
Server not responding.

I changed the line 30 in the yml back to: - "omero:/OMERO", ran the docker compose command and its working now.

When i run lsblk to list all the blocks, it gives me:

NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda    252:0    0  30G  0 disk
└─vda1 252:1    0  30G  0 part /
vdb    252:16   0   1T  0 disk /pvol

What am I doing wrong?

Cheers
Pradeep

[joshmoore]

Does the container have permissions to write to /pvol?

[pr4deepr]

Good point..

Sorry, but how do I check permissions of the container?? How do I set it?

For the directory permissions:

root@omero-server:/home/ubuntu# ls -ld /pvol
drwxrwxr-x 5 root root 4096 Sep  5 23:24 /pvol

---

I gave it another shot to see if I missed anything. I changed the yml file to include /pvol again and got this warning (?):

WARNING: Service "omeroserver" is using volume "/OMERO" from the previous container. Host mapping "/pvol" has no effect. Remove Recreating docker-example-omero_omeroserver_1 ... done

---

Also, if it has changed successfully, will it show up as remaining free space when uploading images using omero.insight client?

[pr4deepr]

Hi
Actually, I figured it out.

I used
docker-compose stop && docker-compose rm -f
to stop containers and remove them

I decided to use a subdirectory /pvol/OMERO for the omero data directory to make it easier rather than the whole external drive.
When checking permissions of the directory /pvol/OMERO

ls -l
dr-xr-xr-x+ 8 ubuntu root  4096 Sep  7 01:09 OMERO

it didnt have write permissions, so I used the following to change permissions to read and write for the user (u) and the group (g)

chmod ug=rwx /pvol/OMERO
ls -l
drwxrwxr-x+ 8 ubuntu root  4096 Sep  7 01:09 OMERO

This was all done as user: ubuntu not as root.

Changed
https://github.com/ome/docker-example-omero/blob/master/docker-compose.yml#L30
to

volumes:
- "/pvol/OMERO:/OMERO"

Restarted docker containers.

Now, I can see the ~1TB free space within omero insight client now

---

BTW, I accessed the omero server container to get a bash prompt:

>docker exec -it OMERO SERVER DOCKER NAME HERE /bin/bash
#got current running docker container names for aboveusing docker ps

#checked user id
>id -u
1000
>whoami
omero-server

The user ubuntu also had userid 1000, hence the reason to set write permissions for/pvol/OMERO using ubuntu.
Is the uid for omero-server in the docker always 1000?

Another question? How do I go about maintaining the server and installing any plugins if needed??

Cheers
Pradeep

[joshmoore]

Hi Pradeep,

Is the uid for omero-server in the docker always 1000?

Yes. That's effectively part of the public API.

Another question? How do I go about maintaining the server and installing any plugins if needed??

You'll want to change the omeroserver container from image: to build: and maintain your own Dockerfile with customizations. You can see an example in https://github.com/ome/docker-example-omero-websockets/blob/a9d5de3f621e17dd747d5ce5b6c1e0f64e4ca55b/server/Dockerfile

~Josh

[pr4deepr]

Thanks for this example.
So, I have been able to run the default docker build quite successfully. Thanks for making this convenient.
One question, when I used OMERO.figure, the export option seems to be grayed out or inactive, regardless of which export format I choose (pdf, tiff etc..). Let me know if I should open another issue instead:

[joshmoore]

@will-moore may want to comment, but did you also update the server image to include the steps under https://github.com/ome/omero-figure#enabling-figure-export (Figure is special in that some steps are for omero-web and some are for omero-server)

[pr4deepr]

Thanks
So, I followed the instructions and copied Figure_To_Pdf.py to figure_scripts directory in the active omero server container.
I installed reportlab and markdown via pip3 in the docker container.pip didn't work..

Now, the export option is active in Omero.Web, but when I export it, it gives the error that I need to install reportlab and markdown.

What am I doing wrong? Should I be installing them somewhere else?
When I run the command python in the omeroserver container I get a python version of 2.7.5

Thanks

[pr4deepr]

Hi again
I think I figured it out. BTW, please let me know if there is a quicker or better way..

I started an interactive bash prompt as root in the omeroserver container (container name is omeroserver):
docker exec -it -u 0 omeroserver /bin/bash

As mentioned in this link: https://github.com/ome/omero-figure#enabling-figure-export
I followed Option 2

At the bash prompt within the omeroserver container:

cd /opt/omero/server/OMERO.server/lib/scripts/omero/figure_scripts

#download Figure_To_Pdf script
wget https://raw.githubusercontent.com/ome/omero-figure/master/omero_figure/scripts/omero/figure_scripts/Figure_To_Pdf.py

#Install into the virtual environment where OMERO server is installed
VENV_SERVER=/opt/omero/server/venv3
python3 -mvenv $VENV_SERVER 

#install reportlab and markdown
$VENV_SERVER/bin/pip install reportlab
$VENV_SERVER/bin/pip install markdown

#Restart the container:
docker restart omeroserver

I can now export pdfs and tiffs, which is great.

I understand if I delete the containers and create them again, or if I want to create containers elsewhere, the export option will be gone. If I would like to automatically enable fIgure export in the future, do I create a custom dockerfile with the above commands?

Thanks heaps.

Pradeep

[sbesson]

@pr4deepr you are right, the installation steps will not be lost recreate the container. With the current infrastructure, your best option is probably to build your own image consuming the upstream openmmicroscopy:omero-server image and installing your additional requirements as per your script above. I would expect the following change:

diff --git a/docker-compose.yml b/docker-compose.yml
index 60fd75c..5c1dd02 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -14,7 +14,7 @@ services:
       - "database:/var/lib/postgresql/data"
 
   omeroserver:
-    image: "openmicroscopy/omero-server:5.6"
+    build: omero-server
     environment:
       CONFIG_omero_db_host: database
       CONFIG_omero_db_user: omero

with an OMERO.server image defined as follows:

[sbesson@idr2-slot3 ~]$ cat omero-server/Dockerfile 
FROM openmicroscopy/omero-server:5.6

USER root
RUN wget https://raw.githubusercontent.com/ome/omero-figure/master/omero_figure/scripts/omero/figure_scripts/Figure_To_Pdf.py -O /opt/omero/server/OMERO.server/lib/scripts/omero/figure_scripts/Figure_To_Pdf.py
RUN /opt/omero/server/venv3/bin/pip install reportlab markdown

USER omero-server

should start an OMERO.server with the server-side figure requirements when calling docker-compose up -d.

[pr4deepr]

Hi @sbesson
Perfect, that worked really well.

Just to summarise so I understand this well
Edit the docker-compose file as you mentioned above.
Create a folder in the same directory: omero-server, and within this folder create a file named Dockerfile which is the OMERO.server image. This gets the image from docker and runs the commands to copy scripts and install packages.

One question, from the output below, is it creating a container at each instance??? for ex, when it says intermediate container??

Output after running docker-compose up -d:

Building omeroserver
Step 1/5 : FROM openmicroscopy/omero-server:5.6
 ---> b725ac2a9d9f
Step 2/5 : USER root
 ---> Running in b158cdfdd1ed
Removing intermediate container b158cdfdd1ed
 ---> 4c03c08ffa44
Step 3/5 : RUN wget https://raw.githubusercontent.com/ome/omero-figure/master/omero_figure/scripts/omero/figure_scripts/Figure_To_Pdf.py -O /opt/omero/server/OMERO.server/lib/scripts/omero/figure_scripts/Figure_To_Pdf.py
 ---> Running in 3aab18ccfc8f
--2020-09-22 02:13:53--  https://raw.githubusercontent.com/ome/omero-figure/master/omero_figure/scripts/omero/figure_scripts/Figure_To_Pdf.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 85536 (84K) [text/plain]
Saving to: '/opt/omero/server/OMERO.server/lib/scripts/omero/figure_scripts/Figure_To_Pdf.py'

     0K .......... .......... .......... .......... .......... 59% 4.45M 0s
    50K .......... .......... .......... ...                  100% 15.8M=0.01s

2020-09-22 02:13:54 (6.25 MB/s) - '/opt/omero/server/OMERO.server/lib/scripts/omero/figure_scripts/Figure_To_Pdf.py' saved [85536/85536]

Removing intermediate container 3aab18ccfc8f
 ---> e281d36efbc3
Step 4/5 : RUN /opt/omero/server/venv3/bin/pip install reportlab markdown
 ---> Running in 34af795527d7
Collecting reportlab
  Downloading https://files.pythonhosted.org/packages/e0/73/bb0e94871ab9314f7728be6553a7d5f8cd8590dbd499d2003bbba977abaa/reportlab-3.5.50-cp36-cp36m-manylinux1_x86_64.whl (2.6MB)
Collecting markdown
  Downloading https://files.pythonhosted.org/packages/a4/63/eaec2bd025ab48c754b55e8819af0f6a69e2b1e187611dd40cbbe101ee7f/Markdown-3.2.2-py3-none-any.whl (88kB)
Requirement already satisfied: pillow>=4.0.0 in /opt/omero/server/venv3/lib/python3.6/site-packages (from reportlab)
Collecting importlib-metadata; python_version < "3.8" (from markdown)
  Downloading https://files.pythonhosted.org/packages/8e/58/cdea07eb51fc2b906db0968a94700866fc46249bdc75cac23f9d13168929/importlib_metadata-1.7.0-py2.py3-none-any.whl
Collecting zipp>=0.5 (from importlib-metadata; python_version < "3.8"->markdown)
  Downloading https://files.pythonhosted.org/packages/b2/34/bfcb43cc0ba81f527bc4f40ef41ba2ff4080e047acb0586b56b3d017ace4/zipp-3.1.0-py3-none-any.whl
Installing collected packages: reportlab, zipp, importlib-metadata, markdown
Successfully installed importlib-metadata-1.7.0 markdown-3.2.2 reportlab-3.5.50 zipp-3.1.0
You are using pip version 9.0.3, however version 20.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Removing intermediate container 34af795527d7
 ---> e4fe97a8f997
Step 5/5 : USER omero-server
 ---> Running in c198f17024d3
Removing intermediate container c198f17024d3
 ---> be47ee09e19b
Successfully built be47ee09e19b
Successfully tagged docker-example-omero_omeroserver:latest
WARNING: Image for service omeroserver was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating docker-example-omero_omeroweb_1    ... done                                          
Creating docker-example-omero_omeroserver_1 ... done                                          
Creating docker-example-omero_database_1    ... done  

Thanks
Pradeep

[sbesson]

@pr4deepr Docker images are composed of a succession of read-only layers, each of them corresponding to the instruction in the Dockerfile. Unless you force a rebuild, the Docker image should not be rebuilt on every invocation unless the instructions in the Dockerfile have changed. Docker containers i.e. the instances of a Docker image will be deleted/created if you do docker-compose down && docker-compose up -d

[pr4deepr]

@sbesson and @joshmoore
Thanks for being patient with me and answering my queries. I've learned a fair bit!
I've got OMERO server and web running. The accessibility of the images via a web browser coupled with all the plugins make it extremely convenient to view, share and publish microscopy images.
I am closing this for now, and will definitely get in touch if I have any another issues.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/how-to-install-omero-web-add-on-apps-with-docker/47434/21