Better docker-ization (startup speed improvement, unused groupadd and useradd, awkward chown, and confusing settings)

[ItaloBorrelli]

There's a few things that I came across while running the docker-ization locally that aren't fixable directly through the docker-compose.

chown on ${CATALINA_HOME}/content/ makes bind mounted content difficult to deal with

In my system the datasets.xml is bind mounted. This aligns with the suggestion in the given docker-compose.yml to create the ./erddapContent folder:

./erddapContent:/usr/local/tomcat/content/erddap

However, with the chown done by entrypoint.sh this folder is ro for the host system, so to modify the file one has to enter the container or copy a change into the container. Instead one could just be more selective with the chown, specifying:

chown -R $USER_ID:$GROUP_ID ${CATALINA_HOME}/{conf,logs,temp,webapps,work}

Long container startup

Issue

It takes a long time for a docker container to get to starting up tomcat. It would be better to do chown and chmod for catalina files in the Dockerfile instead of entrypoint.sh. This does mean that the TOMCAT_USER_ID and TOMCAT_GROUP_ID cannot be modified. Leading to groupadd and useradd not being used, since user ubuntu:1000:1000 exists in the base Ubuntu image.

    USER_ID=${TOMCAT_USER_ID:-1000}
    GROUP_ID=${TOMCAT_GROUP_ID:-1000}

    # create group for GROUP_ID if one doesn't already exist
    if ! getent group $GROUP_ID &> /dev/null; then
      groupadd -r tomcat -g $GROUP_ID
    fi
    # create user for USER_ID if one doesn't already exist
    if ! getent passwd $USER_ID &> /dev/null; then
      useradd -u $USER_ID -g $GROUP_ID tomcat
    fi

Proposed cumulative solution

Dockerfile

Add the following:

# I suppose it's customary to create a `tomcat:tomcat` user and group, though this solution doesn't change much if one decides to just use the ubuntu user and group instead.

ENV TOMCAT_USER_ID=1001
ENV TOMCAT_GROUP_ID=1001

RUN groupadd -r tomcat -g ${TOMCAT_GROUP_ID} && \
    useradd -u ${TOMCAT_USER_ID} -g ${TOMCAT_GROUP_ID} tomcat && \
    usermod -d ${CATALINA_HOME} -s /sbin/nologin $(id -u -n $TOMCAT_USER_ID)

RUN chown -R $TOMCAT_USER_ID:$TOMCAT_GROUP_ID \
    ${CATALINA_HOME}/conf \
    ${CATALINA_HOME}/logs \
    ${CATALINA_HOME}/temp \
    ${CATALINA_HOME}/webapps \
    ${CATALINA_HOME}/work
RUN find ${CATALINA_HOME}/conf \
    -type d -exec chmod 755 {} \; -o -type f -exec chmod 400 {} \;

entrypoint.sh

Remove the following lines

    USER_ID=${TOMCAT_USER_ID:-1000}
    GROUP_ID=${TOMCAT_GROUP_ID:-1000}

    ###
    # Tomcat user
    ###
    # create group for GROUP_ID if one doesn't already exist
    if ! getent group $GROUP_ID &> /dev/null; then
      groupadd -r tomcat -g $GROUP_ID
    fi
    # create user for USER_ID if one doesn't already exist
    if ! getent passwd $USER_ID &> /dev/null; then
      useradd -u $USER_ID -g $GROUP_ID tomcat
    fi
    # alter USER_ID with nologin shell and CATALINA_HOME home directory
    usermod -d "${CATALINA_HOME}" -s /sbin/nologin $(id -u -n $USER_ID)

    ###
    # Change CATALINA_HOME ownership to tomcat user and tomcat group
    # Restrict permissions on conf
    ###

    chown -R $USER_ID:$GROUP_ID ${CATALINA_HOME} && find ${CATALINA_HOME}/conf \
        -type d -exec chmod 755 {} \; -o -type f -exec chmod 400 {} \;

and only do `chown -R $TOMCAT_USER_ID:$TOMCAT_GROUP_ID /erddapData

Documentation on setup values

The documentation currently points to the erddap gold standard page which is quite out of date. It references certain values as if they are modifiable, such as in this section the bigParentDirectory. It took me quite a while to realize that no matter what I set it to be in setup.xml it was being overridden by ERDDAP_bigParentDirectory. In fact, I'm not sure at all if any values from the setup.xml are still used or if only the values set by env with ERDDAP_ prefixes are used. I also don't know where and how they're accessed and used by the ERDDAP webapp, just that they are provided in the setenv.sh. I'm sure I could help document if I understood that part a little bit better.

Thanks for any suggestions and feel free to open a feature issue for me if you'd like me to submit my code changes in a PR.

[rmendels]

@ItaloBorrelli @srstsavage @ChrisJohnNOAA I just want to check that you are using the official docker, which I believe is at https://hub.docker.com/r/erddap/erddap. Look at the information there. I don't know enough about Docker or what has changed in the image to answer your questions, Chris or Shane will have to do that. But first I want to be certain you are using the latest Docker image.

[ItaloBorrelli]

I am! Thanks for the diligence in double checking! The above is pertinent to the official docker image.

[srstsavage]

Hi, first thanks for the investigation and feedback. There are certainly some improvements we can make.

Any opinions I offer below are my own and not necessarily representative of the project lead or others on the technical board.

The entrypoint.sh script is largely based on the Unidata tomcat-docker entrypoint.sh. The chown/chmod operations are done to make sure that any files necessary to run Tomcat/ERDDAP are readable (and when appropriate writeable and/or executable) by the tomcat user, which as you've noted is configurable at runtime.

This is supposed to make it easier for users, in that they can set the TOMCAT_USER_ID and TOMCAT_GROUP_ID to values appropriate for their user/group, edit the mounted files on the host (datasets.xml etc), and still have the tomcat user inside the container have appropriate access to everything it needs to run ERDDAP (including read access to the jar files, write access to logs and temp dirs, exec access to scripts, etc). If we remove the runtime TOMCAT_USER_ID/TOMCAT_GROUP_ID setting then we force /erddapData to be owned by whichever user/group ids we used at image build time, which are possibly inaccessible to the local user and prevent them from interacting with that dir (setting reload flags etc) on the host.

I definitely do think you're right that we don't need to be chowning all of the jar files, class files, etc under /usr/local/tomcat/webapps/erddap/WEB-INF which the tomcat user only needs read access to, and chowning those files takes the vast majority of the time spent of the current /usr/local/tomcat fixup. This slightly modified change from your suggestion seems to work for me (more testing needed obviously):

    chown -v -R $USER_ID:$GROUP_ID \
         ${CATALINA_HOME}/conf \
         ${CATALINA_HOME}/content \
         ${CATALINA_HOME}/logs \
         ${CATALINA_HOME}/temp \
         ${CATALINA_HOME}/work
    find ${CATALINA_HOME}/webapps/erddap -type d -exec chown $USER_ID:$GROUP_ID {} \;
    find ${CATALINA_HOME}/webapps/erddap -type f -name '*.sh' -exec chown $USER_ID:$GROUP_ID {} \;
    find ${CATALINA_HOME}/conf -type d -exec chmod 755 {} \; -o -type f -exec chmod 400 {} \;

That reduces the 3-4 seconds previously spent on chowning /usr/local/tomcat to basically instantaneous. Thoughts?

The chowing of /erddapData is a bit trickier, since we have to ensure that the tomcat user has read/write access to the entire directory hierarchy. If you're concerned about start up time there (and /erddapData could potentially be very large on a large established ERDDAP) maybe we could add an env var setting to skip that chown for users who are confident managing those permissions independently of container startup.

On large ERDDAPs the dataset loading time is going to vastly eclipse the chown/chmod time for these files, but you're right to look for optimizations wherever we can find them.

Regarding mounting your datasets.xml file, does setting TOMCAT_USER_ID to your user's uid on the host and/or TOMCAT_GROUP_ID to a host group id you belong to with a group writable datasets.xml solve your issue?

Regarding the reference to the IOOS gold standard repo, while we should update any outdated setup guidance on that repo, I am not at all a fan of the ERDDAP docs linking off to an external repository for Docker setup instructions, nor the external gold standard repo providing a setup guide which may be out of date with the ERDDAP docker image. That externalization of Docker setup instructions was established before ERDDAP had an official image so we should clean that up; the ERDDAP docs should have sufficient Docker setup instructions, and the IOOS gold standard repo should focus on data standards and ERDDAP dataset configuration specific to the IOOS community standards. Again, my opinion.

That said, as you've found the ERDDAP docs are more up to date/correct, and the section on environment variable configuration does mention that they override any values in setup.xml:

https://erddap.github.io/docs/server-admin/deploy-install#environment-variables

administrators can override any value in setup.xml by specifying an environment variable named ERDDAP_valueName

Again, appreciate your interest/suggestions/contributions, and you're absolutely right that the image can be optimized and docs improved.

[srstsavage]

@ItaloBorrelli I implemented this in #469, please take a look if you have time.