quay · gabriel-rh · Mar 2, 2021 · Mar 1, 2021
diff --git a/modules/proc_deploy_quay_poc_clair.adoc b/modules/proc_deploy_quay_poc_clair.adoc
@@ -11,7 +11,7 @@ Clair is an application for parsing image contents and reporting vulnerabilities
 
 == Deploying a separate database for Clair
 
-Clair requires a database and Postgres is recommended, especially for highly available configurations. You can share a common database between Quay and Clair, but in this example a separate, Clair-specific database is deployed.
+Clair requires a Postgres database.  You can share a common database between Quay and Clair if Quay is also using Postgres, but in this example a separate, Clair-specific database is deployed.
 
 In this proof-of-concept scenario, you will use a directory on the local file system to persist database data. 
 
@@ -37,7 +37,7 @@ $ sudo podman run -d --rm --name postgresql-clairv4 \
 * Ensure that the Postgres `uuid-ossp` module is installed, as it is required by Clair:
 +
 ....
-$ sudo podman exec -it postgresql-quay /bin/bash -c 'echo "CREATE EXTENSION IF NOT EXISTS \"uuid-ossp\"" | psql -d clair -U postgres'
+$ sudo podman exec -it postgresql-clairv4 /bin/bash -c 'echo "CREATE EXTENSION IF NOT EXISTS \"uuid-ossp\"" | psql -d clair -U postgres'
 ....
 
 

diff --git a/modules/proc_deploy_quay_poc_restart.adoc b/modules/proc_deploy_quay_poc_restart.adoc
@@ -13,9 +13,11 @@ By default, Podman generates a unit file for existing containers or pods. You ca
 * Create the systemd unit files from a running {productname} registry as follows:
 +
 ....
-$ sudo podman generate systemd --new --files --name postgresql-quay
 $ sudo podman generate systemd --new --files --name redis
+$ sudo podman generate systemd --new --files --name postgresql-quay
 $ sudo podman generate systemd --new --files --name quay
+$ sudo podman generate systemd --new --files --name postgresql-clairv4
+$ sudo podman generate systemd --new --files --name clairv4
 ....
 
 * Copy the unit files to `/usr/lib/systemd/system` for installing them as a root user:
@@ -24,6 +26,8 @@ $ sudo podman generate systemd --new --files --name quay
 $ sudo cp -Z container-redis.service /usr/lib/systemd/system
 $ sudo cp -Z container-postgresql-quay.service /usr/lib/systemd/system
 $ sudo cp -Z container-quay.service /usr/lib/systemd/system
+$ sudo cp -Z container-postgresql-clairv4.service /usr/lib/systemd/system
+$ sudo cp -Z container-clairv4.service /usr/lib/systemd/system
 ....
 
 
@@ -39,6 +43,8 @@ $ sudo systemctl daemon-reload
 $ sudo systemctl enable --now container-redis.service
 $ sudo systemctl enable --now container-postgresql-quay.service
 $ sudo systemctl enable --now container-quay.service
+$ sudo systemctl enable --now container-postgresql-clairv4.service
+$ sudo systemctl enable --now container-clairv4.service
 ....
 
 
@@ -50,6 +56,8 @@ $ sudo systemctl enable --now container-quay.service
 $ sudo systemctl status container-redis.service
 $ sudo systemctl status container-postgresql-quay.service
 $ sudo systemctl status container-quay.service
+$ sudo systemctl status container-postgresql-clairv4.service
+$ sudo systemctl status container-clairv4.service
 ....
 
 
@@ -59,6 +67,8 @@ $ sudo systemctl status container-quay.service
 $ sudo systemctl stop container-redis.service
 $ sudo systemctl stop container-postgresql-quay.service
 $ sudo systemctl stop container-quay.service
+$ sudo systemctl stop container-postgresql-clairv4.service
+$ sudo systemctl stop container-clairv4.service
 ....
 
 * To start the Quay component services:
@@ -67,6 +77,8 @@ $ sudo systemctl stop container-quay.service
 $ sudo systemctl start container-redis.service
 $ sudo systemctl start container-postgresql-quay.service
 $ sudo systemctl start container-quay.service
+$ sudo systemctl start container-postgresql-clairv4.service
+$ sudo systemctl start container-clairv4.service
 ....
 
 == Testing restart after reboot
@@ -75,11 +87,73 @@ Once you have the services configured and enabled, reboot the system.  When the
 
 ....
 $ sudo podman ps -a
-CONTAINER ID  IMAGE                                      COMMAND               CREATED         STATUS             PORTS                   NAMES
-215ea92f75be  registry.redhat.io/quay/quay-rhel8:v3.4.0  registry              36 seconds ago  Up 35 seconds ago  0.0.0.0:8080->8080/tcp  quay
-82fd8191ef50  docker.io/library/postgres:10.12           postgres              36 seconds ago  Up 35 seconds ago  0.0.0.0:5432->5432/tcp  postgresql-quay
-bca9a7ea090b  docker.io/library/redis:5.0.7              --requirepass str...  36 seconds ago  Up 36 seconds ago  0.0.0.0:6379->6379/tcp  redis
+CONTAINER ID  IMAGE                                       COMMAND         CREATED         STATUS             PORTS                   NAMES
+4e87c7889246  registry.redhat.io/rhel8/postgresql-10:1    run-postgresql  19 seconds ago  Up 18 seconds ago  0.0.0.0:5432->5432/tcp  postgresql-quay
+b8fbac1920d4  registry.redhat.io/rhel8/redis-5:1          run-redis       19 seconds ago  Up 18 seconds ago  0.0.0.0:6379->6379/tcp  redis
+d959d5bf7a24  registry.redhat.io/rhel8/postgresql-10:1    run-postgresql  18 seconds ago  Up 18 seconds ago  0.0.0.0:5433->5432/tcp  postgresql-clairv4
+e75ff8651dbd  registry.redhat.io/quay/clair-rhel8:v3.4.0                  18 seconds ago  Up 17 seconds ago  0.0.0.0:8081->8080/tcp  clairv4
+....
+
+In this instance, the Quay container itself has failed to start up. This is due to the fact that, when security scanning is enabled in Quay, it tries to connect to Clair on startup. However, Clair has not finished initializing and cannot accept connections and, as a result, Quay terminates immediately. To overcome this issue, you need to configure the Quay service to have a dependency on the Clair service, as shown in the following section.
+
+== Configuring Quay's dependency on Clair
+
+In the `systemd` service file for Quay, set up a dependency on the Clair service in the `[Unit]` section by setting `After=container-clairv4.service`. To give the Clair container time to initialize, add a delay in the `[Service]` section, for example `RestartSec=30`. Here is an example of the modified Quay file, after configuring the dependency on Clair:
+
+
+./usr/lib/systemd/system/container-quay.service
+....
+# container-quay.service
+# autogenerated by Podman 2.0.5
+# Tue Feb 16 17:02:26 GMT 2021
+
+[Unit]
+Description=Podman container-quay.service
+Documentation=man:podman-generate-systemd(1)
+Wants=network.target
+After=container-clairv4.service
+
+[Service]
+Environment=PODMAN_SYSTEMD_UNIT=%n
+Restart=on-failure
+RestartSec=30
+ExecStartPre=/bin/rm -f %t/container-quay.pid %t/container-quay.ctr-id
+ExecStart=/usr/bin/podman run --conmon-pidfile %t/container-quay.pid --cidfile %t/container-quay.ctr-id --cgroups=no-conmon -d --rm -p 8080:8080 --name=quay -v /home/user1/quay/config:/conf/stack:Z -v /home/user1/quay/storage:/datastorage:Z registry.redhat.io/quay/quay-rhel8:v3.4.0
+ExecStop=/usr/bin/podman stop --ignore --cidfile %t/container-quay.ctr-id -t 10
+ExecStopPost=/usr/bin/podman rm --ignore -f --cidfile %t/container-quay.ctr-id
+PIDFile=%t/container-quay.pid
+KillMode=none
+Type=forking
+
+[Install]
+WantedBy=multi-user.target default.target
+....
+
+
+
+Once you have updated the Quay service configuration, reboot the server and immediately run `podman ps`:
+
+....
+$ sudo podman ps -a
+CONTAINER ID  IMAGE                                       COMMAND         CREATED         STATUS             PORTS                   NAMES
+4e87c7889246  registry.redhat.io/rhel8/postgresql-10:1    run-postgresql  29 seconds ago  Up 28 seconds ago  0.0.0.0:5432->5432/tcp  postgresql-quay
+b8fbac1920d4  registry.redhat.io/rhel8/redis-5:1          run-redis       29 seconds ago  Up 28 seconds ago  0.0.0.0:6379->6379/tcp  redis
+d959d5bf7a24  registry.redhat.io/rhel8/postgresql-10:1    run-postgresql  28 seconds ago  Up 28 seconds ago  0.0.0.0:5433->5432/tcp  postgresql-clairv4
+e75ff8651dbd  registry.redhat.io/quay/clair-rhel8:v3.4.0                  28 seconds ago  Up 27 seconds ago  0.0.0.0:8081->8080/tcp  clairv4
+....
+
+Initially, the Quay container will not be available, but once the `RestartSec` delay has expired, it should start up:
+
+....
+$ sudo podman ps -a
+CONTAINER ID  IMAGE                                       COMMAND         CREATED         STATUS             PORTS                   NAMES
+4e87c7889246  registry.redhat.io/rhel8/postgresql-10:1    run-postgresql  35 seconds ago  Up 34 seconds ago  0.0.0.0:5432->5432/tcp  postgresql-quay
+ab9f0e6ad7c3  registry.redhat.io/quay/quay-rhel8:v3.4.0   registry        3 seconds ago   Up 2 seconds ago   0.0.0.0:8080->8080/tcp  quay
+b8fbac1920d4  registry.redhat.io/rhel8/redis-5:1          run-redis       35 seconds ago  Up 34 seconds ago  0.0.0.0:6379->6379/tcp  redis
+d959d5bf7a24  registry.redhat.io/rhel8/postgresql-10:1    run-postgresql  34 seconds ago  Up 34 seconds ago  0.0.0.0:5433->5432/tcp  postgresql-clairv4
+e75ff8651dbd  registry.redhat.io/quay/clair-rhel8:v3.4.0                  34 seconds ago  Up 33 seconds ago  0.0.0.0:8081->8080/tcp  clairv4
 ....
 
+The `CREATED` field for the quay container shows the 30 second difference in creation time, as configured in the service definition.
 
 Log in to the {productname} registry at `quay-server:8080` to check that everything has restarted correctly.