Container fails to start subsequent to first (initial) start

[bwulff]

I tried starting the image without ldiff or other seeding. On the first run the image initializes and the containers keeps running. However, when I start the deployment again, the openldap container keeps exiting. Error message below.

docker-compose.yml:

version: '3'

networks:

  infrastructure:
    driver: 'bridge'

services:

  mongo:
    image: mongo
    container_name: infra_mongo
    networks:
      - infrastructure
    volumes:
      - ${SYSTEM_STORAGE_PATH}/mongo:/data/db

  redis:
    image: redis
    container_name: infra_redis
    networks:
      - infrastructure
    volumes:
      - ${SYSTEM_STORAGE_PATH}/redis:/data

  ldap:
    image: osixia/openldap
    container_name: infra_ldap
    networks:
      - infrastructure
    environment:
      LDAP_ADMIN_PASSWORD: "somepassword"
      LDAP_BASE_DN: "dc=institute,dc=cluster"
      LDAP_ORGANISATION: "Institute"
      LDAP_DOMAIN: "institute.cluster"
      LDAP_TLS_ENFORCE: "false"
    # ports:
    #   - 389:389
    volumes:
      - ${SYSTEM_STORAGE_PATH}/ldap/config:/etc/ldap/slapd.d
      - ${SYSTEM_STORAGE_PATH}/ldap/data:/var/lib/ldaph

Log on first start (clean directory):

*** CONTAINER_LOG_LEVEL = 3 (info)
*** Search service in CONTAINER_SERVICE_DIR = /container/service :
*** link /container/service/:ssl-tools/startup.sh to /container/run/startup/:ssl-tools
*** link /container/service/slapd/startup.sh to /container/run/startup/slapd
*** link /container/service/slapd/process.sh to /container/run/process/slapd/run
*** Set environment for startup files
*** Environment files will be proccessed in this order : 
Caution: previously defined variables will not be overriden.
/container/environment/99-default/default.startup.yaml
/container/environment/99-default/default.yaml

To see how this files are processed and environment variables values,
run this container with '--loglevel debug'
*** Running /container/run/startup/:ssl-tools...
*** Running /container/run/startup/slapd...
openldap user and group adjustments
get current openldap uid/gid info inside container
-------------------------------------
openldap GID/UID
-------------------------------------
User uid: 911
User gid: 911
uid/gid changed: false
-------------------------------------
updating file uid/gid ownership
Database and config directory are empty...
Init new ldap server...
  Backing up /etc/ldap/slapd.d in /var/backups/slapd-2.4.50+dfsg-1~bpo10+1... done.
  Creating initial configuration... done.
  Creating LDAP directory... done.
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of restart.
Start OpenLDAP...
Waiting for OpenLDAP to start...
Add bootstrap schemas...
config file testing succeeded
Add image bootstrap ldif...
Add custom bootstrap ldif...
Add TLS config...
No certificate file and certificate key provided, generate:
/container/service/slapd/assets/certs/ldap.crt and /container/service/slapd/assets/certs/ldap.key
2020/09/15 08:55:03 [INFO] generate received request
2020/09/15 08:55:03 [INFO] received CSR
2020/09/15 08:55:03 [INFO] generating key: ecdsa-384
2020/09/15 08:55:03 [INFO] encoded CSR
2020/09/15 08:55:03 [INFO] signed certificate with serial number 462058273364622229330952703208206866340968998577
Link /container/service/:ssl-tools/assets/default-ca/default-ca.pem to /container/service/slapd/assets/certs/ca.crt
Disable replication config...
Stop OpenLDAP...
Configure ldap client TLS configuration...
Remove config files...
First start is done...
*** Set environment for container process
*** Remove file /container/environment/99-default/default.startup.yaml
*** Environment files will be proccessed in this order : 
Caution: previously defined variables will not be overriden.
/container/environment/99-default/default.yaml

To see how this files are processed and environment variables values,
run this container with '--loglevel debug'
*** Running /container/run/process/slapd/run...
5f608167 @(#) $OpenLDAP: slapd 2.4.50+dfsg-1~bpo10+1 (May  4 2020 05:25:06) $
	Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>
5f608167 slapd starting

Log on second start (there indeed is something in the config dir but not in the data dir):

*** CONTAINER_LOG_LEVEL = 3 (info)
*** Search service in CONTAINER_SERVICE_DIR = /container/service :
*** link /container/service/:ssl-tools/startup.sh to /container/run/startup/:ssl-tools
*** link /container/service/slapd/startup.sh to /container/run/startup/slapd
*** link /container/service/slapd/process.sh to /container/run/process/slapd/run
*** Set environment for startup files
*** Environment files will be proccessed in this order : 
Caution: previously defined variables will not be overriden.
/container/environment/99-default/default.startup.yaml
/container/environment/99-default/default.yaml

To see how this files are processed and environment variables values,
run this container with '--loglevel debug'
*** Running /container/run/startup/:ssl-tools...
*** Running /container/run/startup/slapd...
openldap user and group adjustments
get current openldap uid/gid info inside container
-------------------------------------
openldap GID/UID
-------------------------------------
User uid: 911
User gid: 911
uid/gid changed: false
-------------------------------------
updating file uid/gid ownership
Error: the database directory (/var/lib/ldap) is empty but not the config directory (/etc/ldap/slapd.d)
*** /container/run/startup/slapd failed with status 1

*** Killing all processes...

I also noticed that config and data dir had been created by the docker daemon. The config dir was chowned to 911:911 but the data dir was still root:root ..

[bwulff]

Could it be the case that the database is kept in-memory and that we need to commit it to disc at least once before stopping/restarting the container?

[srcnix]

• 1

[zejdad]

Thank you very much for paying attention to this issue!

Any progress or workaround suggestions, please?

It is also most likely the underlying problem for this issue:
Alfresco/alfresco-docker-installer#45

With regards
David

[reinierjh]

I hope the docker-compose you shared is not a copy of the one you are using.

• ${SYSTEM_STORAGE_PATH}/ldap/data:/var/lib/ldaph
  seems to have an extra H in the path. And could explain why (after a restart) the ldap folder is not persistant.

[zejdad]

Hi, my openldap section of docker-compose is:

    openldap:
        image: osixia/openldap:1.4.0
        mem_limit: 128m
        container_name: openldap
        environment:
          LDAP_DOMAIN: "alfresco.org"
          LDAP_BASE_DN: "dc=alfresco,dc=org"
          LDAP_ADMIN_PASSWORD: "admin"
        volumes:
            - ./data/ldap:/var/lib/ldap
            - ./data/slap.d:/etc/ldap/slap.d

but still experiencing the problem described here: Alfresco/alfresco-docker-installer#45

David

[bwulff]

@reinierjh Well, that typo you spotted, was indeed the problem. Removing the 'h' in the second path made the container chown the folder and there are also now files in it, start-ups subsequent to initialization show no errors.

Thanks a lot!

Btw.: Given recent fails committed even by entities thought of to be professional, your suspicion regarding the compose file is sensible. This is nothing we are using in our deployment. I just tried to post an example as complete as possible (not just posting a snipped).

[bwulff]

@zejdad Now that I removed the typo it works for me. I tried version e55926b7c377 of the osixia/openldap image.