Unable to mount pre-existing database to /var/lib/mysql datadir

[taylorturner]

Hello,

We create a nightly backup of our prod datadir, mask the PII, and then store it on S3. We're trying to mount this datadir to the MySQL container and it's failing without much output.

For our local development environments (MacOS), we're attempting to:

• Pull the datadir, untar it to .docker/db/data/mysql file is owned by $USER:staff on the host
• In the docker-compose.yml, we're telling it to mount this dir as a bind mount .docker/db/data/mysql:/var/lib/mysql

docker-compose.yml

  db:
    build:
      context: .docker/db
    environment:
      - MYSQL_DATABASE=<redacted>
      - MYSQL_PASSWORD=password
      - MYSQL_ROOT_PASSWORD=${MYSQL_PASSWORD}
      - MYSQL_USER=user
    image: db:local
    volumes:
      - type: bind
        source: .docker/db/data/mysql
        target: /var/lib/mysql

• When we run docker-compose up db, we get the following output:

> docker-compose up db
Creating db_1 ... done
Attaching to db_1
db_1       | 2021-11-17 20:37:43+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.36-1debian10 started.
db_1       | 2021-11-17 20:37:44+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
db_1       | 2021-11-17 20:37:44+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.36-1debian10 started.
db_1 exited with code 2

• It seems like it may be a permissions issue, on MacOS there is some weirdness with the translated UID/GID. I checked the ID/GID of the /var/lib/mysql dir on a stock container and it's owned by mysql:mysql which is UID/GID 999:999. When trying to set this on the host machine (sudo chown -R 999:999 .docker/db/data/mysql), it still ends up being owned by root:root inside the container.
• To test this, I had it mount the .docker/db/data/mysql to a different location, /var/lib/default.

    volumes:
      - type: bind
        source: .docker/db/data/mysql
        target: /var/lib/default

root@680b36f22bc7:/var/lib# ls -l
total 6189992
drwxr-xr-x  1 root  root        4096 Oct 18 21:36 apt
-rw-r--r--  1 root  root  6338513555 Nov 12 13:46 datadir.tar.gz
drwx------ 22 root  root         704 Nov 12 13:18 default
drwxr-xr-x  1 root  root        4096 Oct 18 21:36 dpkg
drwxr-xr-x  2 root  root        4096 Oct  3 09:00 misc
drwxrwxrwt  6 mysql mysql       4096 Nov 17 21:05 mysql
drwxrwx---  2 mysql mysql       4096 Oct 18 21:36 mysql-files
drwxr-x---  2 mysql mysql       4096 Oct 18 21:36 mysql-keyring
drwxr-xr-x  2 root  root        4096 Oct 11 00:00 pam
drwxr-xr-x  1 root  root        4096 Oct 11 00:00 systems

root@7c365d37a414:/# id mysql
uid=999(mysql) gid=999(mysql) groups=999(MySQL)

• If I try to chmod -R 777 .docker/db/data/mysql on the host, those do end up persisting into the container.

root@80ceebe4d1c0:/# ls -l /var/lib
total 6189992
drwxr-xr-x  1 root  root        4096 Oct 18 21:36 apt
-rw-r--r--  1 root  root  6338513555 Nov 12 13:46 datadir.tar.gz
drwxrwxrwx 22 root  root         704 Nov 12 13:18 default
drwxr-xr-x  1 root  root        4096 Oct 18 21:36 dpkg
drwxr-xr-x  2 root  root        4096 Oct  3 09:00 misc
drwxrwxrwt  6 mysql mysql       4096 Nov 17 21:11 mysql
drwxrwx---  2 mysql mysql       4096 Oct 18 21:36 mysql-files
drwxr-x---  2 mysql mysql       4096 Oct 18 21:36 mysql-keyring
drwxr-xr-x  2 root  root        4096 Oct 11 00:00 pam
drwxr-xr-x  1 root  root        4096 Oct 11 00:00 systems

• However, even with 777 permissions it still fails to come up when mounted to /var/lib/mysql.

> docker-compose up db
Creating db_1 ... done
Attaching to db_1
db_1       | 2021-11-17 20:37:43+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.36-1debian10 started.
db_1       | 2021-11-17 20:37:44+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
db_1       | 2021-11-17 20:37:44+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.36-1debian10 started.
db_1 exited with code 2

• We've tried mounting it to /var/lib/default and passing command: --datadir=/var/lib/default but the behavior is exactly the same.
• For completeness, the contents of .docker/db/data/mysql are as follows:

> ls -l .docker/db/data/mysql
total 3010752
-rwxrwxrwx@   1 taylorturner  staff          56 Sep 26  2019 auto.cnf
-rwxrwxrwx@   1 taylorturner  staff        1680 Nov 20  2019 ca-key.pem
-rwxrwxrwx@   1 taylorturner  staff        1112 Nov 20  2019 ca.pem
-rwxrwxrwx@   1 taylorturner  staff        1112 Nov 20  2019 client-cert.pem
-rwxrwxrwx@   1 taylorturner  staff        1680 Nov 20  2019 client-key.pem
-rwxrwxrwx@   1 taylorturner  staff           0 Feb  2  2021 debian-5.7.flag
-rwxrwxrwx@   1 taylorturner  staff       24393 Nov 12 05:18 ib_buffer_pool
-rwxrwxrwx@   1 taylorturner  staff    50331648 Nov 17 13:14 ib_logfile0
-rwxrwxrwx@   1 taylorturner  staff    50331648 Nov 12 05:25 ib_logfile1
-rwxrwxrwx@   1 taylorturner  staff  1421869056 Nov 17 13:14 ibdata1
-rwxrwxrwx@   1 taylorturner  staff    12582912 Nov 12 05:18 ibtmp1
drwxrwxrwx@  77 taylorturner  staff        2464 Feb  2  2021 mysql
-rwxrwxrwx@   1 taylorturner  staff           6 Feb  2  2021 mysql_upgrade_info
drwxrwxrwx@ 396 taylorturner  staff       12672 Nov 12 04:51 <redacted>
drwxrwxrwx@  90 taylorturner  staff        2880 Feb  2  2021 performance_schema
-rwxrwxrwx@   1 taylorturner  staff        1680 Nov 20  2019 private_key.pem
-rwxrwxrwx@   1 taylorturner  staff         452 Nov 20  2019 public_key.pem
-rwxrwxrwx@   1 taylorturner  staff        1112 Nov 20  2019 server-cert.pem
-rwxrwxrwx@   1 taylorturner  staff        1676 Nov 20  2019 server-key.pem
drwxrwxrwx@ 108 taylorturner  staff        3456 Nov 20  2019 sys

• It seems that no matter what we do, the mount always ends up with root:root ownership when the MySQL daemon is expecting mysql:mysql ownership. We can't figure out how to get around without going to extreme measures (pulling in the entrypoint.sh script from this repo and making our own tweaks).

Other notes:

• Our prod DB is running in AWS RDS, and using version 5.7 but I'm not sure on the patch number (5.7.?).
• We're able to get this working in an Ubuntu container after installing mysql-server-5.7 so we know the datadir itself isn't corrupted or anything. This is also how we know that our $MYSQL_ROOT_PASSWORD is correct.

Thanks!

[andrius-sperque]

+1

[tianon]

I'm guessing there's probably a suitable user: value you could set in your docker-compose.yml to make this work -- even just user: mysql:mysql might work.

However, this is really a Docker for Mac specific issue (Linux doesn't have the same filesystem sharing, which is frankly pretty quirky), and I would suggest trying a dedicated support forum, such as the Docker Community Forums, the Docker Community Slack, or Stack Overflow for further help figuring out a solution.