General docker-compose: ISLE-MYSQL consider use of a volume for data

[br2490]

Having data on volume may make it easier to get at those DB files, maybe even add your own data directly into it.

Not sure (read: highly doubt) if moving entire SQL data folders around is something to do.
May just rely on dump and import for CUSTOMIZATION. To that end: review the MySQL default init or entrypoint script - I think it will scour a folder looking for sql, sql.gz or .tar files and import.

Please review my refactor pull and see if it made sense?

See #63 #64

tagging: @Islandora-Collaboration-Group/isle-steering-committee

[g7morris]

@br2490 I'm confused by this statement

"Having data on volume may make it easier to get at those DB files"

Within the Docker-compose service entry for mysql, exist these volumes:

volumes:

  - ./mysql/initscripts:/docker-entrypoint-initdb.d   

(this is how one can add a database using a sql file, there is an example)

  - ./customize/mysql/my.cnf:/etc/alternatives/my.cnf 

(mysql cnf file peristed for changes)

  - ./data/mysql:/var/lib/mysql

(databases live and persist here)

  - ./data/mysql/log:/var/log/mysql

(logs logs logs)

What is missing exactly? I believe these volumes are doing what you've explained above? Right?

Or are you suggesting simply streamlining down to one volume?

[br2490]

@g7morris, sorryI was not clear! docker volume is what I'm describing as per https://docs.docker.com/engine/admin/volumes/volumes/. This is a fslayer that lets dockerd handle persisted data

[g7morris]

@br2490 Except I'm still not following.

How is the "volume" reference in the current Docker Compose file not a docker volume?

Could you elaborate on what is missing from the current method please?

[br2490]

@g7morris: in a nutshell I think docker volumes are filesystems managed by docker and relatively 'independent' from the host. Docker handling the FS means docker controls the data and can do important stuff with it (e.g., migration). The bind/mount is dependent on the host system having files at a certain location on start - docker doesn't know about it, and will create an empty folder if it doesn't find something there.

An example can be found in #67: in that docker-compose you will see a section for docker-managed volumes and calls for a default isle-mysql volume for data. docker volume and mount points are slightly different. I think that article covers volumes versus binds/mounts well and volume advantages. Just a thought!

[g7morris]

@br2490 Okay! I think I get it now. Thanks for being patient and explaining on this one.

There isn't any apparent distinction in the documentation thus my confusion on what was happening. Perhaps they could use more code eh?

This helped.
https://stackoverflow.com/questions/34357252/docker-data-volume-vs-mounted-host-directory

Specifically several quotes:

Quote 1:

The host directory is, by its nature, host-dependent. For this reason, you can’t mount a host directory from Dockerfile because built images should be portable. A host directory wouldn’t be available on all potential hosts.

Quote 2:

Here we’ve launched a new container and mounted the volume from the dbdata container.
We’ve then mounted a local host directory as /backup. Finally, we’ve passed a command that uses tar to backup the contents of the dbdata volume to a backup.tar file inside our /backup directory. When the command completes and the container stops we’ll be left with a backup of our dbdata volume.

Okay with that bold part it finally clicked. Yes, agreed now that I'm on the same page. We'll need to revisit this entirely.

It might actually make things easier as permissions on Docker hosts vs containers is a tricky thing already. I'd like to avoid it all together.

I think more reading is in my future clearly however as I'm not clear on how to set permissions within this new structure.

[g7morris]

@br2490 Echoing from the Slack post:

New Example Volume structure:

version: '2'
services:

mysql:
image: islandoracollabgroup/isle-mysql:latest
environment:
- MYSQL_ROOT_PASSWORD=islemysqlrootpw2017
- MYSQL_DATABASE=fedora3
- MYSQL_USER=fedora_admin
- MYSQL_PASSWORD=dockerfeddb2017
ports:
- "3306:3306"
volumes:
- mycnf:/etc/alternatives/my.cnf
- mysql_data:/var/lib/mysql
- mysql_logs:/var/log/mysql
container_name: isle-mysql

fedora:
image: islandoracollabgroup/isle-fedora:latest
ports:
- "8080:8080"
- "8777:80"
tty: true
depends_on:
- mysql
volumes:
- fed_tc_conf:/usr/local/tomcat/conf
- fed_tc_logs:/usr/local/tomcat/logs
- fed_data:/usr/local/fedora/data
- fed_conf:/usr/local/fedora/server/config
- fed_logs:/usr/local/fedora/server/logs
- fed_dja_conf:/usr/local/tomcat/webapps/adore-djatoka/index.html
- fed_apache_conf:/etc/apache2/sites-available
- fed_apache_logs:/var/log/apache2
container_name: isle-fedora

solr:
image: islandoracollabgroup/isle-solr:latest
ports:
- "8091:8080"
- "8983:8983"
tty: true
volumes:
- sol_tc_conf::/usr/local/tomcat/conf
- sol_tc_logs:/usr/local/tomcat/logs
- sol_data/usr/local/solr/collection1
container_name: isle-solr

apache:
build: ./apache
image: islandoracollabgroup/isle-apache:latest
ports:
- "80:80"
tty: true
depends_on:
- mysql
- fedora
volumes:
- apache_conf:/etc/apache2/sites-available/site.conf
- apache_data:/var/www/html
- apache_logs/apache:/var/log/apache2
- php_conf:/etc/php5/apache2/php.ini
container_name: isle-apache

This is what it “feels” like we should be moving to. Granted I’ll need to test and work out if this is feasible. Also will need to update each respective Dockerfile to add / remove each structure. This would be most likely the death knell of the customization folder (as intended? and which we’ll handle with proper documentation for copying of files to the data volume e.g. install_site.sh)

Your thoughts?

[br2490]

Voting to close this issue as it is now 'legacy'.

Volumes and mounts are understood and we will implement them to persist data when and where possible (and logical!).

@g7morris