New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker #15

Closed
mstenta opened this Issue Aug 20, 2016 · 29 comments

Comments

Projects
None yet
4 participants
@mstenta
Member

mstenta commented Aug 20, 2016

The farmOS 7.x-1.x branch has a Dockerfile for building a Docker image: https://github.com/farmOS/farmOS/blob/7.x-1.x/Dockerfile

And we have a "docker" branch for figuring out some of the next steps: https://github.com/farmOS/farmOS/compare/docker

We also now have an official automated build on Docker Hub: https://hub.docker.com/r/farmos/farmos/

However, there are still some kinks to work out... I'm creating this issue to create a central place for discussion of those. Let's focus all new general discussion here, and from this we can maybe spin up some pull requests (if necessary).

Here are some previous (closed) discussions for reference and context:

#6
#10
#11
#13

Here are some related issues and PRs that are still open for discussion:

#14
#12

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Aug 20, 2016

Member

Also worth posting a link to this related issue in the official Drupal Docker image queue: docker-library/drupal#3

Some of the same things we are working through might be solved upstream eventually. So we should keep an eye on that.

Member

mstenta commented Aug 20, 2016

Also worth posting a link to this related issue in the official Drupal Docker image queue: docker-library/drupal#3

Some of the same things we are working through might be solved upstream eventually. So we should keep an eye on that.

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Aug 20, 2016

Member

One of the big issues I've found right now: farmOS cannot be installed via 127.0.0.1 - you must install via the Docker container IP.

Also, installation and usage in general seems to be VERY slow.

Here's what I said in the other pull request (#10 (comment)):

Another thing I'm finding: it's possible to access the farmOS container simply by going to 127.0.0.1 (@rjsteinert showed me this) - but for some reason it is not able to connect to the database for installation on that IP (see possible related issue in Wordpress Docker queue here: docker-library/wordpress#89).

If, however, I go to the actual container IP (in my case: 172.18.0.3), then it DOES connect to the db container.

The weird thing is... once farmOS is installed - I can access it just fine from 127.0.0.1 - and the database connection works as expected. I don't understand that...

Also, I'm finding that everything is EXTREMELY slow - especially installation.

Member

mstenta commented Aug 20, 2016

One of the big issues I've found right now: farmOS cannot be installed via 127.0.0.1 - you must install via the Docker container IP.

Also, installation and usage in general seems to be VERY slow.

Here's what I said in the other pull request (#10 (comment)):

Another thing I'm finding: it's possible to access the farmOS container simply by going to 127.0.0.1 (@rjsteinert showed me this) - but for some reason it is not able to connect to the database for installation on that IP (see possible related issue in Wordpress Docker queue here: docker-library/wordpress#89).

If, however, I go to the actual container IP (in my case: 172.18.0.3), then it DOES connect to the db container.

The weird thing is... once farmOS is installed - I can access it just fine from 127.0.0.1 - and the database connection works as expected. I don't understand that...

Also, I'm finding that everything is EXTREMELY slow - especially installation.

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Aug 20, 2016

Member

Also: it seems that the three libraries defined in the Drush make file (Backbone, Modernizr, and Underscore - all dependencies of the Navbar module) are not being detected... but they do exist in /profiles/farm/libraries... hmm...

Member

mstenta commented Aug 20, 2016

Also: it seems that the three libraries defined in the Drush make file (Backbone, Modernizr, and Underscore - all dependencies of the Navbar module) are not being detected... but they do exist in /profiles/farm/libraries... hmm...

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Aug 20, 2016

Member

Ok, sooo... the database issue I mentioned above (#15 (comment)) seems to be behaving inconsistently for me. I destroyed my containers and rebuilt them, and now the opposite happened: it worked on 127.0.0.1, but not on the Docker IP. I'll see if I can find a pattern to it... but maybe it's just a weird issue on my laptop.

@thedude459 - I am testing your libgeos-dev addition in my docker branch... I'll let you know if that works.

Member

mstenta commented Aug 20, 2016

Ok, sooo... the database issue I mentioned above (#15 (comment)) seems to be behaving inconsistently for me. I destroyed my containers and rebuilt them, and now the opposite happened: it worked on 127.0.0.1, but not on the Docker IP. I'll see if I can find a pattern to it... but maybe it's just a weird issue on my laptop.

@thedude459 - I am testing your libgeos-dev addition in my docker branch... I'll let you know if that works.

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Aug 20, 2016

Member

One other thing that needs to be figured out is how to deal with settings.php. When you destroy a container, that gets blown away too. Then when you try to run the site, it brings up install.php (because settings.php doesn't exist), but then complains because the database is already defined.

Member

mstenta commented Aug 20, 2016

One other thing that needs to be figured out is how to deal with settings.php. When you destroy a container, that gets blown away too. Then when you try to run the site, it brings up install.php (because settings.php doesn't exist), but then complains because the database is already defined.

@thedude459

This comment has been minimized.

Show comment
Hide comment
@thedude459

thedude459 Aug 23, 2016

Contributor

This is the docker that I currently use for my database https://hub.docker.com/r/linuxserver/mariadb/

Contributor

thedude459 commented Aug 23, 2016

This is the docker that I currently use for my database https://hub.docker.com/r/linuxserver/mariadb/

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Aug 24, 2016

Member

@thedude459 Cool I've always been curious about trying MariaDB. I'll give it a try with my local docker tests - if it all works without issue we can definitely consider using that over MySQL.

Member

mstenta commented Aug 24, 2016

@thedude459 Cool I've always been curious about trying MariaDB. I'll give it a try with my local docker tests - if it all works without issue we can definitely consider using that over MySQL.

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Sep 22, 2016

Member

Update: I did a little more fiddling with the docker branch yesterday. Might do some more today.

One thing I learned, which should be documented somewhere in farmOS docs: when you spin up the apache container and the database container using docker compose, it can take a little longer for the database container to start up - so if you go to the Drupal install page and try to install it it can't find it. But if you want a minute or two, then it works. This caused me some confusion a few times, but then I realized it was just taking longer.

https://www.reddit.com/r/docker/comments/3sbxv6/why_on_earth_isnt_my_mysql_container_working_with/cwvuf7m?st=iteoliv7&sh=8ee81ed8

Member

mstenta commented Sep 22, 2016

Update: I did a little more fiddling with the docker branch yesterday. Might do some more today.

One thing I learned, which should be documented somewhere in farmOS docs: when you spin up the apache container and the database container using docker compose, it can take a little longer for the database container to start up - so if you go to the Drupal install page and try to install it it can't find it. But if you want a minute or two, then it works. This caused me some confusion a few times, but then I realized it was just taking longer.

https://www.reddit.com/r/docker/comments/3sbxv6/why_on_earth_isnt_my_mysql_container_working_with/cwvuf7m?st=iteoliv7&sh=8ee81ed8

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Sep 22, 2016

Member

@thedude459 Sorry I haven't had a chance to try your branch yet - I will try to today.

The one thing that comes to mind reading your comment, though: if we mount the entire farmOS codebase in a volume it means we can't simply destroy the container and download the new version in order to update farmOS. The update would need to be performed manually using Drush Make in the mounted volume, which sort of defeats one of the main purposes of containers.

However, I still haven't thought of the perfect way to do it... we might need two different variants: one for a production farmOS site, and one for development - the development one would require more manual setup and updating.

I'm spending some time playing around with the vanilla Drupal 8 docker image - gaining a lot of new knowledge and experience. I think things will start to come together as they make sense...

Member

mstenta commented Sep 22, 2016

@thedude459 Sorry I haven't had a chance to try your branch yet - I will try to today.

The one thing that comes to mind reading your comment, though: if we mount the entire farmOS codebase in a volume it means we can't simply destroy the container and download the new version in order to update farmOS. The update would need to be performed manually using Drush Make in the mounted volume, which sort of defeats one of the main purposes of containers.

However, I still haven't thought of the perfect way to do it... we might need two different variants: one for a production farmOS site, and one for development - the development one would require more manual setup and updating.

I'm spending some time playing around with the vanilla Drupal 8 docker image - gaining a lot of new knowledge and experience. I think things will start to come together as they make sense...

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Sep 23, 2016

Member

@thedude459 - Your branch had a lot more changes in it than I was expecting. I'd like to break them up into smaller chunks so we can decide which ones are necessary, and what they all do.

@rjsteinert and @thedude459 - Want to do a Google Hangout sometime soon to talk about this stuff in person? I think we're getting pretty close.

I found these two projects that maybe we can learn from as well:

https://www.drupal.org/node/2736447
http://docker4drupal.org/

Member

mstenta commented Sep 23, 2016

@thedude459 - Your branch had a lot more changes in it than I was expecting. I'd like to break them up into smaller chunks so we can decide which ones are necessary, and what they all do.

@rjsteinert and @thedude459 - Want to do a Google Hangout sometime soon to talk about this stuff in person? I think we're getting pretty close.

I found these two projects that maybe we can learn from as well:

https://www.drupal.org/node/2736447
http://docker4drupal.org/

@thedude459

This comment has been minimized.

Show comment
Hide comment
@thedude459

thedude459 Sep 24, 2016

Contributor

Does the current docker keep all the data needed to keep the website and database intact when you destroy the container?

Contributor

thedude459 commented Sep 24, 2016

Does the current docker keep all the data needed to keep the website and database intact when you destroy the container?

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Sep 25, 2016

Member

No, the files and settings.php are still in the container, so they are lost if you destroy and rebuild. Our image inherits this same issue from the official D7 docker hub image.

Ultimately I think this needs to be fixed upstream in that repository, and it seems like there's some consensus building on exactly how to do that here: docker-library/drupal#3 (essentially following the lead of the Wordpress docker repo). It's a "Drupal on Docker" problem, not one that is specific to farmOS, so I'd like to see it fixed in a general way, in hopes that we don't end up making more work for ourselves in the future.

I still don't feel confident enough with my Docker abilities to pitch in and help with that issue. But I'm slowly getting up to speed on things, so maybe I will soon.

In the meantime, I'm also trying to conceptualize the difference between a "production" farmOS Docker setup and one for local development. I am personally more interested in using Docker for local development than for production hosting, so I'm exploring how to integrate it into my normal development routine, and what is necessary in the Dockerfile and docker-compose.yml to accommodate that. But I want to make sure that they can be used in production too, so I'm learning what those differences entail in the context of Docker... I'll try to organize some of my thoughts and post them here soon...

Member

mstenta commented Sep 25, 2016

No, the files and settings.php are still in the container, so they are lost if you destroy and rebuild. Our image inherits this same issue from the official D7 docker hub image.

Ultimately I think this needs to be fixed upstream in that repository, and it seems like there's some consensus building on exactly how to do that here: docker-library/drupal#3 (essentially following the lead of the Wordpress docker repo). It's a "Drupal on Docker" problem, not one that is specific to farmOS, so I'd like to see it fixed in a general way, in hopes that we don't end up making more work for ourselves in the future.

I still don't feel confident enough with my Docker abilities to pitch in and help with that issue. But I'm slowly getting up to speed on things, so maybe I will soon.

In the meantime, I'm also trying to conceptualize the difference between a "production" farmOS Docker setup and one for local development. I am personally more interested in using Docker for local development than for production hosting, so I'm exploring how to integrate it into my normal development routine, and what is necessary in the Dockerfile and docker-compose.yml to accommodate that. But I want to make sure that they can be used in production too, so I'm learning what those differences entail in the context of Docker... I'll try to organize some of my thoughts and post them here soon...

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Sep 25, 2016

Member

Here are some of my thoughts around the different requirements for a production setup and a development setup with farmOS+Docker:

Common requirements:

  • Volume for storing sites/default/settings.php and sites/default/files outside of the container (this is being figured out here: docker-library/drupal#3)
  • ... (more?)

Production requirements:

  • Ability to update the farmOS codebase to a new version by destroying the container, pulling the new version image, and rebuilding.
  • Ability to add additional modules not included in farmOS core? (maybe by mounting /sites/all on a volume?)
  • No assumptions about the database server (but provide a docker-compose.yml that sets one up if you want to use that)
  • ... (more?)

Development requirements:

  • Entire farmOS codebase stored outside of the container, so that IDE like PHPStorm can be used (or maybe PHPStorm can connect and edit code within the container? See: https://www.jetbrains.com/help/phpstorm/2016.2/docker.html) - This may be at odds with the first production requirement listed above (ability to pull new farmOS image to update farmOS) because the codebase would be outside the container. And in development, we don't want the codebase to be overwritten by pulling a new image.
  • Automatically clone all the individual farmOS module Git repos (ie: farm_log, farm_crop, farm_livestock are all separate repos that are included in the farmOS distribution, and most development happens in the modules, not the distro).
  • Xdebug installed for IDE debugging (this should NOT be installed in the production image).
  • ... (more?)
Member

mstenta commented Sep 25, 2016

Here are some of my thoughts around the different requirements for a production setup and a development setup with farmOS+Docker:

Common requirements:

  • Volume for storing sites/default/settings.php and sites/default/files outside of the container (this is being figured out here: docker-library/drupal#3)
  • ... (more?)

Production requirements:

  • Ability to update the farmOS codebase to a new version by destroying the container, pulling the new version image, and rebuilding.
  • Ability to add additional modules not included in farmOS core? (maybe by mounting /sites/all on a volume?)
  • No assumptions about the database server (but provide a docker-compose.yml that sets one up if you want to use that)
  • ... (more?)

Development requirements:

  • Entire farmOS codebase stored outside of the container, so that IDE like PHPStorm can be used (or maybe PHPStorm can connect and edit code within the container? See: https://www.jetbrains.com/help/phpstorm/2016.2/docker.html) - This may be at odds with the first production requirement listed above (ability to pull new farmOS image to update farmOS) because the codebase would be outside the container. And in development, we don't want the codebase to be overwritten by pulling a new image.
  • Automatically clone all the individual farmOS module Git repos (ie: farm_log, farm_crop, farm_livestock are all separate repos that are included in the farmOS distribution, and most development happens in the modules, not the distro).
  • Xdebug installed for IDE debugging (this should NOT be installed in the production image).
  • ... (more?)
@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Sep 26, 2016

Member

I like the idea of having the docker stuff including in the main farmOS repo... but if it starts to get complex I wonder if we should consider splitting it out into a separate repo? That's the approach that Aegir has taken, and they have different Dockerfiles for development and production, along with related scripts and docker-related stuff: https://github.com/aegir-project/dockerfiles

Something to consider...

Member

mstenta commented Sep 26, 2016

I like the idea of having the docker stuff including in the main farmOS repo... but if it starts to get complex I wonder if we should consider splitting it out into a separate repo? That's the approach that Aegir has taken, and they have different Dockerfiles for development and production, along with related scripts and docker-related stuff: https://github.com/aegir-project/dockerfiles

Something to consider...

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Oct 3, 2016

Member

I spent some time digging into the upstream issue and submitted a pull request: docker-library/drupal#63

It adds an entrypoint script, and allows the entire /var/www/html directory to be mounted as a volume. This accomplishes all of the "common" and "production" requirements listed above, as well as most of the "development" ones.

Of course, that stuff only applies to the Drupal docker images - not ones that inherit from it. So I also created an issue to explore supporting distributions like farmOS: docker-library/drupal#64

Member

mstenta commented Oct 3, 2016

I spent some time digging into the upstream issue and submitted a pull request: docker-library/drupal#63

It adds an entrypoint script, and allows the entire /var/www/html directory to be mounted as a volume. This accomplishes all of the "common" and "production" requirements listed above, as well as most of the "development" ones.

Of course, that stuff only applies to the Drupal docker images - not ones that inherit from it. So I also created an issue to explore supporting distributions like farmOS: docker-library/drupal#64

@kadaan

This comment has been minimized.

Show comment
Hide comment
@kadaan

kadaan Oct 6, 2016

If it is mainly for dev, why not use Vagrant?

kadaan commented Oct 6, 2016

If it is mainly for dev, why not use Vagrant?

@kadaan

This comment has been minimized.

Show comment
Hide comment
@kadaan

kadaan Oct 6, 2016

@mstenta @thedude459
Here is some feedback regarding the current docker state:

  1. I was not able to get farmOS running by starting the dockerhub official farmOS image. I was blocked at trying to setup the DB and couldn't get it to connect to my local instance of mysql.
  2. I was not able to get farmOS running by using thee dockerfile in the 7.x-1.x branch. I was blocked at trying to setup the DB and couldn't get it to connect to my local instance of mysql.
  3. I was able to get farmOS running by using the docker branch and the steps listed above by @thedude459. farmOS started, but the site was not themed and was getting a bunch of errors. I ran cp -rv profiles/farm/modules/farm_theme sites/all/themes and then enabled the theme in Manage -> Appearance. After that the site seems to be working.

NOTE: I also ended up moving all the modules at profiles/farm/modules/farm/* to sites/all/modules/ and clearing caches so that I can work on the files from the host.

kadaan commented Oct 6, 2016

@mstenta @thedude459
Here is some feedback regarding the current docker state:

  1. I was not able to get farmOS running by starting the dockerhub official farmOS image. I was blocked at trying to setup the DB and couldn't get it to connect to my local instance of mysql.
  2. I was not able to get farmOS running by using thee dockerfile in the 7.x-1.x branch. I was blocked at trying to setup the DB and couldn't get it to connect to my local instance of mysql.
  3. I was able to get farmOS running by using the docker branch and the steps listed above by @thedude459. farmOS started, but the site was not themed and was getting a bunch of errors. I ran cp -rv profiles/farm/modules/farm_theme sites/all/themes and then enabled the theme in Manage -> Appearance. After that the site seems to be working.

NOTE: I also ended up moving all the modules at profiles/farm/modules/farm/* to sites/all/modules/ and clearing caches so that I can work on the files from the host.

@rjsteinert

This comment has been minimized.

Show comment
Hide comment
@rjsteinert

rjsteinert Oct 6, 2016

Contributor

@kadaan

If it is mainly for dev, why not use Vagrant?

Docker for Mac and Windows was similar to Vagrant for a while in how it used VirtualBox to get the environment. Now Docker can be run more natively on Mac and Windows., albeit it is still a form of virtualization but the performance is better than that of a VM in VirtualBox. The BIG downside right now is that on Mac if you map a volume from inside the container to the host machine, it's maaaaaad slow. So while Docker technically works on Mac, most case involve mapping a Volume which kills performance which begs the question of why Docker for Mac was ever declared 1.0.

I was blocked at trying to setup the DB and couldn't get it to connect to my local instance of mysql

Were you running docker compose up or setting up the farmos and mysql containers manually?

I ran cp -rv profiles/farm/modules/farm_theme sites/all/themes and then enabled the theme

Interesting. I wonder if you are running into the file ownership issues that other Drupal on Docker installations face as well.

@mstenta To fill the gaps that docker-compose does not, we could write shell scripts for start.sh (production) and develop.sh (development). They might be as simple as...

start.sh

docker-compose up --file docker-compose-production.yml
docker exec <container name> chown ...

develop.sh

docker-compose up --file docker-compose-develop.yml
docker exec <container name> chown ...
Contributor

rjsteinert commented Oct 6, 2016

@kadaan

If it is mainly for dev, why not use Vagrant?

Docker for Mac and Windows was similar to Vagrant for a while in how it used VirtualBox to get the environment. Now Docker can be run more natively on Mac and Windows., albeit it is still a form of virtualization but the performance is better than that of a VM in VirtualBox. The BIG downside right now is that on Mac if you map a volume from inside the container to the host machine, it's maaaaaad slow. So while Docker technically works on Mac, most case involve mapping a Volume which kills performance which begs the question of why Docker for Mac was ever declared 1.0.

I was blocked at trying to setup the DB and couldn't get it to connect to my local instance of mysql

Were you running docker compose up or setting up the farmos and mysql containers manually?

I ran cp -rv profiles/farm/modules/farm_theme sites/all/themes and then enabled the theme

Interesting. I wonder if you are running into the file ownership issues that other Drupal on Docker installations face as well.

@mstenta To fill the gaps that docker-compose does not, we could write shell scripts for start.sh (production) and develop.sh (development). They might be as simple as...

start.sh

docker-compose up --file docker-compose-production.yml
docker exec <container name> chown ...

develop.sh

docker-compose up --file docker-compose-develop.yml
docker exec <container name> chown ...
@kadaan

This comment has been minimized.

Show comment
Hide comment
@kadaan

kadaan Oct 6, 2016

@rjsteinert. I used docker-compose as listed in the steps by @thedude459.

I would also say that it would be preferable for the developer to get a working site, rather than have to go through a manual install and config process.

kadaan commented Oct 6, 2016

@rjsteinert. I used docker-compose as listed in the steps by @thedude459.

I would also say that it would be preferable for the developer to get a working site, rather than have to go through a manual install and config process.

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Oct 6, 2016

Member

Hey @kadaan and @rjsteinert !

So I think I have a pretty clear plan for this... after all the stuff I learned working on docker-library/drupal#63 - and I think it will meet most of the needs described in my comment above #15 (comment)

I'm not sure if that PR will be accepted - and if it is, it might take a while - but in the meantime I think I will just replicate the same approach in this repo. Then, if they do accept the PR in the future, we can simply remove the stuff from this repo and fall back on theirs (I think anyway).

Essentially, to summarize my plan: we will mount the entire /var/www/html folder as a volume, and use a docker-entrypoint.sh script to set up to farmOS codebase in it. The entrypoint script will also automatically handle upgrades of farmOS core.

Thus, the farmOS codebase will be available for development locally within the volume-mounted folder.

The only pieces that it doesn't do are:

Automatically clone all the individual farmOS module Git repos (ie: farm_log, farm_crop, farm_livestock are all separate repos that are included in the farmOS distribution, and most development happens in the modules, not the distro).

and...

Xdebug installed for IDE debugging (this should NOT be installed in the production image).

I see those as good follow-up tasks, as they are not essential to the general task of getting a working Docker image that can be used for both production and development. And maybe we can use @rjsteinert suggestion to include a bash script that sets those up for development specifically.

What do you think? If that makes sense, I can go ahead and cherry-pick my commits form the pull request into this repo pretty quickly, and we can call this done!

Member

mstenta commented Oct 6, 2016

Hey @kadaan and @rjsteinert !

So I think I have a pretty clear plan for this... after all the stuff I learned working on docker-library/drupal#63 - and I think it will meet most of the needs described in my comment above #15 (comment)

I'm not sure if that PR will be accepted - and if it is, it might take a while - but in the meantime I think I will just replicate the same approach in this repo. Then, if they do accept the PR in the future, we can simply remove the stuff from this repo and fall back on theirs (I think anyway).

Essentially, to summarize my plan: we will mount the entire /var/www/html folder as a volume, and use a docker-entrypoint.sh script to set up to farmOS codebase in it. The entrypoint script will also automatically handle upgrades of farmOS core.

Thus, the farmOS codebase will be available for development locally within the volume-mounted folder.

The only pieces that it doesn't do are:

Automatically clone all the individual farmOS module Git repos (ie: farm_log, farm_crop, farm_livestock are all separate repos that are included in the farmOS distribution, and most development happens in the modules, not the distro).

and...

Xdebug installed for IDE debugging (this should NOT be installed in the production image).

I see those as good follow-up tasks, as they are not essential to the general task of getting a working Docker image that can be used for both production and development. And maybe we can use @rjsteinert suggestion to include a bash script that sets those up for development specifically.

What do you think? If that makes sense, I can go ahead and cherry-pick my commits form the pull request into this repo pretty quickly, and we can call this done!

@kadaan

This comment has been minimized.

Show comment
Hide comment
@kadaan

kadaan Oct 6, 2016

That sounds great to me!!!

kadaan commented Oct 6, 2016

That sounds great to me!!!

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Oct 6, 2016

Member

Another thing: we may want to use PHP 5.6 instead of PHP 7 (for farmOS 7.x-1.x at least), to avoid any potential PHP 7 compatability issues ( @kadaan said he's seeing some, and I've seen some too, although haven't dug in specifically - also I think the GEOS library may only work with PHP 5.6 currently - see #12 (comment)).

In which case, it means we would need to inherit from PHP 5.6 image directly, rather than using the Drupal image as a base... because that uses PHP 7.

Member

mstenta commented Oct 6, 2016

Another thing: we may want to use PHP 5.6 instead of PHP 7 (for farmOS 7.x-1.x at least), to avoid any potential PHP 7 compatability issues ( @kadaan said he's seeing some, and I've seen some too, although haven't dug in specifically - also I think the GEOS library may only work with PHP 5.6 currently - see #12 (comment)).

In which case, it means we would need to inherit from PHP 5.6 image directly, rather than using the Drupal image as a base... because that uses PHP 7.

@kadaan

This comment has been minimized.

Show comment
Hide comment
@kadaan

kadaan Oct 8, 2016

@rjsteinert What about using something like docker-sync to get rid of the slowness? https://github.com/kadaan/farmOS/tree/DockerSync

kadaan commented Oct 8, 2016

@rjsteinert What about using something like docker-sync to get rid of the slowness? https://github.com/kadaan/farmOS/tree/DockerSync

@kadaan

This comment has been minimized.

Show comment
Hide comment
@kadaan

kadaan Oct 11, 2016

@mstenta I'm thinking about this again, and I can see four scenarios:

  1. Module Developer - As a module developer, I want to start farmOS from a git clone of the source and modules, so that I can easily address issues and add features.
  2. Installer Developer - As a farmOS installer developer, I want to start farmOS and run through the installer, so that I can improve the installer and verify that it will work for the end user.
  3. New User - As a new farmOS user, I want to be able to start farmOS with the minimum number of steps necessary, so that I can see if it meets my needs.
  4. Advanced User - As an advanced farmOS user, I want to be able to customize my farmOS installation, without losing my data, so that it can handle more load.

Any other scenarios that I'm missing? While I think that all of these scenarios can have their needs met by a Docker solution, I don't think that it is a requirement. I believe that simplifying the module developer and new user scenarios will allow more community involvement with the project. The advanced user and installer developer are inherently more tolerant of the steps needed to setup the system.

What do you think?

kadaan commented Oct 11, 2016

@mstenta I'm thinking about this again, and I can see four scenarios:

  1. Module Developer - As a module developer, I want to start farmOS from a git clone of the source and modules, so that I can easily address issues and add features.
  2. Installer Developer - As a farmOS installer developer, I want to start farmOS and run through the installer, so that I can improve the installer and verify that it will work for the end user.
  3. New User - As a new farmOS user, I want to be able to start farmOS with the minimum number of steps necessary, so that I can see if it meets my needs.
  4. Advanced User - As an advanced farmOS user, I want to be able to customize my farmOS installation, without losing my data, so that it can handle more load.

Any other scenarios that I'm missing? While I think that all of these scenarios can have their needs met by a Docker solution, I don't think that it is a requirement. I believe that simplifying the module developer and new user scenarios will allow more community involvement with the project. The advanced user and installer developer are inherently more tolerant of the steps needed to setup the system.

What do you think?

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Oct 12, 2016

Member

@kadaan That's a great summary. I think you hit the nail on the head identifying the new user and module developer scenarios as the most important. The other two (installer developer and advanced user) are very closely related to module developer - as they require a lot of the same skills/knowledge (module developer is arguably the most in-depth of them all).

And I think we can accommodate all four of them through a combination of things like this Docker image, along with some helper scripts for setup, and above all: thorough documentation.

In response to your earlier question:

If it is mainly for dev, why not use Vagrant?

I'd love for the docker image to be for BOTH production and development use. Of course, my experience with Docker is limited, so I am not very familiar with what production use generally looks like. But I am following the patterns of some of the official Docker images as examples, so I think we'll be on the right track.

With regard to Vagrant: I use Vagrant for all of my local development currently. So this exploration of Docker is a movement away from virtual machines for me. I still love Vagrant, and perhaps there is an argument for including a Vagrantfile in the repo as well. But let's start with Docker and see if it gets us what we need.

Member

mstenta commented Oct 12, 2016

@kadaan That's a great summary. I think you hit the nail on the head identifying the new user and module developer scenarios as the most important. The other two (installer developer and advanced user) are very closely related to module developer - as they require a lot of the same skills/knowledge (module developer is arguably the most in-depth of them all).

And I think we can accommodate all four of them through a combination of things like this Docker image, along with some helper scripts for setup, and above all: thorough documentation.

In response to your earlier question:

If it is mainly for dev, why not use Vagrant?

I'd love for the docker image to be for BOTH production and development use. Of course, my experience with Docker is limited, so I am not very familiar with what production use generally looks like. But I am following the patterns of some of the official Docker images as examples, so I think we'll be on the right track.

With regard to Vagrant: I use Vagrant for all of my local development currently. So this exploration of Docker is a movement away from virtual machines for me. I still love Vagrant, and perhaps there is an argument for including a Vagrantfile in the repo as well. But let's start with Docker and see if it gets us what we need.

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Oct 15, 2016

Member

I've been working on this (as time allows with a new baby) - making good progress. Hoping to push some commits up soon.

One thing to note: I'm still having issues trying to install farmOS in an SQLite database. I created a separate issue for that here: https://www.drupal.org/node/2818907

I will follow up on that afterwards.

Member

mstenta commented Oct 15, 2016

I've been working on this (as time allows with a new baby) - making good progress. Hoping to push some commits up soon.

One thing to note: I'm still having issues trying to install farmOS in an SQLite database. I created a separate issue for that here: https://www.drupal.org/node/2818907

I will follow up on that afterwards.

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Oct 18, 2016

Member

Alright! I just pushed a bunch of commits. Here's the summary of the docker stuff as it works currently:

  • Now it inherits from the php:5.6-apache image, rather than the drupal:7 image, so that we could use PHP 5.6 instead of PHP 7.0. I replicated the two additional things that the drupal:7 image was doing:
    • Installs the PHP extensions that Drupal needs.
    • Enables the Apache rewrite module.
  • Changed from mysql:5.7 to mysql:latest in docker-compose.yml, so that we don't need to manually update that if/when new versions come out.
  • Set up docker-compose.yml and Dockerfile to mount the entire farmOS codebase (/var/www/html) as a volume. See below for how this is managed automatically.
  • Removed the code from Dockerfile that built farmOS via drush make - this moved to the new docker-entrypoint.sh script (see below).
  • Added three environment variables, which are used in the docker-entrypoint.sh script (see below).
    • FARMOS_VERSION - the packaged version to download
    • FARMOS_DEV_BRANCH - the development branch to build from
    • FARMOS_DEV - whether or not to create a development environment or use the packaged version - this defaults to false
  • Added a docker-entrypoint.sh ENTRYPOINT script that is run every time that a container is created with docker run. This script does the following:
    • Creates the farmOS codebase in /var/www/html in one of two ways:
      • If FARMOS_DEV is true, it builds farmOS with drush make --working-copy
      • If FARMOS_DEV is false (default), it downloads and unpacks the official packaged release from drupal.org
    • Each time a container is spun up, the farmOS codebase in /var/www/html is rebuilt. The /var/www/html/sites directory, however, is preserved - so settings.php, uploaded files, and extra modules are not lost.

I also added documentation that describes all of that, along with instructions for use. It goes into a bit more detail than this comment, so be sure to read through that, and please test the instructions! http://farmos.org/development/docker/

All of this has been merged into the main 7.x-1.x branches. I've tested it a bunch locally, but let me know what you think, and if anything isn't working or doesn't make sense.

There are a few things that still need to be done, but we can do them as follow-up tasks I think:

  • #12
  • Currently emails can't be sent, and on install the following error shows: Unable to send e-mail. Contact the site administrator if the problem persists.
  • PECL Uploadprogress should be installed since all the file fields in farmOS use the uploadprogress widget.
Member

mstenta commented Oct 18, 2016

Alright! I just pushed a bunch of commits. Here's the summary of the docker stuff as it works currently:

  • Now it inherits from the php:5.6-apache image, rather than the drupal:7 image, so that we could use PHP 5.6 instead of PHP 7.0. I replicated the two additional things that the drupal:7 image was doing:
    • Installs the PHP extensions that Drupal needs.
    • Enables the Apache rewrite module.
  • Changed from mysql:5.7 to mysql:latest in docker-compose.yml, so that we don't need to manually update that if/when new versions come out.
  • Set up docker-compose.yml and Dockerfile to mount the entire farmOS codebase (/var/www/html) as a volume. See below for how this is managed automatically.
  • Removed the code from Dockerfile that built farmOS via drush make - this moved to the new docker-entrypoint.sh script (see below).
  • Added three environment variables, which are used in the docker-entrypoint.sh script (see below).
    • FARMOS_VERSION - the packaged version to download
    • FARMOS_DEV_BRANCH - the development branch to build from
    • FARMOS_DEV - whether or not to create a development environment or use the packaged version - this defaults to false
  • Added a docker-entrypoint.sh ENTRYPOINT script that is run every time that a container is created with docker run. This script does the following:
    • Creates the farmOS codebase in /var/www/html in one of two ways:
      • If FARMOS_DEV is true, it builds farmOS with drush make --working-copy
      • If FARMOS_DEV is false (default), it downloads and unpacks the official packaged release from drupal.org
    • Each time a container is spun up, the farmOS codebase in /var/www/html is rebuilt. The /var/www/html/sites directory, however, is preserved - so settings.php, uploaded files, and extra modules are not lost.

I also added documentation that describes all of that, along with instructions for use. It goes into a bit more detail than this comment, so be sure to read through that, and please test the instructions! http://farmos.org/development/docker/

All of this has been merged into the main 7.x-1.x branches. I've tested it a bunch locally, but let me know what you think, and if anything isn't working or doesn't make sense.

There are a few things that still need to be done, but we can do them as follow-up tasks I think:

  • #12
  • Currently emails can't be sent, and on install the following error shows: Unable to send e-mail. Contact the site administrator if the problem persists.
  • PECL Uploadprogress should be installed since all the file fields in farmOS use the uploadprogress widget.

@mstenta mstenta closed this Oct 18, 2016

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Oct 18, 2016

Member

Added followup issues:

#16

#17

Member

mstenta commented Oct 18, 2016

Added followup issues:

#16

#17

@mstenta

This comment has been minimized.

Show comment
Hide comment
@mstenta

mstenta Oct 22, 2016

Member

FYI, there have been some changes made to the way the docker-entrypoint.sh script works, to address issues discovered since this issue was closed. For more information, see the following:

#19

#20

It seems to be working well now.

I'm currently exploring this one: #21

Member

mstenta commented Oct 22, 2016

FYI, there have been some changes made to the way the docker-entrypoint.sh script works, to address issues discovered since this issue was closed. For more information, see the following:

#19

#20

It seems to be working well now.

I'm currently exploring this one: #21

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment