New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a base docker image for layer caching purposes #3593

Merged
merged 4 commits into from Jul 21, 2017

Conversation

Projects
None yet
2 participants
@nilbus
Member

nilbus commented Jul 20, 2017

This will drastically improve docker dev environment initial setup speed. Rather than building locally, the local docker build (built from Dockerfile.dev) is now based on the public, automated exercism/exercism.io build (built from Dockerfile). With this as a starting point, a developer's docker-compose build need only install any new gems that are introduced in the feature branch. With no local changes, a local docker build, excluding download time, now takes approximately 6 seconds, vs. 50 minutes before.

@kytrinyx: Before merging this, you will need to create an Automated Build on Docker Hub. The new Dockerfile.dev build depends on this. This will result in Docker Hub hosting and building our Dockerfile each time you push to master on GitHub, such that people can docker pull exercism/exercism.io.

  1. Visit https://hub.docker.com/r/exercism/.
  2. In the top navigation bar, select Create > Create Automated Build.
  3. Select GitHub.
  4. Select exercism/exercism.io.
  5. Select the exercism namespace if not already selected.
  6. Fill a Short Description. Suggested: "exercism.io application development base image"
  7. Optional: Limit branches to be built. Customize the branch matching behavior (Click here to customize). Leave the default custom behavior (build master only).
  8. Create the automated build.

Changes:

  • .dockerignore vendor/

    Vendored gems make the docker context directory size large (+1.1 G in vendor/), causing longer build times. Additionally, as individual developers, rather than being able to inherit most or all gems from an application base image, we are forced to install the full set of gems when first setting up a development environment.

  • .dockerignore node_modules/

    Reduces docker build context size. node_modules can get large and adds time to the context upload to the Docker engine. In our development environment, node_modules is mounted from the host with the rest of the application source and will always be present.

  • Dockerfile is now used by the automated build to build the exercism/exercism.io base image. See https://hub.docker.com/r/exercism/exercism.io/

  • docker-compose builds Dockerfile.dev, which is built FROM the public exercism/exercism.io docker image. This Dockerfile need only contain commands specific to the development environment and changes from the local git repository that need to be built. Presently, it checks for new gems to install.

  • Dockerfile: Set WORKDIR before RUN bundle install to invalidate one fewer layer when the bundle is updated.

Based on #3590 to avoid conflicts.

nilbus added some commits Jul 19, 2017

Upgrade docker-compose format to version 2.1
This enables the use of defaults with variable interpolation.

Followed procedure documented at
https://docs.docker.com/compose/compose-file/compose-versioning/#version-1-to-2x
Introduce a base docker image for layer caching purposes
This will drastically improve docker dev environment initial setup
speed. Rather than building locally, the local docker build (built from
Dockerfile.dev) is now based on the public, automated
exercism/exercism.io build (built from Dockerfile). With this as a
starting point, a developer's `docker-compose build` need only install
any new gems that are introduced in the feature branch. With no local
changes, a local docker build, excluding download time, now takes
approximately 6 seconds, vs. 50 minutes before.

Changes:

- .dockerignore vendor/

    Vendored gems make the docker context directory size large
    (+1.1 G in vendor/), causing longer build times. Additionally, as
    individual developers, rather than being able to inherit most or all
    gems from an application base image, we are forced to install the
    full set of gems when first setting up a development environment.

- Dockerfile is now used by the automated build to build the
  exercism/exercism.io base image. See
  https://hub.docker.com/r/exercism/exercism.io/

- docker-compose builds Dockerfile.dev, which is built FROM the public
  exercism/exercism.io docker image. This Dockerfile need only contain
  commands specific to the development environment and changes from the
  local git repository that need to be built. Presently, it checks for
  new gems to install.

- Dockerfile: Set WORKDIR before RUN bundle install to invalidate one
  fewer layer when the bundle is updated.
Reduce docker build context size by ignoring node_modules
node_modules can get large and adds time to the context upload to the
Docker engine. In our development environment, node_modules is mounted
from the host with the rest of the application source and will always be present.
Default to installing gems on system
There are good reasons for vendoring gems for production deployments,
but in my experience, vendoring in development has given my teams more
trouble than the default behavior installing gems on the system.

Existing development environments will continue to use vendor/ until
they modify/remove .bundle/config, because these are mounted with the
source code. This will now slow docker image builds, since .dockerignore
now includes vendor/ and .bundle/.

@nilbus nilbus referenced this pull request Jul 20, 2017

Merged

Setup script #2582

@kytrinyx kytrinyx merged commit f641519 into exercism:master Jul 21, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage remained the same at 86.524%
Details
@kytrinyx

This comment has been minimized.

Show comment
Hide comment
@kytrinyx

kytrinyx Jul 21, 2017

Member

@nilbus Thank you so much for the detailed, step-by-step walkthrough. That was the least painful setup on a new-to-me service that I've ever had.

Member

kytrinyx commented Jul 21, 2017

@nilbus Thank you so much for the detailed, step-by-step walkthrough. That was the least painful setup on a new-to-me service that I've ever had.

@nilbus nilbus deleted the nilbus:docker-dev-base-image branch Jul 25, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment