Add Dockerfile #1348

JanKoppe · 2017-04-21T21:37:15Z

This is a work in progress.

Adds a Dockerfile based on Alpine Linux 3.4.

JanKoppe · 2017-04-24T09:27:05Z

This seems to be working just fine. Can be merged imho!

strugee · 2017-04-29T17:27:41Z

@evanp do you want to do a review on this? You're more familiar with Docker than me

strugee

Went through and did a review since @evanp never got to this. Thanks for working on this! :)

strugee · 2017-05-09T07:08:25Z

docker/README.md

+```
+
+The Dockerfile also allows to define the UID and GUID of the `pumpio` user which runs pump.io via build arguments.
+```


Newline before this one

strugee · 2017-05-09T07:09:04Z

docker/README.md

+
+## Running
+
+By default this Dockerfile only includes the `databank-mongodb` driver. Therefore you will have to add the configuration parameters for the mongodb. The recommended way to configure pump.io is to configure the options via environment variables, as described in the projects README. You can see some example variables in the `docker-compose.yml` file. This file wil also show a basic working version which you can expand on.


s/the mongodb/MongoDB/

strugee · 2017-05-09T07:09:25Z

docker/README.md

+
+## Running
+
+By default this Dockerfile only includes the `databank-mongodb` driver. Therefore you will have to add the configuration parameters for the mongodb. The recommended way to configure pump.io is to configure the options via environment variables, as described in the projects README. You can see some example variables in the `docker-compose.yml` file. This file wil also show a basic working version which you can expand on.


s/projects/project's/

strugee · 2017-05-09T07:09:41Z

docker/README.md

+
+## Running
+
+By default this Dockerfile only includes the `databank-mongodb` driver. Therefore you will have to add the configuration parameters for the mongodb. The recommended way to configure pump.io is to configure the options via environment variables, as described in the projects README. You can see some example variables in the `docker-compose.yml` file. This file wil also show a basic working version which you can expand on.


Typo: wil should be will

strugee · 2017-05-09T07:09:58Z

docker/README.md

+By default this Dockerfile only includes the `databank-mongodb` driver. Therefore you will have to add the configuration parameters for the mongodb. The recommended way to configure pump.io is to configure the options via environment variables, as described in the projects README. You can see some example variables in the `docker-compose.yml` file. This file wil also show a basic working version which you can expand on.
+
+Running via `docker-swarm`:
+```


Newline before this one

strugee · 2017-05-09T07:20:12Z

docker/Dockerfile

+     && adduser -S -D -H -G "pumpio" -h "/usr/local/pumpio" -u "${PUMPIO__UID}" "pumpio" \
+     && chown -R "pumpio:pumpio" /usr/local/pumpio \
+     && mkdir -p /usr/local/bin \
+     && ln -s ${PUMP_LOCATION}/bin/pump /usr/local/bin/pumpio \


Probably this should link to pump instead of pumpio a) for consistency with the npm package and b) since you're calling pump down below

strugee · 2017-05-09T07:21:02Z

docker/Dockerfile

+     && npm install databank-mongodb \
+     && addgroup -S -g "${PUMPIO__GUID}" "pumpio" \
+     && adduser -S -D -H -G "pumpio" -h "/usr/local/pumpio" -u "${PUMPIO__UID}" "pumpio" \
+     && chown -R "pumpio:pumpio" /usr/local/pumpio \


Why is this in /usr/local? /var/local would probably be better

This is just so if you would ever want to drop into a shell inside the container your home would be the place that pump.io is located at. Just for convenience, though it does clash a bit with LFS.

wait, scratch that comment, thought you were referring to L21.

No, I was. Though I think I got it confused - /usr/local/pumpio is supposed to be the place where the source is, right? Not where datadir is? In that case it should be left as root:root. No reason to make it owned by the pump user and it increases attack surface.

strugee · 2017-05-09T07:21:45Z

docker/Dockerfile

+FROM alpine:3.5
+LABEL maintainer Jan Koppe <post@jankoppe.de>
+
+ARG PUMPIO__GUID=1337


Not sure that it makes a difference practically but these should probably be system users (<1000), though I do appreciate the choice of 1337 :)

Yes, it could be a system user. I don't really see if/how this would make any practical difference, but I don't have any strong opinion about this. Do you have a proposal for a new UID/GUID?

I intentionally made this a build arg, because sometimes it's more comfortable to change the UID/GUID for running in production (file permissions, etc)

@JanKoppe I wasn't suggesting it not be a build argument, just that the default should be a system user. I don't know if this makes any practical difference either, but it seems like we might as well play it safe :)

Maybe make it 42? Not sure if that's too low (i.e. already occupied); I don't think so though.

strugee · 2017-05-09T07:22:05Z

docker/Dockerfile

+ARG PUMPIO__GUID=1337
+ARG PUMPIO__UID=1337
+
+ENV PUMP_LOCATION="/usr/local/pumpio"


Probably /usr/local/lib/pumpio is more standard?

Thinking about this a bit more, it should probably be placed in /usr/lib/node_modules. Not entirely sure about this though.

I don't think so. Not if it's not installed directly via npm install -g.

strugee · 2017-05-09T07:22:49Z

docker/Dockerfile

+ARG PUMPIO__UID=1337
+
+ENV PUMP_LOCATION="/usr/local/pumpio"
+ENV PUMP_GIT_TAG="v4.0.0-beta.5"


Lop off the beta tag here since 4.0 final shipped since this was sent :)

Trololololol, I like how @aurbot-user 👍'd this ;)

JanKoppe · 2017-05-09T11:23:39Z

Hi,

thanks a lot for your review strugee! I addressed most of your points in the last commits. One thing to note is that in order to build from the local repository, the Dockerfile et. al. have to live in the root of the repository ( COPY statements with parent directories are not allowed).

The last open discussion point as far as I can see is the UID and GUID of the pumpio system account.

I would like to leave the UID and GUID configurable via a build argument, this is especially useful when running this in production with a host filesystem location mounted as a volume. This would allow you to easily change the file ownerships to a UID/GUID that suits your environment.

Using IDs < 1000 would be a "cleaner" way, yes. I don't see how the current ID hurts though, so we could just leave it at that. It's a bikeshedding point IMO, but if you have any suggestions for a ID < 1000 I will happily change it! :)

Please excuse the huge amount of stupid spelling mistakes and typos in the original PR, it was quite late when I wrote that (UTC+2) . :)

strugee · 2017-05-12T02:35:50Z

@JanKoppe thanks for updating this! Some points (some of which I made inline too, but ¯_(ツ)_/¯):

One thing to note is that in order to build from the local repository, the Dockerfile et. al. have to live in the root of the repository ( COPY statements with parent directories are not allowed).

Right. I personally wouldn't mind that. I dunno, maybe this isn't the norm in the Docker space (feel free to tell me if that's the case), but I personally feel like if we ship a Dockerfile in-repo I would expect it to build from what's on the local disk, in the same way that if a project ships e.g. a Makefile, I expect that invoking make will create a build from local source, not a fresh clone.

I would like to leave the UID and GUID configurable via a build argument, this is especially useful when running this in production with a host filesystem location mounted as a volume.

Definitely wasn't suggesting it shouldn't be a build argument, just that the default should be different.

Using IDs < 1000 would be a "cleaner" way, yes. I don't see how the current ID hurts though, so we could just leave it at that. It's a bikeshedding point IMO, but if you have any suggestions for a ID < 1000 I will happily change it! :)

I don't really feel all that strongly about it, but I figured we might as well play it safe :)

Perhaps 42 could serve as an equally amusing UID? :D

strugee · 2017-05-12T02:37:19Z

Also wanted to explicitly point out #1348 (comment) and #1348 (comment) since they're hidden in outdated diffs

JanKoppe · 2017-05-26T06:33:39Z

Hey!

took a while to get back on that. I've incorporated your suggestions and prepared the datadir for uploads, etc, which wasn't the case before.

Sadly 42 was not available, so I just picked a number to move forward.

strugee

Found a couple small things inline but otherwise this LGTM!

Last thing: if you could rebase on top of master, as well as adjusting your commit messages to match git style, that'd be awesome. Alternately if you're okay with me force-pushing to your branch, I can do this for you.

Thanks so much for doing this, and for your patience!

strugee · 2017-07-06T19:58:36Z

DOCKER.md

@@ -0,0 +1,44 @@
+# Dockerfile for pump.io
+
+This Dockerfile is based on Alpine Linux 3.5 and node 6.9. It build pump.io from the Git repository.


Capitalize "Node"

strugee · 2017-07-06T20:00:08Z

DOCKER.md

+
+## Running
+
+By default this Dockerfile only includes the `databank-mongodb` driver. Therefore you will have to add the configuration parameters for the MongoDB. The recommended way to configure pump.io is to configure the options via environment variables, as described in the project's README. You can see some example variables in the `docker-compose.yml` file. This file will also show a basic working version which you can expand on.


"MongoDB" instead of "the MongoDB"

strugee · 2017-07-06T20:03:17Z

Dockerfile

+
+RUN apk add --no-cache graphicsmagick openssl nodejs python make g++ git \
+     && cd "${PUMP_LOCATION}" \
+     && sed -i 's/"bcrypt": "0.8.x"/"bcrypt": "^1.0.2"/g' package.json \


This is no longer needed :)

strugee · 2017-08-08T15:48:29Z

@JanKoppe ping?

JanKoppe · 2017-08-08T20:47:08Z

Dang, this has completely slipped my mind. Will take a look at it tomorrow as it's getting quite late here!

JanKoppe · 2017-08-10T08:57:11Z

Hope this is how you want it. If you have any other change request, just force push to my branch, that's fine by me. :)

strugee

LGTM with a single nit I found inline and reworded commit messages. I'll just force-push these to your branch 👍

strugee · 2017-08-18T23:25:41Z

DOCKER.md

+
+## Building
+
+The image should build without any further options. The compile-error from node-gyp during the `npm install` can be ignored, as they only affect optional dependencies.


"compilation errors"

strugee · 2017-08-18T23:35:29Z

DOCKER.md

+
+## Running
+
+By default this Dockerfile only includes the `databank-mongodb` driver. Therefore you will have to add the configuration parameters for MongoDB. The recommended way to configure pump.io is to configure the options via environment variables, as described in the project's README. You can see some example variables in the `docker-compose.yml` file. This file will also show a basic working version which you can expand on.


It'd be nice if we supported passing in another driver, but I'll file a followup for this

This commit adds a basic WIP Dockerfile for pump.

Adds a missing 'npm run build' directive, without which the container was not usable.

Adds missing graphicsmagick and databank-mongo packages. Also adds a very basic docker-compose configuration.

Fixed typos and formatting, more consistent use of env variables.

With this commit the container uses the sources in the current working directory instead of cloning the sources from a remote repository each time. This speeds up builds and ensures that the Dockerfile always uses the currently checked out code.

This patch changes the UID and GUID for the pumpio system user into the system range (< 1000). The installation location for the pump.io source is moved to a local dir instead of the npm installation directory. The datadir directory is prepared and permissions are set so that uploads will work out of the box. For persistence users should use a volume for this location.

strugee · 2017-08-19T02:29:48Z

@JanKoppe thank you so much for working on this! I'll merge when Travis is green :)

JanKoppe changed the title ~~WIP: Add Dockerfile~~ Add Dockerfile Apr 24, 2017

strugee assigned evanp and JanKoppe Apr 29, 2017

strugee requested a review from evanp April 29, 2017 17:27

strugee requested changes May 9, 2017

View reviewed changes

strugee requested changes Jul 6, 2017

View reviewed changes

strugee approved these changes Aug 19, 2017

View reviewed changes

JanKoppe and others added 9 commits August 18, 2017 19:26

Add basic Dockerfile

35023e9

This commit adds a basic WIP Dockerfile for pump.

Add missing 'npm run build'

135cbdb

Adds a missing 'npm run build' directive, without which the container was not usable.

Add missing packages; create docker-compose config

ecef693

Adds missing graphicsmagick and databank-mongo packages. Also adds a very basic docker-compose configuration.

Change container user, add README, various fixes

b7f6e14

Fixed typos and formatting, more consistent use of env variables.

Change formatting and wording of Dockerfile README

99a7efd

Remove temporary bcrypt dependency fix from Dockerfile

d4610e0

Fix minor grammar problem

ac24d26

strugee merged commit ac24d26 into pump-io:master Aug 19, 2017

strugee mentioned this pull request Aug 19, 2017

Added Dockerfile for a Docker deployment #1307

Closed

Add Dockerfile #1348

Add Dockerfile #1348

JanKoppe commented Apr 21, 2017

JanKoppe commented Apr 24, 2017

strugee commented Apr 29, 2017

strugee left a comment

strugee May 9, 2017

strugee May 9, 2017

strugee May 9, 2017

strugee May 9, 2017

strugee May 9, 2017

strugee May 9, 2017

strugee May 9, 2017

JanKoppe May 9, 2017

JanKoppe May 9, 2017

strugee May 12, 2017

strugee May 9, 2017

JanKoppe May 9, 2017

strugee May 12, 2017

strugee May 9, 2017

JanKoppe May 9, 2017

strugee May 12, 2017

strugee May 9, 2017

strugee May 12, 2017

JanKoppe commented May 9, 2017

strugee commented May 12, 2017

strugee commented May 12, 2017

JanKoppe commented May 26, 2017

strugee left a comment

strugee Jul 6, 2017

strugee Jul 6, 2017

strugee Jul 6, 2017

strugee commented Aug 8, 2017

JanKoppe commented Aug 8, 2017

JanKoppe commented Aug 10, 2017

strugee left a comment

strugee Aug 18, 2017

strugee Aug 18, 2017

strugee commented Aug 19, 2017


		## Running

		By default this Dockerfile only includes the `databank-mongodb` driver. Therefore you will have to add the configuration parameters for the mongodb. The recommended way to configure pump.io is to configure the options via environment variables, as described in the projects README. You can see some example variables in the `docker-compose.yml` file. This file wil also show a basic working version which you can expand on.

		@@ -0,0 +1,44 @@
		# Dockerfile for pump.io

		This Dockerfile is based on Alpine Linux 3.5 and node 6.9. It build pump.io from the Git repository.


		## Building

		The image should build without any further options. The compile-error from node-gyp during the `npm install` can be ignored, as they only affect optional dependencies.

Add Dockerfile #1348

Add Dockerfile #1348

Conversation

JanKoppe commented Apr 21, 2017

JanKoppe commented Apr 24, 2017

strugee commented Apr 29, 2017

strugee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JanKoppe commented May 9, 2017

strugee commented May 12, 2017

strugee commented May 12, 2017

JanKoppe commented May 26, 2017

strugee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

strugee commented Aug 8, 2017

JanKoppe commented Aug 8, 2017

JanKoppe commented Aug 10, 2017

strugee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

strugee commented Aug 19, 2017