Add Dockerfile #66

jessfraz · 2015-04-28T19:26:09Z

Can be useful for testing everything compiles & building.

WARNING:
this does not work yet, but I am no jdbc/flyaway expert so idk

I figured I would open and if you want it you can help me fix, but if not just close this no worries :)

I think the problem is, the model pom.xml is trying to create a database then use it.

Failing in the container with:

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Keywhiz (Parent) ................................... SUCCESS [  7.557 s]
[INFO] Keywhiz Testing .................................... SUCCESS [  8.528 s]
[INFO] Keywhiz API ........................................ SUCCESS [  2.565 s]
[INFO] Keywhiz Client ..................................... SUCCESS [  0.443 s]
[INFO] Keywhiz CLI ........................................ SUCCESS [  6.017 s]
[INFO] Keywhiz HKDF ....................................... SUCCESS [  0.443 s]
[INFO] Keywhiz Model ...................................... FAILURE [  1.460 s]
[INFO] Keywhiz Server ..................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.591 s
[INFO] Finished at: 2015-04-28T19:21:04+00:00
[INFO] Final Memory: 41M/707M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:3.2:migrate (default) on project keywhiz-model: org.flywaydb.core.api.FlywayException: Unable to obtain Jdbc connection from DataSource (jdbc:postgresql:keywhizdb_test) for user 'root': Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. -> [Help 1]
[ERROR]

Let me know if you want the full stack trace. But it's probably just some java thing I don't know.

jessfraz · 2015-04-28T19:32:04Z

btw, I know it is because postgres is not actually running in the container, but I was wondering if there was a way to skip that test just to build
if not and its super necessary then I don't think this will work this way so feel free to close

barnumbirr · 2015-04-28T19:44:28Z

Hi there,

@jfrazelle don't you think it'd be best to keep the postgresql part separate (i.e in a second container) and --linkthe two together?

Cheers.

jessfraz · 2015-04-28T19:54:54Z

Yes but, I cannot do that on build :(

Also I put that specifically as a comment in the Dockerfile fyi

On Tue, Apr 28, 2015 at 12:44 PM, Martin Simon notifications@github.com
wrote:

Hi there,

@jfrazelle https://github.com/jfrazelle don't you think it'd be best to
keep the postgresql part separate (i.e in a second container) and --linkthe
to together?

Cheers.

—
Reply to this email directly or view it on GitHub
#66 (comment).

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

barnumbirr · 2015-04-28T20:02:09Z

True that...
I'd probably go with the mvn install part in a run.sh file you could call with CMD.
If you want to keep everything in one Dockerfile you could add a step (code is untested):

RUN \
    service postgresql start && \
    su - postgres -c "psql createdb keywhizdb_development && psql createuser keywhiz"

Cheers.

jessfraz · 2015-04-28T20:06:24Z

there is no way with maven to just ignore the test about creating the db?

On Tue, Apr 28, 2015 at 1:02 PM, Martin Simon notifications@github.com
wrote:

True that...
I'd probably go with the mvn install part in a run.sh file you could call
with CMD
If you want to keep everything in one Dockerfile you could add a step
(code is untested):

RUN
service postgresql start &&
su - postgres -c "psql createdb keywhizdb_development && psql createuser keywhiz"

Cheers.

—
Reply to this email directly or view it on GitHub
#66 (comment).

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

barnumbirr · 2015-04-28T20:11:13Z

I'm no java expert (quite the opposite actually, total noob inside 😄 ) but I think you can do something like:

mvn install -Dmaven.test.skip=true

Cheers.

alokmenghrajani · 2015-04-29T05:02:47Z

We can do one of two things: make sure that -Dmaven.test.skip=true (or -DskipTests) works, or run all the tests using h2 (right now, it's a combination of h2 and postgres). H2 is an embedded SQL engine. I am leaning towards using h2 for all development purpose. It would imply one less thing people have to care about.

jessfraz · 2015-04-29T05:42:41Z

Ah cool so there is no way to skip just that one test I guess because I
would definitely hate to skip all

On Tuesday, April 28, 2015, Alok Menghrajani notifications@github.com
wrote:

We can do one of two things: make sure that -Dmaven.test.skip=true (or
-DskipTests) works, or run all the tests using h2 (right now, it's a
combination of h2 and postgres). H2 is an embedded SQL engine. I am leaning
towards using h2 for all development purpose. It would imply one less thing
people have to care about.

—
Reply to this email directly or view it on GitHub
#66 (comment).

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

mbentley · 2015-05-01T18:17:41Z

@jfrazelle - I have a branch that is able to build successfully by setting up the pg database https://github.com/mbentley/keywhiz/tree/add-dockerfile-hack

It is super hackish right now but I'm hoping to get it cleaned up quite a bit to make it not quite so terrible

jessfraz · 2015-05-01T19:52:17Z

Does it require running pg in the container w the tests, so it's a
container w multiple processes because you could do that but I might just
turn a blind eye then

On Friday, May 1, 2015, Matt Bentley notifications@github.com wrote:

@jfrazelle https://github.com/jfrazelle - I have a branch that is able
to build successfully by setting up the pg database
https://github.com/mbentley/keywhiz/tree/add-dockerfile-hack

It is super hackish right now but I'm hoping to get it cleaned up quite a
bit to make it not quite so terrible

—
Reply to this email directly or view it on GitHub
#66 (comment).

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

diogomonica · 2015-05-01T20:29:42Z

@mbentley

rm: invalid option -- 'p'
Try 'rm --help' for more information.
INFO[0019] The command [/bin/sh -c apt-get update && apt-get install -y     postgresql     --no-install-recommends     && rm -rf /var/lib/apt/lists/* RUN mkdir -p /usr/src/app] returned a non-zero code: 1

mbentley · 2015-05-01T20:35:25Z

@jfrazelle - At build time, yes it runs pg in the background and killing it once it is complete. It should also probably clean up a bit of junk after it does that but I was hacking on it this morning. Totally not ideal, hench the -hack in the branch name 😄

@diogomonica - Sorry, I just rebased and squashed my commits so it is fixed now if you pull. It broke from a missed \ when I was cleaning things up.

diogomonica · 2015-05-05T22:57:46Z

@sul3n3t @alokmenghrajani thoughts? We would love to have a clean solution for people to setup/run keywhiz server quickly.

Porting the tests to H2 seems like it removes the major hurdle (having to bundle PG in the container).

alokmenghrajani · 2015-05-05T23:37:17Z

Running everything under H2 by default is nice in many ways but I won't be able to get to it right away. I'm ok merging this PR for now if everyone agrees it's a step forward?

diogomonica · 2015-05-08T22:30:59Z

@alokmenghrajani not yet.

@mbentley's Docker file works for building Keywhiz. Right now it doesn't run keywhiz server because it fails to connect to Postgres.

mbentley · 2015-05-09T12:22:10Z

So I did this and I cringed pretty hard but it is something to start from:

Start postgres:
docker run -d --name postgres -e POSTGRES_USER=root postgres

Start keywhiz (you need to build this image first):
docker run -it --rm --link postgres:postgres -p 4444:4444 -p 8085:8085 mbentley/keywhiz bash

Once in the container, run the following:

sed -i "s#localhost/keywhizdb_development#${POSTGRES_PORT_5432_TCP_ADDR}/root#g" /usr/src/app/server/src/main/resources/keywhiz-development.yaml
sed -i "s#127.0.0.1/keywhizdb_development#${POSTGRES_PORT_5432_TCP_ADDR}/root#g" /usr/src/app/server/src/main/resources/keywhiz-development.yaml
java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar preview-migrate server/src/main/resources/keywhiz-development.yaml
java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar migrate server/src/main/resources/keywhiz-development.yaml
java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar server server/src/main/resources/keywhiz-development.yaml

calavera · 2015-05-20T19:04:18Z

It looks like 6556985 broke the docker build because failing to connect to the database is an error in the build process 😿

jessfraz · 2015-05-20T19:05:23Z

it was never working in the first place ;)

On Wed, May 20, 2015 at 12:04 PM, David Calavera notifications@github.com
wrote:

It looks like 6556985
6556985
broke the docker build because failing to connect to the database is an
error in the build process [image: 😿]

—
Reply to this email directly or view it on GitHub
#66 (comment).

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

jessfraz · 2015-05-20T19:06:20Z

it's against my morals to start a db in the build container....

On Wed, May 20, 2015 at 12:04 PM, Jessica Frazelle me@jessfraz.com wrote:

it was never working in the first place ;)

On Wed, May 20, 2015 at 12:04 PM, David Calavera <notifications@github.com

wrote:

It looks like 6556985
6556985
broke the docker build because failing to connect to the database is an
error in the build process [image: 😿]

—
Reply to this email directly or view it on GitHub
#66 (comment).

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu
http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

sul3n3t · 2015-05-20T19:09:34Z

master is a little influx between releases. We're trying to make postgres, mysql, and H2 all work. H2 would be the default test DB. It's embedded and in-memory, so that will remove the concern about starting a DB for build/test.

calavera · 2015-05-20T19:10:00Z

cool, good to know. I don't mind waiting.

alokmenghrajani · 2015-06-03T18:14:56Z

#92 was merged yesterday and might help you?

jessfraz · 2015-06-03T18:23:44Z

hell ya, let me update ;)

alokmenghrajani · 2015-06-04T23:15:20Z

Dockerfile

+ADD . /usr/src/app
+
+# install
+RUN mvn install


don't you want mvn install -P h2?

updated my b, i am no maven extraordinaire

On Thu, Jun 4, 2015 at 4:15 PM, Alok Menghrajani
notifications@github.com wrote:

In Dockerfile:

+# docker build --rm --force-rm -t square/keywhiz .
+#
+# Example usage:
+# docker run square/keywhiz java -jar
server/target/keywhiz-server-*-SNAPSHOT-shaded.jar server
server/src/main/resources/keywhiz-development.yaml
+#
+FROM maven:3.3-jdk-8
+
+# mkdir for app
+RUN mkdir -p /usr/src/app
+WORKDIR /usr/src/app
+
+# add the source
+ADD . /usr/src/app
+
+# install
+RUN mvn install

don't you want mvn install -P h2?

—
Reply to this email directly or view it on GitHub.

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu

Can be useful for testing everything compiles & building. Signed-off-by: Jessica Frazelle <princess@docker.com>

alokmenghrajani · 2015-06-05T15:42:46Z

@diogomonica time to merge this?

jessfraz · 2015-06-05T16:27:25Z

you can trust me :D

Proof it works:

.... bunch of logs blah.....
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Keywhiz (Parent) ................................... SUCCESS [ 17.628 s]
[INFO] Keywhiz Testing .................................... SUCCESS [ 18.352 s]
[INFO] Keywhiz API ........................................ SUCCESS [  3.973 s]
[INFO] Keywhiz Client ..................................... SUCCESS [  0.738 s]
[INFO] Keywhiz CLI ........................................ SUCCESS [  9.546 s]
[INFO] Keywhiz HKDF ....................................... SUCCESS [  0.644 s]
[INFO] Keywhiz Model ...................................... SUCCESS [  6.315 s]
[INFO] Keywhiz Server ..................................... SUCCESS [01:19 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:17 min
[INFO] Finished at: 2015-06-05T16:27:01+00:00
[INFO] Final Memory: 159M/704M
[INFO] ------------------------------------------------------------------------
 ---> 423a9acee051
Removing intermediate container 76999042827c
Successfully built 423a9acee051

alokmenghrajani · 2015-06-05T16:30:27Z

Sounds good, thanks for doing this! I just wanted to make sure we weren't waiting on anything else.

Add Dockerfile

alokmenghrajani mentioned this pull request Apr 30, 2015

Dockerize Keywhiz server #69

Closed

alokmenghrajani reviewed Jun 4, 2015
View reviewed changes

Add Dockerfile

da5b033

Can be useful for testing everything compiles & building. Signed-off-by: Jessica Frazelle <princess@docker.com>

alokmenghrajani added a commit that referenced this pull request Jun 5, 2015

Merge pull request #66 from jfrazelle/add-dockerfile

9a5023d

Add Dockerfile

alokmenghrajani merged commit 9a5023d into square:master Jun 5, 2015

jessfraz deleted the add-dockerfile branch June 5, 2015 16:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Dockerfile #66

Add Dockerfile #66

jessfraz commented Apr 28, 2015

jessfraz commented Apr 28, 2015

barnumbirr commented Apr 28, 2015

jessfraz commented Apr 28, 2015

barnumbirr commented Apr 28, 2015

jessfraz commented Apr 28, 2015

barnumbirr commented Apr 28, 2015

alokmenghrajani commented Apr 29, 2015

jessfraz commented Apr 29, 2015

mbentley commented May 1, 2015

jessfraz commented May 1, 2015

diogomonica commented May 1, 2015

mbentley commented May 1, 2015

diogomonica commented May 5, 2015

alokmenghrajani commented May 5, 2015

diogomonica commented May 8, 2015

mbentley commented May 9, 2015

calavera commented May 20, 2015

jessfraz commented May 20, 2015

jessfraz commented May 20, 2015

sul3n3t commented May 20, 2015

calavera commented May 20, 2015

alokmenghrajani commented Jun 3, 2015

jessfraz commented Jun 3, 2015

alokmenghrajani Jun 4, 2015

jessfraz Jun 4, 2015

alokmenghrajani commented Jun 5, 2015

jessfraz commented Jun 5, 2015

alokmenghrajani commented Jun 5, 2015

Add Dockerfile #66

Add Dockerfile #66

Conversation

jessfraz commented Apr 28, 2015

jessfraz commented Apr 28, 2015

barnumbirr commented Apr 28, 2015

jessfraz commented Apr 28, 2015

barnumbirr commented Apr 28, 2015

jessfraz commented Apr 28, 2015

barnumbirr commented Apr 28, 2015

alokmenghrajani commented Apr 29, 2015

jessfraz commented Apr 29, 2015

mbentley commented May 1, 2015

jessfraz commented May 1, 2015

diogomonica commented May 1, 2015

mbentley commented May 1, 2015

diogomonica commented May 5, 2015

alokmenghrajani commented May 5, 2015

diogomonica commented May 8, 2015

mbentley commented May 9, 2015

calavera commented May 20, 2015

jessfraz commented May 20, 2015

jessfraz commented May 20, 2015

sul3n3t commented May 20, 2015

calavera commented May 20, 2015

alokmenghrajani commented Jun 3, 2015

jessfraz commented Jun 3, 2015

alokmenghrajani Jun 4, 2015

Choose a reason for hiding this comment

jessfraz Jun 4, 2015

Choose a reason for hiding this comment

alokmenghrajani commented Jun 5, 2015

jessfraz commented Jun 5, 2015

alokmenghrajani commented Jun 5, 2015