Common: publish each Kinesis app individually to DockerHub

[alexanderdean]

Depends on #2492

[danielzohar]

Is that for Docker containers? I'm happy to contribute the containers I've built previously which we use in production. I'm just in the process of building new, updated ones

[alexanderdean]

Thanks @danielzohar ! That would be great...

[danielzohar]

@alexanderdean my pleasure!
So we currently have the containers at:
https://github.com/Memrise/docker-snowplow-stream-elasticsearch-sink
https://github.com/Memrise/docker-snowplow-stream-s3-sink
https://github.com/Memrise/docker-snowplow-stream-collector

I'm just in the process of upgrading to 78 rc2 and moving to Linux Alpine which will make the containers really small (about 40mb + JAR size = ~100MB).

How should we go from there? Do you want to open official repos on github.com/snowplow/ or do you want it hosted on this repo?

[danielzohar]

@alexanderdean I created a new repo that contains collector + enrich + elasticsearch-sink at https://github.com/Memrise/docker-snowplow

How would you like to proceed from here? I assume you'd like to take ownership. If that's the case, would you like to create a new repo on your account and I can a PR with those files?

[alexanderdean]

Hey @danielzohar - first off thanks for doing all this, it's appreciated! I've caught up with @jbeemster and this is our plan:

• From R78 onwards we will support Docker for Snowplow
• We will roll in other projects over time too (e.g. Schema Guru Web, Iglu Server, Kinesis S3)
• Semantic versioning doesn't make sense with lots of orthogonal artifacts in the same repo, so we will cut the releases as Release 1, Release 2 etc

I hope that all makes sense - I've created a repo here: https://github.com/snowplow/docker-snowplow

PR very welcome!

[alexanderdean]

Just to let you know that R78 is now out @danielzohar

[danielzohar]

thanks @alexanderdean
I'll update and try to create a PR today.

As an aside, having separated JARs for each app would speed up build times and simplify the structure of the Dockerfiles a bit

[danielzohar]

Can't fork that repository. See snowplow/snowplow-docker#8

[alexanderdean]

Hey @danielzohar - the apps do have separated JARs. Do you mean separate artifacts on Bintray?

[danielzohar]

yes @alexanderdean, sorry

you can see here that I need to download the entire zip file - https://github.com/Memrise/docker-snowplow/blob/master/base/Dockerfile

then in each docker file, I keep only the relevant JAR and delete the reset

[alexanderdean]

Sure thing @danielzohar - bringing forwards: #1707

[alexanderdean]

This work is being done in a separate repo now: https://github.com/snowplow/snowplow-docker/