Attempting to create Dockerfile, running into issues

[blha303]

FROM ubuntu:14.04
MAINTAINER Steven Smith <stevensmith.ome@gmail.com>
RUN apt-get update && apt-get install -y git python-pbs wget
RUN git clone https://github.com/zulip/zulip /srv/zulip
WORKDIR /srv/zulip
RUN python provision.py
EXPOSE 9991

This installs all dependencies, but services (e.g rabbitmq) don't appear to start, so later parts of provision.py (i.e sh.configure_rabbitmq(**LOUD)) fail. Also emoji generation seems to be failing too, due to a missing dependency. I'm burned out from working on this, if anyone else can offer a suggestion it would be appreciated.

[mikew]

I've been trying to use the distributed .tar.gz file:

FROM ubuntu:14.04

ENV ZULIP_VERSION=1.3.2
RUN apt-get update \
    && apt-get -y install wget python-six \
    && wget --output-document /tmp/zulip.tar.gz https://www.zulip.com/dist/releases/zulip-server-${ZULIP_VERSION}.tar.gz \
    && cd /root \
    && tar zxvf /tmp/zulip.tar.gz \
    && rm /tmp/zulip.tar.gz \
    && mv zulip-server-${ZULIP_VERSION} zulip \
    && cd /root/zulip \
    && ./scripts/setup/install \
    && su zulip -c /home/zulip/deployments/current/scripts/setup/initialize-database

But the RabbitMQ server fails to start. Even running the commands in an ubuntu:14.04 container and manually trying /etc/init.d/rabbitmq-server start it just hangs there.

[timabbott]

Hmm, that sounds like a docker/rabbitmq compatibility issue; it might be worth seeing how docker packages for rabbitmq work.

The only things the Zulip install process does to configure rabbitmq that could cause it to not start is install puppet/zulip/files/rabbitmq/rabbitmq.config into the right place in /etc...

[f-f]

I've tried the tar.gz too, and run like @mikew into the RabbitMQ issue. Here's the Dockerfile up to now:

FROM phusion/baseimage

RUN apt-get update && apt-get upgrade -y && apt-get install wget python-six
RUN curl https://www.zulip.com/dist/releases/zulip-server-1.3.5.tar.gz | tar xvz
RUN mv zulip-server-1.3.5 /root/zulip
RUN cd /root/zulip && ./scripts/setup/install
COPY settings.py /etc/zulip/
RUN su zulip -c /home/zulip/deployments/current/scripts/setup/initialize-database

I'd also like to make more than one container out of this (e.g. one for postgres, one for redis, one for rabbitmq) - to end up with a nice docker-compose.yml file - but I feel it's a lot more work due to the complicated install/config process.

@timabbott it also would be great to have a latest.tar.gz to avoid worrying about release versions into the Dockerfile.

[timabbott]

I can look at this a bit more after breakfast but:

(1) latest.tar.gz should be easy to add.

(2) timabbott@f2248d2 is what I needed to do in order to get Travis CI (based on docker) installing a development type environment, which may be helpful for debugging (e.g. I did have to run a rabbitmq restart in the middle of the install process there)

[timabbott]

I created a zulip-latest.tar.gz symlink in https://www.zulip.com/dist/releases/. Enjoy!

[njuaplusplus]

Hi, I also want to use docker to deploy Zulip. But I don't know which ports should be exposed. Could you help me?

[blha303]

My Dockerfile above has EXPOSE 9991, that's the tcp port. Development server runs on 9991-9993.

[njuaplusplus]

@blha303 but what are the ports in production? 80 and 443?

[timabbott]

https://groups.google.com/d/msg/zulip-devel/d2f2eJANtv4/JTnLkvJWBgAJ answers that I think

[njuaplusplus]

@timabbott Yes, thank you.

[njuaplusplus]

@timabbott

Notice: /Stage[main]/Zulip::Nginx/File[/etc/nginx/zulip-include/proxy_longpolling]/ensure: defined content as '{md5}e32de15d9e2f10be88c07f26a6228675'
Notice: /Stage[main]/Zulip::App_frontend/File[/etc/nginx/zulip-include/app.d/]/ensure: created
Notice: /Stage[main]/Zulip::App_frontend/File[/etc/nginx/zulip-include/app]/ensure: defined content as '{md5}485b3b6a5f268439635d8abe46c76db6'
Notice: /Stage[main]/Zulip::App_frontend/File[/etc/nginx/zulip-include/upstreams]/ensure: defined content as '{md5}84fcb491b0cf30d21de0b029dc7afc1d'
Notice: /Stage[main]/Zulip::App_frontend/File[/etc/nginx/zulip-include/uploads.types]/ensure: defined content as '{md5}0d51d78a483570dc725581b6918d08ce'
Notice: /Stage[main]/Zulip::Nginx/Service[nginx]/ensure: ensure changed 'stopped' to 'running'
Notice: /Stage[main]/Zulip::Rabbit/Package[rabbitmq-server]/ensure: ensure changed 'purged' to 'present'
Notice: /Stage[main]/Zulip::Rabbit/File[/etc/cron.d/rabbitmq-queuesize]/ensure: defined content as '{md5}4273ccb225e37b8d04090187122806d1'
Notice: /Stage[main]/Zulip::Rabbit/File[/etc/rabbitmq/rabbitmq.config]/ensure: defined content as '{md5}2dd16948c9f36d8797732cb4c9e43bc8'
Notice: /Stage[main]/Zulip::Rabbit/File[/etc/default/rabbitmq-server]/content: content changed '{md5}7cc5216f193f6be1cb32c45021197b39' to '{md5}d0d028b94fbee9c1f4ff98e109cce3f4'
Notice: /Stage[main]/Zulip::Rabbit/File[/etc/cron.d/rabbitmq-numconsumers]/ensure: defined content as '{md5}dc5158f03148f188b3bc7b7418a4c16a'
Notice: /Stage[main]/Zulip::Rabbit/Exec[epmd]/returns: executed successfully

After call the /root/zulip/scripts/setup/install, it is stuck. The logs is as above.
Maybe it's the bug between rabbitmq and docker.
However, as @ff- mentioned, it would better to use separate docker containers.
And there are many official and unofficial dockerfiles:
FYI, such as:

1. RabbitMQ and its github repo
2. redis
3. postgresql

Though I want to make some efforts, but I am not quite good at reading the scripts.

[timabbott]

@njuaplusplus, to be clear, are you saying the install script is hanging? I've never heard of that happening before. Are you running in an environment with low resource (RAM, CPU, etc.) of some sort?

[njuaplusplus]

@timabbott Yes, the install script is hanging. It's my fault that I didn't make it clear. I ran the install script in the dockerfile to build an image. The resource is a bit low as I use Linode VPS with 4GB RAM and 4 Cores CPU.

Maybe this situation is same with @mikew as he said "manually trying /etc/init.d/rabbitmq-server start it just hangs there."

[timabbott]

That should be more than enough resources, so yeah I'd guess this is the same problem @mikew had​... Seems worth looking at what dockerfiles for RabbitMQ on Ubuntu Trusty do -- this can't be a problem unique to Zulip.

[chrisamoore]

#18 is this +1

[galexrt]

If anyone here is curious, it is possible to get Zulip working in a container. ;)

• In the settings.py there is no setting for changing the rabbitmq host, easy to fix by altering zerver/lib/queue.py a bit: pika.ConnectionParameters('localhost', to pika.ConnectionParameters(settings.RABBITMQ_HOST, and not forgetting to add the RABBITMQ_HOST to the settings.py.
• Some things that get installed using puppet, aren't needed in the docker image, you have to alter the zulip puppet manifests to get zulip working (see https://github.com/Galexrt/docker-zulip/tree/master/includes/zulip-puppet)
• Setup of database, rabbitmq and some other setup stuff, needs to be moved to an extra entrypoint "file" for docker, for example see my https://github.com/Galexrt/docker-zulip/blob/master/entrypoint.sh

A bit of self advertising of my Galexrt/docker-zulip repo, but I hope it helps someone here. :)
P.S. I know my entrypoint.sh is a bit messy, but it serves it purpose.

[mijime]

👍

[timabbott]

Cool thanks for posting your work @galexrt ! I'd be interested in getting a working sample Dockerfile into the main Zulip repository if we can figure out a good way to do it...

[galexrt]

@timabbott Where shall I put the Dockerfile?
Should I create a new branch for you in my repo or create a pull request with the Dockerfile and the needed files in it?

BTW since my first post here, I added a bunch of new features to the image, for example user creation(without the need of an smtp server) (Currently some features like log2zulip configuration, automatic backup of the database, backup creation and restoration are in the dev branch only, but should be ready by tomorrow to be pushed to the master branch).

P.S. I just submitted the Dropbox Contributor License Agreement, so I would be ready to create a pull request now.

[timabbott]

@galexrt I think putting it under a new top-level directory docker/ is reasonable but try to avoid using relative paths that would make moving it somewhere else later a pain.

A pull request with the needed files would be great; it'd be awesome if you could clean it up a bit to avoid creating duplicated content where we can (e.g. I notice in your repo you have a fork of the Zulip puppet config; ideally we'd figure out a solution that doesn't require updating things in 2 places whenever we make config changes...)

[galexrt]

@timabbott I had to fork them, because the zulip/zulip puppet manifests install a lot of unneeded stuff in the container.
If you mean by config changes, changes in the puppet/zulip/files folder, that wouldn't be a problem to fix. I would just use the downloaded files in the puppet folder instead of mine. But for the first pull request I would still use my files and later move to the original files if thats okay for you.

But I have to insist on using my modified manifests, because they are the ones that make up the smallness of the image.

[timabbott]

@galexrt OK that's reasonable; I think we will want to refactor so those unnecessary-in-docker items like NTP config are just optional in the config (or a separate optional manifest), but we can start with copying them in.

[galexrt]

@timabbott If it's okay, I'm going to create the pull request now.

Shall I include deployment files for kubernetes, docker-compose in the pull request?
They currently point to my quay.io/galexrt/zulip image.

[timabbott]

Sounds good!

Let's leave off the kubernetes/docker-compose stuff for now but I think once we get the main stuff merged it looks like it'd be easy for me to setup automated image building off of Zulip releases and point to that? I'm not super familiar with the docker ecosystem but it looks like quay.io is free for open source projects...

[galexrt]

@timabbott Yes, quay.io should be free for open source projects

Can I use Quay for free?
Yes! We offer unlimited storage and serving of public repositories. We strongly believe in the open source community and will do what we can to help!

From https://quay.io/plans

[galexrt]

@timabbott Sorry for the delay. I had to first finish my clean up (zulip/docker-zulip#31).
I now created the pull request, here #227.

Also you said that I should leave off the docker-compose stuff, I have added a basic docker-compose.yml, because you'll need more than just the zulip container for the image to run and the docker-compose.yml makes it easy to start all containers needed:

• database (modified postgres image with the tsearch_extras installed, quay.io/galexrt/zulip-postgresql-tsearchextras (without the tsearch_extras in the database image the zulip search fails)
• memcached
• rabbitmq
• camo
• redis

The docker-compose.yml I added, has only the required options set for an easy and fast start with the image. Yes it's still pointing to my quay.io repo. But you can easily change that in the docker-compose.yml.

[heitorlessa]

awesome! I've delayed my initial dev environment to start learning about Zulip's frontend and contribute thereafter as it's quite a pain to install it.

Will give it a try next week. Tks for that @galexrt