Why include mosquitto in Docker?

[Diaoul]

I wanted to run recorderd next to my Mosquitto instance but it seems they are both included in the docker image of recorderd.
Running multiple services in the same container is against Docker standards, the best way to run both is with a docker-compose configuration.

Can you provide a docker image without Mosquitto?

[jpmens]

Contributions accepted.

[xRmg]

I've created a docker setup without MQTT and without docker
https://github.com/xRmg/owntracks-recorder @Diaoul this should get you in the right direction.

Regards,

[MyIgel]

I think you should create a pull request ;)

A nice feature could be to create two images, a base recorderd image (only the recorder) and an extended recorderd-mosquitto (buldled with mosquitto and its config) or better an example docker compose file

[jpmens]

I will gladly repeat myself: we accept contributions and pull requests.

[MyIgel]

I think the base docker image should look like

FROM debian:stretch

ADD http://repo.owntracks.org/repo.owntracks.org.gpg.key /tmp/owntracks.gpg.key
RUN apt update &&\
    apt install -yqq gnupg && \
    apt-key add /tmp/owntracks.gpg.key && \
    rm /tmp/owntracks.gpg.key && \
    echo "deb  http://repo.owntracks.org/debian stretch main" > /etc/apt/sources.list.d/owntracks.list && \
    apt update && \
    apt install -yq ot-recorder && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

EXPOSE 8083

# For more options see https://github.com/owntracks/recorder#configuration-file
ENV OTR_HOST=mqtt
ENV OTR_PORT=1883

USER owntracks
CMD /usr/sbin/ot-recorder --http-host 0.0.0.0

[OmgImAlexis]

Here's a docker-compose.yml I whipped up today. Still a little rough though.

version: '3.4'

services:
    recorder:
        build: '.'
        networks:
            - default
        ports:
            - 11883:1883
            - 18883:8883
            - 8083:8083
        environment:
            - MQTTHOSTNAME: mqtt
            - IPLIST: '192.168.1.1 127.0.0.83 192.168.1.82'
            - HOSTLIST: 'foo.example.com bar.org.example.com ${cthostname}'
        deploy:
            replicas: 1
            restart_policy:
              condition: any

    mosquitto:
        image: eclipse-mosquitto
        networks:
            - default
        ports:
            - 1883:1883
            - 9001:9001
        volumes:
            - mosquitto-data:/mosquitto/data/
            - mosquitto-logs:/mosquitto/mosquitto.log
        configs:
            - mosquitto-conf:/mosquitto/config/mosquitto.conf
        deploy:
            replicas: 1
            restart_policy:
              condition: any

volumes:
    - mosquitto-data:
    - mosquitto-logs:
configs:
    - mosquitto-conf:

[deisi]

I started to work on this here:
https://github.com/deisi/docker-ot-recorder

My Idea is the following:

• A minimal docker image to run ot-recorder
  This is more or less already there, because I reused: https://github.com/kabili207/docker-ot-recorder
  and some parts of https://github.com/owntracks/recorder.
• Provide a set of docker-compose.yml files to help you run the whole thing with minimal effort.
  Started doing so with: https://github.com/deisi/docker-ot-recorder/blob/master/docker-compose.yml
  and https://github.com/deisi/docker-ot-recorder/blob/master/docker-compose-mqtt.yml
  although the mosquitto part is currently totally untested.

What needs to be done:

• Test mosquitto and find a useful default config
• Write some documentation
• Find a way to include encryption. Currently I think it mostly needs to be documented, because
  the eclipse-mosquitto container already brings interfaces to do so.
  The way it is done here, manly defeats the purpose of docker to begin with.

What I would like to know:

• Are there more people interested in the whole thing? I mean, I have my setup working by now, so this is for the fun and to give something back to the owntracks project. But there is no point in spending time if it will be ignored/rejected anyways.

[jpmens]

It will not be ignored or rejected. If it works well and is well documented, we will gladly take it.

[simonszu]

@deisi I am very much interested in the whole thing. I have a dedicated mosquitto running, mostly for smart home stuff, so the "official" image with the embedded mqtt broker is not suitable for me. I'll check your repo and try to get it running.

[deisi]

I updated the documentation a little. I would be glad if I get some feedback https://github.com/deisi/docker-ot-recorder

[jpmens]

@deisi for those of us (i.e. myself) who are not very proficient in things Docker, it would be good if you could elaborate a bit on the Storing data section.

1. where is the data then actually stored
2. can a host directory be passed as -v

[simonszu]

You cannot specifically say where the data is actually stored since Docker volumes are created somewhere in /var/lib/docker/volumes/<a quite long hash value> and then managed by docker only. If you want to extract data from a docker volume, there are subcommands like docker cp.

A host directory can very well passed as -v, just create it and pass it to the flag as an absolute path, beginning with /. However, host directorys can bring the pain of using proper filesystem permissions. If the process inside the container is running as root (which is not always a good idea), that's no problem, but since that's not a good idea, you have to poke around the permission settings, which can be avoided when volume handling is done by docker itself.

[deisi]

@jpmens
of course I could add some more words. However I would like to be rather minimal here since this is fairly standard docker stuff and more then extensively documented at https://docs.docker.com/storage/volumes/
But let me try to answer your question here, then we might find out what could be added.

To understand volumes, one has to know a bit about how docker works. The thing with docker is, that one creates an image. This image contains the program/package you want to run together with its dependencies. The image must not be changed after creation and this is made sure by using checksum.
Now, from this image you derive a container this container is what you actually run aka the programm if you want. People using something like docker run ... take an image and create a container from it. During this step you can add some configuration to your container, by using docker run -e blabla ... arguments.
This would be enough to configure your container. However you can't save any data within your container this way, because instead of stopping a container one usually deletes and recreates it.
To cope with this issue, you use a volume. A volume is basically a folder within your container that is mounted from the host machine into your container. This way it can be changed after container creation and doesn't conflict with the checksums. Also it will survive container recreation.
Does this help somewhat?

Now the reason why I didn't add much more to this section is that the user usually doesn't have to care. If you use the docker-compose.yml to create you container. docker-compose takes care of this volume handling for you. It will automatically create a volume called: docker-ot-recorder_storage for you and mount it into the container under /storage. Upon a restart the data is saved and persistent, just like you would expect.
If you remove this section form the docker-compose.yml file and run your container, again docker-compose will create a volume for you and mount it into /storage of the container. However this time, the volume will have a random unique uuid like name. Upon recreation of the container this process will be repeated but this time a new uuid will be assigned and mounted. Leading to a container that "lost" your data.

Sry for the many words.

[jpmens]

the user usually doesn't have to care.

until things go wrong :) You're right in that being how docker works, but I still think it's important to know what goes on under the hood. Both your explanations (@simonszu and @deisi) are good, and I'd like to see them merged together into, at least, an additional page explaning a bit of the "why you should do this in this particular way".

[jpmens]

As far as I'm concerned, you can submit a PR with your changes. Please separate (if they can be separated) the two installation methods in the README (ie. via docker commands and via docker-compose) and, if possible, add a sentence describing the utility of either method.

[deisi]

Okay, how should I do it? Should I make a PR for the owntracks/recorderd repository? From my point of view, this is almost a new project but I wouldn't mind. However the name recorderd is rather confusing in my opinion. I would suggest either: docker-recorder or even docker-ot-recorder.

I think docker-recorder is the bet option. It would allow us to have an image with the name owntracks/recorder on dockerhub and a repo with the name docker-recorder within the owntracks project of github.

[jpmens]

Please make the PR against this project. I will then asap rename this repository to docker-recorder, thereby keeping its history, issues, etc, even though it will, as you say, basically be something new.

[jpmens]

PR has been submitted in #20

[jpmens]

Thanks to all for your comments and to @deisi for the work. Closing this as I've now merged #20