Proposal: Build a single docker container based on Ubuntu 16.04 as our recommend install

[Robbt]

The technical details can be discussed under #551 but this is more for weighing out the pros and cons and making a decision as to whether this theoretical solution would be the best to direct our effort at.

My thoughts in favor of it are the following:

• we could bundle fixes for any broken packages automatically such as Add PPA to ensure latest Silan for Ubuntu based installations #552 & Airtime Analyzer mp3 import fails on some Debian installs #380
• it would dramatically simplify the install process for most users (see these installer issues)
• we could avoid support requests caused by weird configuration issues related to unsupported distros Problem: Playout wont work if locale is missing #553
• we could make it easier to deploy fixes without breaking instances being ran in production Feature-request: Document best practices and/or provide upgrade scripts for running instances #623
• it could be treated as the goal for our beta release see Proposal: Define our goals and roadblocks to release a 3.0 beta #544
• it would probably be easy with a proxy to support multiple libretime instances on a single server see this
• it is the most logical and simplest way to deploy complex software like libretime that basically depends on a whole server to run

Potential reasons we might want to avoid it:

• docker might be intimidating to people unfamiliar with it (but so is the linux command line)
• we would need to do more work testing and write docs for how to deploy it in production
• it could make us dependent upon a 3rd party to deploy if we used docker hub (but we already depend upon lots of 3rd party package managers)
• we could accomplish the same ease of install with debian packages, flatpak or snaps w/o introducing another level of abstraction and complexity Support for Ubuntu snaps #390

Feedback and thoughts are welcome.

[frecuencialibre]

i'm in!

the only thing i'd add is the experience of another excellent project - Azuracast, which has recommended installation with docker since 2017.

[ned-kelly]

@Robbt - Probably worth reading my comments here: #552 (comment)

This is already in my personal backlog so happy for this one to be assigned to me...

In short to support docker properly there needs to be a few "types" of containers:

• Multiple containers (for larger setups who don't just want everything bundled in one image)
• Single "minimal" container/build.

I'd also be keen to experiment on using something like Alpine Linux over Ubuntu for the minimal build so there's less overhead/try to keep the layers smaller - The Current build with all the Docker Layers is around 1.5GB which is a lot to pull down if you're on ADSL.

[hairmare]

flatpak or snaps

Those are basically just another way to run the same kind of containers... I think flatpak will (or already does) support running OCI containers.

I'd also be keen to experiment on using something like Alpine Linux over Ubuntu

I think some parts of liquidsoap are hard to come by without a glibc (but I haven't tried yet). The new Debian-slim flavor might also be interesting here. Generally speaking I'm not sure why everyone shys to Ubuntu rather than their being more Debian installs as I've found those to be more reliable.

My CentOS share a common base layer without much more on top so pulling them is usually ok since you only pull the large base once and everything else is shared. I haven't found the time to write actual helm charts to run and deploy them in a useable fashion... When I last really looked into this I was still waiting for beta things like InitContainers in K8s to become stable for the way they are currently set up to work. If I pursue those containers further I'll going to write helm charts next. To run all of this you'll probably need quite a cluster if you want everything to be HA and fast.

Right now audio related container things is more of an R&D thing for me right now. I think it would be interesting to start testing the current icecast beta and aim at writing some kind of K8s operator to manage it at some point. This is the kind of infra that my containers are actually waiting for and so long all the things going on in that space that I've seen have been closed source.

[Robbt]

When the Ubuntu liquidsoap package fix gets into the debian-backport as @paddatrapper is working on that should make building off of Debian feasible. I'm also excited about the idea of providing Debian & Ubuntu packages to install LibreTime as an alternative to docker but I do think a single vanilla docker container would be a good idea.

[JohnnyC1951]

#1 Generally, if it works in Debian, it works in the equivalent Ubuntu. Have I missed something? I am often raiding the Debian repos for use with Ubuntu.
#2 Docker seems entirely reasonable
#3 Making Docker the ONLY supported option simplifies support.
#4 Noobs like a choice of one. Less brain-ache. (#metoo)

[paddatrapper]

Generally that holds true, however in this case there are slight version mismatches in the various Debian versions that 16.04 happened to get right

I definitely do not want to see Docker be the only supported option. While it is simple to setup, it means that anyone using a different containerisation system (LXC for example in my case) cannot use it. Supporting a dedicated host install means that there is flexibility and people with different setups can still use LT.

[JohnnyC1951]

? Being the officially supported version does not mean you can't use the GIT, Snap, Flatpack, Marmite or Paddington Marmalade.
It means: does it work in Docker? It does? Next..

[paddatrapper]

It also means, when an issue is raised - does this happen in Docker? No, closed. Supported isn't just installation, it also includes post install.

Also if someone is willing to do the work of creating a deb installation path (i.e. me), then we should think hard before deciding to drop support for that platform. It is the same as the install script. Does only supporting Docker mean that the script falls away? If so, then installing from git is very difficult and is practically a copy-paste exercise that requires you knowing where everything goes and what dependencies are required

[ned-kelly]

My CentOS share a common base layer without much more on top so pulling them is usually ok since you only pull the large base once and everything else is shared. I haven't found the time to write actual helm charts to run and deploy them in a useable fashion... When I last really looked into this I was still waiting for beta things like InitContainers in K8s to become stable for the way they are currently set up to work. If I pursue those containers further I'll going to write helm charts next. To run all of this you'll probably need quite a cluster if you want everything to be HA and fast.

Do you have this working with Minikube for local testing/dev or is it a bit too much - I'd be guessing the overhead is about the same as the Ubuntu docker build?

[JohnnyC1951]

@paddatrapper There is no reason why the docker install should not use Debian, Centos, Ubuntu (or Alpine?) in its container. The install script COULD be built into the docker file run script and have a small docker file suck in the rest of the install, like now or a big fat docker image with it already inside. These are dev team choices. Docker is only a container for a pre-configured system. You can easily ssh into the container and administer it, like any other system, and have the /srv/airtime/stor, /etc/airtime/ and postgres data files outside the container. Upgrading LT then becomes a simple job of swapping out the docker container or reinstalling it. All the persistent stuff can stay outside. This has already be done by other LT users, look in DockerHub. or https://github.com/HaJoHe/docker-libretime

[Robbt]

I'd suggest that since we are generally a welcoming community and patient with users that we can "officially" support anything that people are willing to field Q&A regarding. I think recommending docker for people new the project might work especially if we can fully document the entire process to setting up a station (ie inlcuding port forwarding etc).

I'm also totally cool with the idea of supporting deb packages as another install path if @paddatrapper wants to put the work in. Since everything is community ran at this point I don't think we need to have a strict interpretation of what we provide help with. But we might want to consider pushing support requests to the forum when they don't represent a bug.

[JohnnyC1951]

Just a note. In a plain vanilla host install, ie no apache or nginx on 80 or 443, it should be possible to run it right out of the box with no tweaks at all.

[paddatrapper]

@paddatrapper There is no reason why the docker install should not use Debian, Centos, Ubuntu (or Alpine?) in its container.

Yes, but there are other containerization platforms out there. For example - I want to be able to stand a LT instance up using LXC, as this is what my infrastructure is designed around. Thus using the install script on what effectively becomes a bare-metal install (inside the LXC container) works for this use-case.

I don't disagree that a Docker container is a good idea and something that we should push to people new to the project, but it should not be the only way we distribute LT. We should rather continue supporting the dedicated host install that we currently have and have Docker for those who just want to run quickly and simply.

[frecuencialibre]

I don't disagree that a Docker container is a good idea and something that we should push to people new to the project, but it should not be the only way we distribute LT. We should rather continue supporting the dedicated host install that we currently have and have Docker for those who just want to run quickly and simply.

well put.

[frecuencialibre]

hi all HAPPY NEW YEAR!!!

...well almost ;P greetings from the sao paulo, brazil airport.

given progress on several fronts, including @hairmare 's recent work towards modern php version compatibility, the new deb package, and all of @Robbt's product and issue queue work, i'm now wondering whether it would make sense to refine this proposal. basically i'd love for our single-container docker install to be as dead-simple as possible. just install the deb using the user's environment vars on a modern linux distro, set up content (db, music files) persistence, and that's it. no cron job, no fqdn fixes, nothing that shoudl rightfully be fixed within the main project ./install process. this seems to be the way to be as un-opinionated as possible regarding which containerization solution you choose. fixes would go into the dockerfile only as a last resort in order to provide bugfixes asap that are necessary for a beta.

feasible? even a good idea? @ned-kelly any update on when you see yourself having time to dig in here? take care folks! all the best for 2019!

[paddatrapper]

I think that is a very good idea. Then include the build in the release notes for each new version going forward. Between the distro packages, two Docker images and the install script, we've covered most use cases I think. Fixes and improvements should definitely be upstreamed as a first priority

[frecuencialibre]

hello again folks, I'm now thinking of biting the bullet and creating a single-container docker image myself. Not sure what's happened to @ned-kelly, probably work, but given his availability over the course of the last few months I've decided not to wait since I also want to test #670. So let's start the ball rolling:

Which distro? for me this choice is mostly about php versions and getting the liquidsoap and silan versions we need, any other major considerations that don't fall into personal preference/familiarity?

• ubuntu 18 lts: i see it shipping with silan (0.3.3-1) and liquidsoap (1.1.1-7.2ubuntu1). ubuntu is well known by many, but not libre, any other downsides?
• debian 10 buster: ships with silan (0.4.0-1+b1) and liquidsoap (1.3.3-2). this choice is the one with which i personally am most familiar, but i'm not qualified to determine whether using buster now is a good idea.
• Centos 7.6. Would silan and liquidsoap packages, and the php version in 7.6 meet our needs? Where can i find that info? i personally have only used centos a few times on servers for paid work.

thanks everyone!

[paddatrapper]

Debian Buster is pretty much stable, it goes into transition freeze on 12 Jan and soft freeze on 12 Feb. It shouldn't see much volatility any more (and any that exist will be fixed quickly). Buster should also have the latest version of liquidsoap once I finish packaging the dependencies. Just a note - silan is version 0.4.0-1, not 0.4.0-1+b1. Personally, I would suggest Buster, but that is also because I am pretty biased towards Debian :)

[frecuencialibre]

@Robbt and I discussed this briefly via slack and are also leaning towards Debian 10 given all of Kyle's contributions and the proximity of official release. we weren't sure what specific benefits the latest version of liquidsoap could bring us that do not already come in 1.1.1-7.2ubuntu1, but using the most recent version seems like a good general policy given #192 . php 7.2 v. 7.3 is another somewhat unknown, but choosing 7.3 seems reasonable. @hairmare sound ok?

see also https://github.com/LibreTime/libretime/wiki/Distro-Packages#support-matrix

[frecuencialibre]

@paddatrapper the ./install script doesn't yet support buster, and the .deb i saw available for download here appears to target xenial. is this something you'd be interested in adding? i've started testing with ubuntu 18 in the meantime.

[paddatrapper]

@frecuencialibre I'll add support for Buster in the installer as soon as #670 is merged and to the deb once alpha7 is released

[paddatrapper]

@frecuencialibre - I created #682 to track the installer issue. I would like to get it done before alpha7

[frecuencialibre]

now wondering based on @hairmare 's comment whether a single container repo is even the best way to achieve simplicity and prioritize fixes to upstream rather than dockerfile acrobatics

-----------------------------------------------------
              * Configuring PostgreSQL *              
-----------------------------------------------------
System has not been booted with systemd as init system (PID 1). Can't operate.

how complicated would it be to get postgresql working within a single container?

[Robbt]

I don't think the problem is using PostgresSQL per se but that evidently Docker doesn't have its own systemd running so basically it needs its own custom installation and initialization script. See this stack overflow post.
So the idea of creating a docker image that installs the debian file is probably not the best idea for creating the docker setup and we will need to create a custom install script for docker. It should be doable as a single docker instance but I don't have a ton of experience with docker.

[Robbt]

Personally I'm thinking that we might want to rethink this proposal considering the realization that docker discourages systemd or a single container with multiple processes. I think that if @paddatrapper is on-hand to continue work on the debian packages than picking Debian 10 and/or Ubuntu 18.04 as our official distributions and providing a single easy to install and upgrade package this might be the preferred route for us to go as the official install. This is contingent on #580 being solved which it seems like #670 is making progress on.

I also think that we could also support docker both the multi-container and a single container version and encourage people who want to give it a test run via docker with the single container.

[Robbt]

I've decided that this shouldn't be a blocker for 3-0 beta, it is a good goal but considering we aren't using docker exclusively internally it is probably not a requirement.

[Aphris-Karu]

I love the docker idea but I am heavy into docker so a little biased. lol (Personal docker swarm is 72 cores, 512G ram, and 9Tb or Drive space) I have a few projects on Docker Hub that I do the builds for.

I don't think the problem is using PostgresSQL per se but that evidently Docker doesn't have its own systemd running so basically it needs its own custom installation and initialization script. See this stack overflow post.

That is correct, that docker builds do not use systemd when starting the container. However it does not mean that you can not use it. There are two tricks when building a docker container.

1. The entry point can not exit.
2. log data should be sent to STDOUT/STDERR

So if you build the container with multiple services you can create an entry script that is

systemctl start postgresql
systemctl start rabbitmq
systemctl start icecast
/usr/sbin/httpd –DFOREGROUND

Thus it starts the services and the last one, in this case apache, is started in foreground so it does not exit.

As to the Distro discussion, I would pick the one that works best with the software. Docker container users do not make changes in the container because the changes are not persistent without a way to attach persistent storage. (a whole other discussion that I dont see in the thread) so best compatibility is the deciding factor.

Persistent storage.

My current container (which runs everything in a single container and was not built by me) uses the following persistent storage connections

/etc/airtime
/srv/airtime/stor
/srv/airtime/work
/var/lib/postgresql/10/main
/usr/local/lib/python2.7/dist-packages/airtime_playout-1.0-py2.7.egg/liquidsoap/

Which works but has a few issues. The most notable one being that /srv/airtime can not be a single volume mount and has to be two (stor/work) mounts. I've thought about redesigning it as that is not an optimal config and the user should have the option of doing a single mount. There are a lot of other reasons like the way they start stuff and make changes in the entry script that I would do different as well.

I think I have rambled enough. Im more than happy to help and answer any questions anyone has on Docker or Docker Swarm.

[zklosko]

So, I've been trying to make a LibreTime instance in Docker and it keeps giving me an error that it can't recognize the OS version, even if the container is running Ubuntu 18.04 or Debian 10.

[paddatrapper]

See #949 for a working Docker image

[zklosko]

Just want to throw in another idea for the install package: has anyone given thought to making a snap for Snapcraft.io? It says it can auto-build from our Github repo.

[paddatrapper]

@zklosko it was briefly discussed in libretime/libretime-debian-packaging#3