[Question] comparison to iocage

[ngortheone]

Hi @cedwards
Can you briefly explain how bastille compares to iocage? (and to Docker for that matter)

Thanks!

[cedwards]

We haven't put together an official list, but this is as good a place as any to start one...

bastille vs iocage

bastille has zero dependencies beyond FreeBSD base.
iocage has hard requirements on ZFS filesystem and Python3.6+.

bastille install size (at 0.5.x) is still < 200K.
iocage install size (including depends) is over 300M. (roughly the size of extracted FreeBSD base.txz)

bastille vs docker

bastille implements lightweight OS containers
docker implements app containers
(I believe app containers offer a subset of OS container functionality while OS containers can do both)

bastille provides cron and logging in each container
docker makes both of these more complicated

[ngortheone]

So from what I see iocage does not provide the ability to script the creation of jails (aka dockerfile) while Bastille has the concept of templates.

Do you plan on building a registry, akin to docker hub, so users can share and re-use layers?

[cedwards]

We have a growing collection of pre-built templates on GitLab (https://gitlab.com/BastilleBSD-Templates) which are auto-CI/CD tested with GitLab-CI to ensure functionality.

Long term this collection will likely be consolidated into a curated collection of common / popular templates.

[ngortheone]

What about compiled versions of templates? Basically versioned zfs snapshots?

Usecase: A used builds a template that is based on another templated, which is also built on another template... (chain goes arbitrarily deep)

Building everything from scratch may take a while, re-building too.

Docker solves this problem with layers - think of versioned zfs snapshots, that can also be stored in some central location

[cedwards]

Bastille 0.6.x will have support to export (and import) compressed container archives for both ZFS and UFS backed containers. This makes similar functionality possible, but would need to be explored more at this point.

Let me stew on the implications and see what I come up with.

[zenny]

We have a growing collection of pre-built templates on GitLab (https://gitlab.com/BastilleBSD-Templates) which are auto-CI/CD tested with GitLab-CI to ensure functionality.

Long term this collection will likely be consolidated into a curated collection of common / popular templates.

@cedwards, happy to see your jail management script.

I love applications with small codebase with minimal dependency (that's what I advised to Mr. Toth prior to the new incarnation of his endeavor later known as iocage to be written in bash, fyi) as smaller and independent apps tend to be less buggy!

However, I just want to know from the horsemouth how much the information I read at https://jpmens.net/2019/12/11/on-building-an-ansible-training-environment-on-freebsd/ holds truth?!

The main reason for choosing iocage over BastilleBSD for jail creation is iocage’s templates which allow me to build a base jail containing all I need in it and then fire off creation of jails based on that template. (BastilleBSD also has templating – a bit like automation which is applied to the jail after it’s launched. This is practical, but for my purposes iocage templates are faster.)

Just curious!?

Cheers,
/z

[cedwards]

We're currently working on extending this functionality based on community feedback. Partial functionality exists now.

Previous to 0.6.x we only had the base templating system which would automate from zero-to-template, but no defined method of beginning from non-zero (like a pre-built image).

0.6.x added support to export and import containers. This now allows you to define the state using the templates, apply them to a container and then export that container. This export could then be used as a "base template" later.

The complete workflow is not finished but we've added a few more improvements toward this goal since the 0.6.20200202 release.

[zenny]

We're currently working on extending this functionality based on community feedback. Partial functionality exists now.

Previous to 0.6.x we only had the base templating system which would automate from zero-to-template, but no defined method of beginning from non-zero (like a pre-built image).

0.6.x added support to export and import containers. This now allows you to define the state using the templates, apply them to a container and then export that container. This export could then be used as a "base template" later.

The complete workflow is not finished but we've added a few more improvements toward this goal since the 0.6.20200202 release.

Thanks for sharing your roadmap. Thumbs up!

[ngortheone]

@cedwards thanks, I am satisfied with the provided information, although I am not sure whether I should close the issue, or maybe you want to keep it open as some sort of reference. Please feel free to close the issue.

[josejachuf]

So from what I see iocage does not provide the ability to script the creation of jails (aka dockerfile) while Bastille has the concept of templates.

Do you plan on building a registry, akin to docker hub, so users can share and re-use layers?

Hi I use both in production, but I've been using Iocage for a long time. I met Bastille a long time later. I like both of them.
With Iocage I never had problems. Iocage does have a template system that calls it plugins and the "templates" themselves are something else.
I like Bastille's null dependency, that it can be used with USF, I think the way to define the templates is simpler than in Iocage

[cedwards]

Included for reference is 10 Bastille containers created on Raspberry Pi 4.

As of version 0.8.20210101 Bastille is still <200K and 0 dependencies.

[davidchisnall]

I think the Docker comparison is missing the most important part. The OCI container ecosystem (of which Docker is the most well-known part) is huge. It defines container file formats and, with containerd, interfaces that large orchestration frameworks are built atop. I would love to see a FreeBSD-native container system that could build and deploy OCI containers and talk to something like Kubernetes. The lack of such a solution is the biggest reason that I've heard for people switching from FreeBSD to Linux over the last two years.

Is there any potential for providing compatibility with the various OCI bits on top of Bastille?

[josejachuf]

Hi @davidchisnall

I never tested or installed it, I don't know what state it is in, but this may be what you are looking for

https://papers.freebsd.org/2020/fosdem/pizzamig-orchestrating_jails_with_nomad_and_pot/

[davidchisnall]

Thanks @josejachuf. That seems to have a custom container format and so on. Unfortunately, there's a huge amount of investment in OCI-derived infrastructure across cloud providers (and, as a result, from other projects wanting to run in these environments) and so anything that doesn't live in this ecosystem is going to lose out.

[avatar4d]

Having pre-built images is the biggest gap for jails when comparing to Docker. I've had two services in the last couple of years fail on install/upgrade because port updates were committed since they built without error, but they unfortunately failed to run. The first had many bugs filed against it and was rectified six months later, the other is still open for over a year. I had to pin the pkg system to 12.1-RELEASE in order to install a working version at jail creation time with iocage plugins.

Bastille seems to be heading in the right direction on this front with the export/import idea. It would be great to see a repository of exported self-contained jails someday. Then we can just download/import a jail that just works rather than installing software on the fly at jail creation time and praying the pkg system is in a good state (not to mention do a bunch of configuration to get it running, such as when installing complex software like LibreNMS).

In the meantime, I am likely going to start playing with Docker since some of the things I want to run don't have ports yet, such as Home Assistant. Of course if I am going to have to go that route anyway, I might just end up moving all my apps over to Docker and just mount my FreeBSD NAS via NFS. I'm certainly hoping this project gains momentum though!

[patmaddox]

IMO the whole point of jails is to avoid the whole docker / kubernetes / OCI mess!

If you know, you know.