Rethinking Distro Packages

[callahad]

Preface

TL;DR: Unit must diverge from the practices that made NGINX successful.

Unit owes its existence to NGINX, but software has changed dramatically since 2004. What works for an established stalwart won't work for an upstart. Specifically:

1. Unit is not mainstream
   → Unit's future is with people and projects who are forward-looking and comfortable stepping off the beaten path.

2. Unit does not have critical mass
   → Maintaining the status quo is neither viable nor sufficient.

3. Containers won
   → Native packages are an increasingly rare way to distribute applications, and we should invest our effort accordingly.

With that in mind, I'd like to formalize and constrain Unit's scope so we can focus our attention where it is likely to have the greatest impact.

Status Quo

Unit's release process currently builds artifacts for:

• Docker
• .deb:
  • Ubuntu: 20.04 LTS, 22.04 LTS, 23.10, 24.04 LTS
  • Debian: 11, 12
• .rpm:
  • CentOS: 8, 9
  • Amazon Linux: 2, 2023
  • Fedora: 39

Proposal

I think we should have two tiers of support. We would actively remove support for distros that fall out of Tier 2.

Tier 1 (Fully Supported)

We will always build, test, and release packages for:

• Docker
• Latest Debian Stable
• Latest Ubuntu LTS
• Latest RHEL

Tier 2 (Best Effort)

We will strive to build and release for the following targets, but reserve the right to skip or drop them at any time:

• Previous Debian Stable
• Previous Ubuntu LTS
• Previous RHEL
• Latest non-LTS Ubuntu (only while newer than the latest LTS)
• Latest Fedora
• Latest Amazon Linux

Change

Immediately, we'd change our support matrix as such:

 Docker
 Debian 11 (Bullseye)
 Debian 12 (Bookworm)
-Ubuntu 20.04 LTS (Focal)
 Ubuntu 22.04 LTS (Jammy)
-Ubuntu 23.10 (Mantic)
 Ubuntu 24.04 LTS (Noble)
-CentOS 8
-CentOS 9
+RHEL 8
+RHEL 9
-Amazon Linux 2
 Amazon Linux 2023
-Fedora 39
+Fedora 40

Broken into Tiers, that'd be:

Tier 1:

• Docker
• Debian 12 (Bookworm)
• Ubuntu 24.04 LTS (Noble)
• RHEL 9

Tier 2:

• Debian 11 (Bullseye)
• Ubuntu 22.04 LTS (Jammy)
• RHEL 8
• Amazon Linux 2023
• Fedora 40

Questions

1. Does this approach make sense? Do we have the right number / types of tiers? (Maybe Docker should be Tier 0?)
2. Are there more distros we should cut? Or that we should add?

[callahad]

I don't want to get too into the weeds on specific entries in Tier 2 until we agree on the overall structure, but to give a sense of where my head's at, I think we should proactively drop distros from Tier 2 whenever they cause annoyance or when a successor has been out "long enough."

Specifically, here's my thinking on the current Tier 2 list:

• RHEL 8. Candidate for dropping. Over 5 years old, Full Support has ended, and its successor has been out for 2 years.

• Debian 11. Keep, but drop at first sign of friction. Nearly 3 years old, security support ends next month, successor has been out for over a year.

• Ubuntu 22.04 LTS. Keep. Successor has only been out a month, and this is the prior LTS release.

• Amazon Linux 2023. Keep. Newer than some Tier 1 distros, no signs of pain yet.

• Fedora 40. Keep. It's new, and the latest Fedora.

It's really important to keep containers in mind when thinking about this. Just because we stop publishing RPMs for RHEL 7 doesn't mean those users are left out in the cold. They always have the option of running a supported version of Unit inside a container.

[ac000]

I'm not sure it's a good idea to just stop providing packages for a currently active distribution, potentially leaving users with vulnerable software.

I suspect CentOS and RHEL are basically created from the same thing....

Ideally, none of this would be of any concern to us as distributions would package it up (of course some do...)

[andrey-zelenkov]

The FreeBSD status is not clear from this proposal.

[ac000]

How would / should FreeBSD fit in here? We don't produce binary packages for it,

Maybe we don't, but https://www.freshports.org/www/unit/

[callahad]

In #1332 I tried to say we'd make sure we build on FreeBSD, but I don't think it makes sense for us to ship first-party binary packages for FreeBSD the way we do for Debian and RHEL derived distros, hence FreeBSD not showing up in this post.

[andrey-zelenkov]

yeah, I didn't check that freebsd is in the Community Repositories so it's not part of this discussion. Thanks and please ignore.

[callahad]

I think we should say something about FreeBSD, otherwise people will wonder. Thanks for raising this comment as a reminder of that.

[ac000]

At the end of the day we tend to follow POSIX (there are some exceptions, like memmem(3), which is not in POSIX (or any C standard) but is available in a number of systems, incl; glibc, musl libc, the BSD's and macOS. A trivial alternate implementation can be easily added if ever required).

On top of what POSIX/C provides we also implement OS specific functionality for performance reasons, e.g epoll(7) on Linux and kqueue(2) on FreeBSD rather than poll(2).

Or also some OS specific features which are only available on specific systems, e.g namespaces(7) on Linux (other systems may provide similar functionality which could be implemented in future).

In theory, if your system follows POSIX and there is a port of GCC or Clang, Unit should build and run on it (or at least require very minimal changes)...

On the other hand if Unit only targeted Linux, you could end up ripping out many thousands of lines of code and could more easily use Linux only features... not that I'm suggesting we do that...

[ac000]

An obvious question is; "What are our users actually running Unit on?"

I think we need to understand that first...

[ac000]

So those numbers

Linux packages: 4162
Docker images: 12,040

are coming from a different source to what the graph is showing?

If those numbers are for a month, then if we divide the graph numbers (excl docker) by 3 (they seem to cover 90 days), then they still far outnumber docker.... (second graph, with the massively reduced numbers, comes out about 50K)

Also how many of those dockers are just temporary/throwaway/automated downloads...

[lcrilly]

I have already explained the source for all of the data points.

Please don't try to compare 'number of downloads' with 'unique IP addresses' - it will tie you up in knots :)

[ac000]

EDIT: Put this in the right place...

I have already explained the source for all of the data points.
Please don't try to compare 'number of downloads' with 'unique IP addresses' - it will tie you up in knots :)

So there's no way to do an apples to apples comparison of Linux packges vs Docker images?

I think your SQL could do with a couple of tweaks

Should probably discount the wasm language module packages as you do the rest...

AND request NOT LIKE '%-wasm-%'

and the

AND request NOT LIKE '%devel%'

should probably be like

AND request NOT LIKE '%dev%'

To catch Debian etc...

[lcrilly]

So there's no way to do an apples to apples comparison of Linux packges vs Docker images?

We absolutely can compare packages/images like apples.

Comparing the size of the audience can be done by using 'number of unique IP addresses accessing images/packages'. This is precisely the comparison I gave to show the 3x difference (in favour of Docker) in May. Or 4x if we look at April.

Comparing the 'number of downloads' is also possible but automation workflows for VM and container builds makes this a very difficult thing to draw conclusions from. You already called out the ad hoc use cases.

Let's get on a call to clean up the SQL and get the Grafana dashboard into a state where we can be confident in it.

[sboulkour]

Hello everyone ! On low container image numbers : we are using Unit + PHP on production, in containers. But we build our own. And as we're using a ubuntu base image, it is currently counted in Ubuntun packages numbers I suppose :)

[thresheek]

From the packaging point of view, supporting "old stable" distros like RHEL 8 / Debian 11 / Ubuntu 20.04 is essentially free. The biggest timesink is adding new distributions as they (as they should, really) typically break some things or require fine-tuning. When the distro is added, there is almost no overhead in supporting it.

I understand that there might be other reasons like a more modern version of C being utilized, though.

[callahad]

Makes sense. I think the more important idea around the tiers is: "if a Tier 1 distro causes packaging issues, we'll absolutely put in the engineering effort to keep it working, and we absolutely will not move to a more modern version of C, etc. than a Tier 1 distro can handle."

...whereas Tier 2 is more "if it causes maintenance burden, we'll give ourselves the option of choosing not to put in the work to fix it."

[javorszky]

I like the tiered systems. I think we should focus on Docker as a tier 0 as it seems to be the most popular way of folks using it, plus the easiest way to get more folks started with unit in an easy way.

Tier 1's "we'll spend engineering effort to make sure it's supported" makes sense.

Tier 2's "we'll drop it if effort exceeds benefit" also makes sense.

We should definitely come up with a standardised way to communicate how and when and in what timeframe will we move Tier 1 distros into Tier 2, like

"Ubuntu LTS 24 has just launched, which means that has entered Tier 1, and LTS 22 will move to Tier 2 in 3 month on <specific date>"