Feat/check storage driver

[mike-sul]

• Check the docker storage driver type during installation if possible.
• Check the docker storage driver in the fioup systemd service.

[mike-sul]

@doanac Initially, I considered enforcing the use of the overlay2 storage driver unless it was already specified in daemon.json. However, after exploring and analyzing various edge cases, I realized that it would be safer and less intrusive not to modify any configuration and to fail only when it is certain that the current setup is incorrect.

For example, it is quite difficult impossible to distinguish between scenarios where docker.io was just installed as a dependency during apt install fioup, and those where it was already installed but is not currently running.

[doanac]

this seems sane to me. let's get a debian person's opinion once you are ready

[basak-qcom]

What happens when docker.io isn't installed (yet), and someone runs apt install docker.io fioup or apt install fioup (in the latter case, the dependency on docker.io would make it equivalent to the former case)? Or, if someone runs apt install docker.io and then apt install fioup? If the default storage driver is no longer overlay2, won't that fail?

What happens if the Docker configuration is changed while the fioup service is running, or is that not possible?

[lool]

Why not check at fioup runtime instead, before doing whatever action that would be problematic?

Preventing package installs is usually hard to deal with for end-users, I'd recommend doing this inside your app instead.

[mike-sul]

What happens when docker.io isn't installed (yet), and someone runs apt install docker.io fioup or apt install fioup (in the latter case, the dependency on docker.io would make it equivalent to the former case)?

The preinst script won't check anything in such the case and just exit with 0. There is no reliable way to check the docker storage driver at this stage in such the case.

if someone runs apt install docker.io and then apt install fioup?

In such the case the preinst will check the storage driver type based on the output of docker info command since the docker daemon will be running after apt install docker.io.

If the default storage driver is no longer overlay2, won't that fail?
Yes, it will fail.

What happens if the Docker configuration is changed while the fioup service is running, or is that not possible?

It is possible, however it is very unlikely can happen on production devices/hosts as change of a docker storage type implies loosing all currently installed container images (unless some migration has been conducted before).

The goal of this preinst check is to do the best effort in early determination of the inappropriate docker configuration. As the next step, I plan to add similar checks in the fioup command itself.

[mike-sul]

Why not check at fioup runtime instead, before doing whatever action that would be problematic?

My plan is to do both.

At first, to do the best effort in determining of inappropriate docker configuration during installation of fioup. What is the point to install it at all if it cannot be used.

Secondly, do similar check in the fioup command itself since on dev hosts/devices people can play with the docker storage types. However, it is very unlikely to happen on production hosts/devices.

[lool]

Why not check at fioup runtime instead, before doing whatever action that would be problematic?

My plan is to do both.

At first, to do the best effort in determining of inappropriate docker configuration during installation of fioup. What is the point to install it at all if it cannot be used.

Secondly, do similar check in the fioup command itself since on dev hosts/devices people can play with the docker storage types. However, it is very unlikely to happen on production hosts/devices.

As a user, I personally prefer being able to install the program and it telling me "your current config doesn't allow this program to work" rather than getting a package installation error. Perhaps you can print a warning message instead of failing the installation.

[mike-sul]

Why not check at fioup runtime instead, before doing whatever action that would be problematic?

My plan is to do both.
At first, to do the best effort in determining of inappropriate docker configuration during installation of fioup. What is the point to install it at all if it cannot be used.
Secondly, do similar check in the fioup command itself since on dev hosts/devices people can play with the docker storage types. However, it is very unlikely to happen on production hosts/devices.

As a user, I personally prefer being able to install the program and it telling me "your current config doesn't allow this program to work" rather than getting a package installation error. Perhaps you can print a warning message instead of failing the installation.

The question is whether users would be willing to change their configuration after installing fioup, upon seeing the error message produced by the fioup command: “Your current configuration does not allow this program to work.”

This is not just another Docker configuration parameter — it defines the storage backend. Changing it means that all existing container images will no longer be visible or runnable after restarting dockerd. If changing the configuration is not an option, then it raises the question of what the point of installing fioup is in the first place.

In that sense, this behaves more like an installation-time dependency. Ideally, we should validate it during installation and fail if the requirement is not met.

That said, I think it’s acceptable to emit only a warning for now, since fioup is currently targeted at hobbyist use cases rather than production environments. I'll amend the PR accordingly.

[basak-qcom]

FWIW, this is an interesting interaction between configuration and packaging that we bump into every so often. Unfortunately they were never designed to interact like this, so it tends to never work very well :-(

[mike-sul]

Perhaps you can print a warning message instead of failing the installation.

@lool I've updated the PR accordingly.

[lool]

The question is whether users would be willing to change their configuration after installing fioup, upon seeing the error message produced by the fioup command: “Your current configuration does not allow this program to work.”

Yeah, so the scenario would be:

• I have setup this system
• I install fioup because I want to use it to manage containers, I haven't read docs etc. because I'm a real hacker
• I fire fioup and a friendly warning tells me my docker container storage is not a supported config (and BTW we mentioned that in the docs, but we love you anyway)

I have only now seen that you already skip the tests if docker is not present, so it was already possible to install fioup even if it can't be used to manage containers at all because docker is missing. So now we're consistent and letting you install it if you have docker installed and the backend is wrong.

This is not just another Docker configuration parameter — it defines the storage backend. Changing it means that all existing container images will no longer be visible or runnable after restarting dockerd. If changing the configuration is not an option, then it raises the question of what the point of installing fioup is in the first place.

...and that's ok in the end, right? People will try things and then question their life choices. I've installed Atlassian's ACLI yesterday only to realize I couldn't point it at our onprem instance (I'm a real hacker, I don't read docs). Then I removed it :)

In that sense, this behaves more like an installation-time dependency. Ideally, we should validate it during installation and fail if the requirement is not met.

I think that's only needed if you automate some system-level integration. For instance:

sudo apt istall fioup-auto-enroll
# ... installs fioup, then registers the system automaticallly to appear in the web UI

that I would expect to fail at package installation time if the system setup doesn't allow the desired setup to be achieved.

But for an interactive user-level command, I personally think it's user-friendlier to defer the checks.

As an end-user, the pattern I'm thinking about here are things like brew doctor, or kubeadm diagnose etc.

If you had a fioup healthcheck, perhaps you could call from postinst directly instead of duplicating the checks?

[lool]

I'm not qualified to review the other changes, but I think the debian/preinst logic to diagnose system state would be better in postinst, that would allow:

• calling fioup itself to do the tests
• relying on dependencies being installed (but I'm not sure if fioup depends on docker; that itself might be a bad idea since there are multiple docker deb sources available with different package names)

[mike-sul]

I think the debian/preinst logic to diagnose system state would be better in postinst

Good point on moving this to postinst — I agree that’s a better place for this kind of check, especially since we only print a warning and don’t interrupt installation.

calling fioup itself to do the tests

For now, I’d keep the checks local to the postints script rather than introducing a dedicated healthcheck command — the logic is still quite small. If it grows and/or we find more reuse cases, we can revisit and consolidate it later.

[mike-sul]

@lool I suppose we might want (or need) to enforce the overlay2 graph driver setting (as a default value) in the Debian image we ship on UNO Q when/if the default version of docker.io becomes >= v29.

[lool]

@lool I suppose we might want (or need) to enforce the overlay2 graph driver setting (as a default value) in the Debian image we ship on UNO Q when/if the default version of docker.io becomes >= v29.

We should probably discuss this a bit more; qcom-deb-images ambitions to track Debian closely, so if wants to use whatever Debian docker defaults are used if possible. Perhaps for QLI we need to diverge in some ways, but I don't think we integrate FIO in meta-qcom either. Then there are the Arduino images, which include the Arduino UX, IDE, demos, device onboarding logic etc.; if we want these to be FIO ready out of the box, it could make sense to make changes there.

Perhaps also worth researching how to cope with the new storage driver if that's an option.

[mike-sul]

We should probably discuss this a bit more; qcom-deb-images ambitions to track Debian closely, so if wants to use whatever Debian docker defaults are used if possible. Perhaps for QLI we need to diverge in some ways, but I don't think we integrate FIO in meta-qcom either. Then there are the Arduino images, which include the Arduino UX, IDE, demos, device onboarding logic etc.; if we want these to be FIO ready out of the box, it could make sense to make changes there.

I think this is primarily a product decision. @mike-scott FYI.

Making Arduino's Debian “fio-ready out of the box” (for UNO Q) means choosing which storage model we standardize on. Today, that’s a trade-off between the proven overlay2 (stable, and well understood) and the newer containerd-based image store (less battle-tested, with higher storage overhead (2x)).

Defaulting to the new storage bakend now may introduce unnecessary cost and risk for IoT/embedded users. At the same time, it’s likely we’ll need to adapt to it eventually (or build our own optimized solution).

My suggestion:

1. Keep Arduino Debian on overlay2 for now.
2. Evaluate the new storage backend separately before making it the default.

Perhaps also worth researching how to cope with the new storage driver if that's an option.

This is already in progress.

From our side, adapting fioup to the new storage backend is not a major issue — it already mostly works and would only require minor adjustments.

The bigger concerns are with the storage driver itself:

1. ~2× storage usage for container images
2. Limited validation/testing on resource-constrained devices compared to the current graph driver (we tested it quite well, and patched the problematic spots) (e.g. behavior on power loss during image load).

So technically it’s feasible for us to support it, but the main risks are around efficiency and robustness in embedded scenarios.

[basak-qcom]

Do we consider qcom-deb-images to be tied to what we or Arduino ship for the UNO Q, or could it be possible for production UNO Q images to use overlay2 against the defaults where qcom-deb-images doesn't?

[lool]

Do we consider qcom-deb-images to be tied to what we or Arduino ship for the UNO Q, or could it be possible for production UNO Q images to use overlay2 against the defaults where qcom-deb-images doesn't?

Yeah exactly, they are not tied, and could differ if that's the vision; it's Arduino's decision to make though