meson: create enablement symlinks in /etc during installation for cross builds

[jameshilliard]

We need to restore the previous behavior for cross builds since we often won't have access to a systemctl that can be executed on the build system.

This should hopefully resolve #12767.

[floppym]

I don't think it makes sense to couple this to cross compiling. If it really must be restored, maybe add a separate meson option for it.

[jameshilliard]

Yeah, I'll do that.

[jameshilliard]

I've added a link-units option.

[keszybz]

I don't like this. I understand what why you want to do this, and why it makes sense in your circumstances, but I don't think the overall tradeoff is worth it for the project. Essentially, it comes down to re-implementing parts of the functionality of tools in the build system, because your host system is too old and too limited. That's something that should be solved at your end. I think building a second copy of systemd (with everything optional disabled) is an acceptable solution. -Dauto_features=disabled makes this pretty easy.

There are other steps which need to run in the installed image: journalctl --update-catalog, systemd-hwdb update, machine-id creation, etc. So it's not just systemctl that needs to be called. Overall, we're better off making the effort to use our "real" tooling to do this post-installation setup, rather than adding ad-hoc partial reimplementations.

And if calling the tools is not possible, we do preset-all at boot if /etc is empty. That's another solution to this issue.

[jameshilliard]

Essentially, it comes down to re-implementing parts of the functionality of tools in the build system, because your host system is too old and too limited.

No, that's not the main reason, build systems like buildroot build everything from source including the toolchain itself, they intentionally do not rely on the toolchain that comes with the host OS for anything other than initial bootstrapping of the buildroot cross compilation toolchain to ensure reproducibility/consistency, having systemctl be a required part of the buildroot cross compilation toolchain significantly increases cross compliation complexity and points of failure.

That's something that should be solved at your end.

The only way buildroot could reasonably handle this is to build systemd twice which is not ideal, there's been a few other projects where this type of issue has come up and it's almost always a huge pain to deal with.

There are other steps which need to run in the installed image: journalctl --update-catalog, systemd-hwdb update, machine-id creation, etc.

Aren't these typically done at runtime on the target system?

And if calling the tools is not possible, we do preset-all at boot if /etc is empty. That's another solution to this issue.

That's not really a viable option IMO, it wouldn't work for read only rootfs builds from what I can tell.

[jktjkt]

Essentially, it comes down to re-implementing parts of the functionality of tools in the build system, because your host system is too old and too limited.

It is really just about not calling native code (built for target) on the build machine. An outdated host system is not an issue (for example, my Buildroot configuration uses GCC 8.3).

The other tools you mention (machine-id(5) setup, for example) run on the target once it boots. That's perfectly OK.

The original commit mentions concerns about not overwriting end users' configuration. Is there a reason why these enabling symlinks go to /etc instead of /lib/systemd/system/multi-user.target.wants/? That way, users would be free to keep their configuration in /etc while the vendor presets in /lib would remain, well, static vendor presets.

I think building a second copy of systemd (with everything optional disabled) is an acceptable solution. -Dauto_features=disabled makes this pretty easy.

It's doable, and Buildroot comes with excellent support for this. However, it has costs (time and energy in repeated CI builds and a slower turnaround), and if this is only about installing a couple of symlinks, I think it makes sense to look into alternatives.

[mbiebl]

Having a second way of creating those symlinks, means that it's yet another option which is likely not tested well and bound to bitrot.
If you can't run preset-all in your cross-build during deployment time, couldn't you just create the enablement symlinks manually? After all, you probably want control over what services are active by default anyway.

[yuwata]

I do not like this. Even if you manually create the simlink by ln, then the command comes from the host system. Why you do not want to use the host's systemctl. Though I may not catch your point correctly...

[jameshilliard]

Why you do not want to use the host's systemctl.

Since you have no guarantees a particular host even has a useable systemctl you would have to build systemd twice. This adds quite a bit of unnecessary complexity to the build process.

[yuwata]

My question is why you believe host's ln, but not systemctl.

[jktjkt]

As an alternative, I just submitted a PR which moves the enablement symlinks below lib/systemd/system/*.target.wants/. That way, there's no additional code path, and users can still make their preference and mask these services (or choose not to build them in the first place, as usual).

[jameshilliard]

My question is why you believe host's ln, but not systemctl.

Buildroot actually carries a patch to deal with ln issues already. All build hosts should have at least a basic ln, however with systemctl that is not the case.

[boucman]

There are all sorts of cases similar to systemd in buildroot, these are how they are usually dealt with

• never rely on the host version of the tool (risk of incompatible version)
• cross-build the tools and cross install them in a dedicates directory/sysroot

So, there is nothing special with systemd here, and building the host systemd, properly installed seems the way to go. There are a couple of things that can be done to help the cross-generation use-case, though

• Avoiding as much of the build as possible on the host, especially parts that might not be compatible between host and target (kernel headers, libc version)
• Being able to cross-execute as much of the first boot as possible as a systemd build target. This is safer from a maintanenc PoV, since the systemd people are more able to know what needs to be done.

I'm not worried about building the host systemd (yeah, it's wasted build time... ) I'm more worried about the first boot stuff being done properly offline.

[jameshilliard]

I wonder if we can build a systemctl binary that can be executed as part of the compilation process using this. That we can avoid having to build systemd twice.

[michaelolbrich]

@jameshilliard that just makes it more complicated. We just need to be able to compile as little as possible. And more importantly, do it on a platform that is a lot older that systemd expects to run on.
Right now, I usually start by building new kernel headers just to compile some networking code (I think) that won't be used anyways.

But fixing that won't be easy either. This means splitting the the static and shared libraries in a way that allows us to just build the code necessary to build the needed tool. I has been a while since I looked into this, but something like that won't be easy to maintain either.

[poettering]

I fail to see why systemd is in any way special here now? There are plenty of packages that build tools that are needed to build the rest, most build systems distinguish the host and the output compiler due to that, for example the linux kernel. Why do we need to cover this specfically, if tons of other projects need this too and our build system brings all the right distinctions of the two compilers already?

[jameshilliard]

The main issue seems to be that systemctl is fairly dependency heavy for a build tool, especially one that is only creating a few symlinks. From my understanding build tools usually are designed to be fairly self contained and have minimal build host dependencies, which is not the case for systemctl.