Skip to content

fix(os): ship systemd drop-ins as vendor config, not into /etc - #1158

Merged
kvinwang merged 1 commit into
nextfrom
fix/dropins-vendor-dir
Sep 2, 2026
Merged

fix(os): ship systemd drop-ins as vendor config, not into /etc#1158
kvinwang merged 1 commit into
nextfrom
fix/dropins-vendor-dir

Conversation

@kvinwang

@kvinwang kvinwang commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Why

The image stages its systemd drop-ins into /etc/systemd/system/<unit>.d/. That is the administrator's layer, and systemctl(1) is explicit about what living there means:

revert UNIT… removes drop-in configuration files that modify the specified units […] the matching directories foo.service.d/ with all their contained files are removed, both below the persistent and runtime configuration directories (i.e. below /etc/systemd/system and /run/systemd/system)

So a perfectly ordinary systemctl revert docker deletes dstack-prepare.conf and dstack-guest-agent.conf, and docker loses its ordering against dstack-prepare.service and dstack-guest-agent.service. Same for containerd, and for the simulator's ordering drop-in on dstack-prepare.service.

None of these are operator configuration. They are ordering the image itself depends on:

[Unit]
Wants=dstack-prepare.service
After=dstack-prepare.service

The inconsistency was already visible inside a single drop-in directorydstack-prepare.service.d/ holds 10-nvidia-gpu-ordering.conf under ${systemd_system_unitdir} and tee-simulator.conf under ${sysconfdir}.

What changed

Four drop-ins move from /etc/systemd/system/ to /usr/lib/systemd/system/, in both image backends:

drop-in unit
dstack-guest-agent.conf docker.service
dstack-prepare.conf docker.service
dstack-prepare.conf containerd.service
tee-simulator.conf dstack-prepare.service

parity.json is updated so the backend comparison tracks the new paths.

Nothing is lost

Drop-ins from every search directory are applied, ordered lexically by filename across directories, so systemctl edit still wins with its override.conf. The image simply stops occupying the namespace reserved for whoever operates it — and gains the property that systemctl revert restores the vendor state instead of destroying it.

There is no measurement consequence either way: dstack overlays /etc, /usr and /bin alike, with measured content in the lower layer and writes in a tmpfs that resets each boot.

Testing

Rebuilt both affected recipes against the vendored yocto tree. bitbake -c package succeeds and the drop-ins land in the new location:

/dstack-guest/usr/lib/systemd/system/docker.service.d/dstack-guest-agent.conf
/dstack-guest/usr/lib/systemd/system/docker.service.d/dstack-prepare.conf
/dstack-guest/usr/lib/systemd/system/containerd.service.d/dstack-prepare.conf
/dstack-tee-simulator/usr/lib/systemd/system/dstack-prepare.service.d/tee-simulator.conf

shellcheck -x -P SCRIPTDIR clean on the mkosi build script, parity.json still parses.

Scope

The NVIDIA fabric-manager drop-in has the same problem but is fixed in #1157, which already touches that file — this PR deliberately leaves it alone to avoid a conflict. After both land, no image-shipped drop-in remains under /etc.

The image stages its drop-ins into /etc/systemd/system/<unit>.d. That is the
administrator's layer, and systemctl(1) is explicit about what it means:

  revert UNIT... removes drop-in configuration files that modify the
  specified units [...] the matching directories foo.service.d/ with all
  their contained files are removed, both below the persistent and runtime
  configuration directories (i.e. below /etc/systemd/system and
  /run/systemd/system)

So an ordinary `systemctl revert docker` deletes dstack-prepare.conf and
dstack-guest-agent.conf, and docker loses its ordering against
dstack-prepare.service and dstack-guest-agent.service. The same applies to
containerd, and to the tee simulator's ordering drop-in on
dstack-prepare.service. None of these are operator configuration; they are
ordering the image itself depends on.

The split was already visible inside a single drop-in directory:
dstack-prepare.service.d holds 10-nvidia-gpu-ordering.conf under
${systemd_system_unitdir} and tee-simulator.conf under ${sysconfdir}.

Nothing is lost by moving them. Drop-ins from every search directory are
applied, ordered lexically by filename, so `systemctl edit` still wins with
its override.conf. The image simply stops occupying the namespace reserved
for whoever operates it.

There is no measurement consequence either way: dstack overlays /etc, /usr
and /bin alike, measured content in the lower layer and writes in a tmpfs
that resets each boot.

Verified by rebuilding both affected recipes; dstack-guest and
dstack-tee-simulator now package their drop-ins under
/usr/lib/systemd/system/.
Copilot AI lite review requested due to automatic review settings September 2, 2026 05:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The path changes are consistent across both image backends and the parity tracking update matches the new vendor drop-in location.

Pull request overview

This PR fixes image-shipped systemd drop-in placement by moving vendor-owned drop-ins from the operator layer (/etc/systemd/system/<unit>.d/) into the vendor unit directory (/usr/lib/systemd/system/<unit>.d/), preventing systemctl revert from deleting required ordering configuration.

Changes:

  • Update Yocto recipes to install the affected drop-ins under ${systemd_system_unitdir} (vendor) instead of ${sysconfdir}/systemd/system (admin).
  • Update the mkosi build staging script to ship docker/containerd drop-ins and the simulator drop-in under /usr/lib/systemd/system.
  • Adjust os/mkosi/parity.json to track the simulator drop-in at its new path.
File summaries
File Description
os/yocto/layers/meta-dstack/recipes-core/dstack-tee-simulator/dstack-tee-simulator.bb Move tee-simulator.conf drop-in install + packaging path into ${systemd_system_unitdir}.
os/yocto/layers/meta-dstack/recipes-core/dstack-guest/dstack-guest.bb Install docker/containerd drop-ins into ${systemd_system_unitdir} and update FILES:${PN} accordingly.
os/mkosi/parity.json Update simulator drop-in path from etc/systemd/system/... to usr/lib/systemd/system/... for backend parity checks.
os/mkosi/components/dstack-rust/dstack-rust-build.sh Stage docker/containerd drop-ins and simulator drop-in under /usr/lib/systemd/system instead of /etc/systemd/system.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@kvinwang
kvinwang merged commit c31aa2c into next Sep 2, 2026
15 checks passed
@kvinwang
kvinwang deleted the fix/dropins-vendor-dir branch September 2, 2026 05:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants