systemd-run --scope --user fails in systemd 249.6

[diabonas]

systemd version the issue has been seen with
249.6

Used distribution
Arch Linux (downstream tracking bug report: FS#72701)

Linux kernel version used (uname -a)
Linux hostname 5.15.1-arch1-2 #1 SMP PREEMPT Tue, 09 Nov 2021 16:16:12 +0000 x86_64 GNU/Linux

CPU architecture issue was seen on
x86_64

Expected behaviour you didn't see
In systemd version <= 249.5, running something like systemd-run --scope --user sleep 100 results in an output like

Running scope as unit: run-rf06cf365f1ed444fa9313bf2c24da800.scope

and the command (sleep 100 in this case) successfully being executed in a scope unit.

Unexpected behaviour you saw
In systemd version 249.6, systemd-run --scope --user sleep 100 results in an error message and the command is not getting run:

Job failed. See "journalctl -xe" for details.

Note that running the command as root (using systemd-run --scope sleep 100) works perfectly fine, only user units appear to be affected.

Steps to reproduce the problem
Run systemd-run --scope --user sleep 100 with systemd 249.6.

Additional program output to the terminal or log subsystem illustrating the issue
Running journalctl -xe as suggested by the above error message gives the following errors:

Nov 10 21:09:13 hostname systemd[677]: run-rf06cf365f1ed444fa9313bf2c24da800.scope: No PIDs left to attach to the scope's control group, refusing: Success
Nov 10 21:09:13 hostname systemd[677]: run-rf06cf365f1ed444fa9313bf2c24da800.scope: Failed with result 'resources'.

This points towards commit 8d3e4ac, backported to systemd 249.6 as systemd/systemd-stable@7ecb1b8, as the culprit. Indeed, reverting this commit fixes the issue for me.

[diabonas]

If I understand correctly, the issue is that attaching the process to the cgroup might be delegated to the system instance here:

systemd/src/core/cgroup.c

Lines 2278 to 2287 in 8b212f3

	/* If we are in a user instance, and we can't move the process ourselves due
	* to permission problems, let's ask the system instance about it instead.
	* Since it's more privileged it might be able to move the process across the
	* leaves of a subtree whose top node is not owned by us. */
	z = unit_attach_pid_to_cgroup_via_bus(u, pid, suffix_path);
	if (z < 0)
	log_unit_info_errno(u, z, "Couldn't move process "PID_FMT" to requested cgroup '%s' (directly or via the system bus): %m", pid, empty_to_root(p));
	else
	continue; /* When the bus thing worked via the bus we are fully done for this PID. */

However, the thusly attached process is not counted in the same way that commit 8d3e4ac added here:

systemd/src/core/cgroup.c

Lines 2294 to 2295 in 8b212f3

	} else if (ret >= 0)
	ret++; /* Count successful additions */

which leads to the "No PIDs left to attach to the scope's control group" failure.

If my analysis is correct, a fix would be counting delegated additions in ret, like this:

diff --git a/src/core/cgroup.c b/src/core/cgroup.c
index abc30e3990..c942db8d05 100644
--- a/src/core/cgroup.c
+++ b/src/core/cgroup.c
@@ -2283,8 +2283,11 @@ int unit_attach_pids_to_cgroup(Unit *u, Set *pids, const char *suffix_path) {
                                 z = unit_attach_pid_to_cgroup_via_bus(u, pid, suffix_path);
                                 if (z < 0)
                                         log_unit_info_errno(u, z, "Couldn't move process "PID_FMT" to requested cgroup '%s' (directly or via the system bus): %m", pid, empty_to_root(p));
-                                else
+                                else {
+                                        if (ret >= 0)
+                                                ret++; /* Count successful additions */
                                         continue; /* When the bus thing worked via the bus we are fully done for this PID. */
+                                }
                         }

                         if (ret >= 0)

I confirm that this patch fixes the issue on my system, so I opened PR #21298 for it.