core/device: ignore DEVICE_FOUND_UDEV bit on switching root

The issue systemd#12953 is caused by the following:
On switching root,
- deserialized_found == DEVICE_FOUND_UDEV | DEVICE_FOUND_MOUNT,
- deserialized_state == DEVICE_PLUGGED,
- enumerated_found == DEVICE_FOUND_MOUNT,
On switching root, most devices are not found by the enumeration process.
Hence, the device state is set to plugged by device_coldplug(), and then
changed to the dead state in device_catchup(). So the corresponding
mount point is unmounted. Later when the device is processed by udevd, it
will be changed to plugged state again.

The issue systemd#23208 is caused by the fact that generated udev database in
initramfs and the main system are often different.

So, the two issues have the same root; we should not honor
DEVICE_FOUND_UDEV bit in the deserialized_found on switching root.

This partially reverts c6e892b.

Fixes systemd#12953 and systemd#23208.
Replaces systemd#23215.

Co-authored-by: Martin Wilck <mwilck@suse.com>

src/core/device.c

@@ -158,19 +158,65 @@ static void device_set_state(Device *d, DeviceState state) {
 
 static int device_coldplug(Unit *u) {
         Device *d = DEVICE(u);
+        DeviceFound f;
+        DeviceState s;
+        Manager *m;
 
         assert(d);
         assert(d->state == DEVICE_DEAD);
 
         /* First, let's put the deserialized state and found mask into effect, if we have it. */
+        if (d->deserialized_state < 0)
+                return 0;
+
+        m = u->manager;
+        f = d->deserialized_found;
+        s = d->deserialized_state;
+
+        /* On initial boot, switch-root, reload, reexecute, the following happen:
+         * 1. MANAGER_IS_RUNNING() == false
+         * 2. enumerate devices: manager_enumerate() -> device_enumerate()
+         *    Device::enumerated_found is set.
+         * 3. deserialize devices: manager_deserialize() -> device_deserialize()
+         *    Device::deserialize_state and Device::deserialized_found are set.
+         * 4. coldplug devices: manager_coldplug() -> device_coldplug()
+         *    deserialized properties are copied to the main properties.
+         * 5. MANAGER_IS_RUNNING() == true: manager_ready()
+         * 6. catchup devices: manager_catchup() -> device_catchup()
+         *    Device::enumerated_found is applied to Device::found, and state is updated based on that.
+         *
+         * Notes:
+         * - On initial boot, no udev database exist. Hence, no devices are enumerated in the step 2.
+         *   Also, there is no deserialized device. Device units are (a) generated based on dependencies of
+         *   other units, or (b) generated when uevents are received.
+         *
+         * - On switch-root, no udev databse exist, except for devices with sticky bit, i.e. OPTIONS="db_persist".
+         *   Hence, almost no devices are enumerated in the step 2. However, in general, we have several
+         *   serialized devices. But, DEVICE_FOUND_UDEV bit in the deserialized_found must be ignored, as udev
+         *   rules in initramfs and the main system are often different. Hence, if the deserialized state is
+         *   DEVICE_PLUGGED, we need to downgrade it to DEVICE_TENTATIVE (or DEVICE_DEAD if nobody sees the device).
+         *   Unlike the other starting mode, Manager::honor_device_enumeration == false (maybe, it is better to
+         *   rename the flag) when device_coldplug() and device_catchup() are called. Hence, let's conditionalize
+         *   the operations by using the flag. After switch-root, systemd-udevd will (re-)process all devices,
+         *   and the Device::found and Device::state will be adjusted.
+         *
+         * - On reload or reexecute, we can trust enumerated_found, deserialized_found, and deserialized_state.
+         *   Of course, deserialized parameters may be outdated, but the unit state can be adjusted later by
+         *   device_catchup() or uevents. */
+
+        if (!m->honor_device_enumeration && !MANAGER_IS_USER(m)) {
+                f &= ~DEVICE_FOUND_UDEV; /* ignore DEVICE_FOUND_UDEV bit */
+                if (s == DEVICE_PLUGGED)
+                        s = DEVICE_TENTATIVE; /* downgrade state */
+                if (f == DEVICE_NOT_FOUND)
+                        s = DEVICE_DEAD; /* If nobody sees the device, downgrade more */
+        }
 
-        if (d->deserialized_state < 0 ||
-            (d->deserialized_state == d->state &&
-             d->deserialized_found == d->found))
+        if (d->found == f && d->state == s)
                 return 0;
 
-        d->found = d->deserialized_found;
-        device_set_state(d, d->deserialized_state);
+        d->found = f;
+        device_set_state(d, s);
         return 0;
 }
 
@@ -639,13 +685,9 @@ static void device_found_changed(Device *d, DeviceFound previous, DeviceFound no
 }
 
 static void device_update_found_one(Device *d, DeviceFound found, DeviceFound mask) {
-        Manager *m;
-
         assert(d);
 
-        m = UNIT(d)->manager;
-
-        if (MANAGER_IS_RUNNING(m) && (m->honor_device_enumeration || MANAGER_IS_USER(m))) {
+        if (MANAGER_IS_RUNNING(UNIT(d)->manager)) {
                 DeviceFound n, previous;
 
                 /* When we are already running, then apply the new mask right-away, and trigger state changes