dev-xvdd takes a long time

[Rudd-O]

Without Plymouth in the boot transaction, the unit that takes the longest is dev-xvdd.device.

What program mounts it? I cannot find any reference to it.

[Rudd-O]

(systemd-analyze blame, systemd-analyze plot)

[Rudd-O]

(systemd-analyze blame, systemd-analyze plot)

[marmarek]

On Sun, Oct 23, 2016 at 02:06:27PM -0700, Rudd-O wrote:

Without Plymouth in the boot transaction, the unit that takes the longest is dev-xvdd.device.

What program mounts it? I cannot find any reference to it.

This is partition with kernel modules, mounted by initramfs.
https://github.com/QubesOS/qubes-linux-utils/blob/master/dracut/simple/init.sh

What waits for dev-xvdd.device? It isn't even mentioned in /etc/fstab...

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Sun, Oct 23, 2016 at 02:06:27PM -0700, Rudd-O wrote:

Without Plymouth in the boot transaction, the unit that takes the longest is dev-xvdd.device.

What program mounts it? I cannot find any reference to it.

This is partition with kernel modules, mounted by initramfs.
https://github.com/QubesOS/qubes-linux-utils/blob/master/dracut/simple/init.sh

What waits for dev-xvdd.device? It isn't even mentioned in /etc/fstab...

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[andrewdavidwong]

@Rudd-O: This is on R3.2, right?

[andrewdavidwong]

@Rudd-O: This is on R3.2, right?

[Rudd-O]

Yes.

[Rudd-O]

Yes.

[Rudd-O]

Some of the waiting happens here:

https://github.com/QubesOS/qubes-linux-utils/blob/master/dracut/simple/init.sh#L70

but this should absolutely and totally not show up in the systemd analyze logs, since systemd has no such information.

Frankly, the mounting of modules.img should just happen inside the VM as it boots, with either a fstab mountpoint or a systemd mount unit. We really should have a proper initrd with systemd support so that we get full boot time analysis, not just from the switchroot point onwards.

I guess we should revisit this issue once we have moved away from having kernel and modules stored in dom0 instead of directly in the template. Then it will be much clearer what to do.

[Rudd-O]

Some of the waiting happens here:

https://github.com/QubesOS/qubes-linux-utils/blob/master/dracut/simple/init.sh#L70

but this should absolutely and totally not show up in the systemd analyze logs, since systemd has no such information.

Frankly, the mounting of modules.img should just happen inside the VM as it boots, with either a fstab mountpoint or a systemd mount unit. We really should have a proper initrd with systemd support so that we get full boot time analysis, not just from the switchroot point onwards.

I guess we should revisit this issue once we have moved away from having kernel and modules stored in dom0 instead of directly in the template. Then it will be much clearer what to do.

[marmarek]

Frankly, the mounting of modules.img should just happen inside the VM as it boots, with either a fstab mountpoint or a systemd mount unit.

It's hard to get it that way:

1. systemd-udevd and systemd itself assumes it can access modules (or any of modules.* metadata files) as early as it want to. And things fails when it isn't there.
2. modules.img is read only. We want to mount only specific subdirectory for the running kernel, to not interfere with other kernel installation in the VM (previously we have kernel package excluded there, but it was really PITA). This means static fstab line or unit file will not be enough.

We really should have a proper initrd with systemd support so that we get full boot time analysis, not just from the switchroot point onwards.

If you want, use pvgrub. The initramfs generated inside the VM use "full" flavor of dracut module, which doesn't supersede the systemd running there. The one provided by dom0 specifically have systemd disabled to not worry about version difference. systemctl daemon-reexec you've seen helps somehow, but for larger version difference it crashes...

[marmarek]

Frankly, the mounting of modules.img should just happen inside the VM as it boots, with either a fstab mountpoint or a systemd mount unit.

It's hard to get it that way:

1. systemd-udevd and systemd itself assumes it can access modules (or any of modules.* metadata files) as early as it want to. And things fails when it isn't there.
2. modules.img is read only. We want to mount only specific subdirectory for the running kernel, to not interfere with other kernel installation in the VM (previously we have kernel package excluded there, but it was really PITA). This means static fstab line or unit file will not be enough.

We really should have a proper initrd with systemd support so that we get full boot time analysis, not just from the switchroot point onwards.

If you want, use pvgrub. The initramfs generated inside the VM use "full" flavor of dracut module, which doesn't supersede the systemd running there. The one provided by dom0 specifically have systemd disabled to not worry about version difference. systemctl daemon-reexec you've seen helps somehow, but for larger version difference it crashes...