Boot Partition

[fwskvogt]

In a Debian Buster installation I tried to run

ltsp image /

At the end of the output I get:

Could not locate vmlinuz and initrd.img in /srv/ltsp/images/x86_64.img

I think this is because I have a seperate /boot partition which is not mounted at the start of the process of building the image. The output mentions:

Running: mount --bind --make-private -o ro / /tmp/tmp.1kY9JjhRSf/root/

So all the files in the original /boot directory are missing in /tmp/tmp.1kY9JjhRSf/root/.

What can I do?

With kind regards

Jürgen Vogt

[alkisg]

Hi Jürgen,

You need "advanced image sources", documented in man ltsp ipxe. Specifically, try this:

ltsp image /,,/boot,subdir=boot

[fwskvogt]

Hello Alkis,

thanks for your reply. I tried it. Here is the output:

root@****:~# ltsp image /,,/boot,subdir=boot
Using x86_64 as the base name of image /
Running: mount --bind --make-private -o ro / /tmp/tmp.zbqqGAmh0q/root/
Running: mount --bind --make-private -o ro /boot /tmp/tmp.zbqqGAmh0q/root/boot
Package management locking isn't supported in your distribution, continuing without it...
Running: mount -t overlay -o upperdir=/tmp/tmp.zbqqGAmh0q/tmpfs/up,lowerdir=/tmp/tmp.zbqqGAmh0q/root,workdir=/tmp/tmp.zbqqGAmh0q/tmpfs/work overlay /tmp/tmp.zbqqGAmh0q/root
Cleaning up x86_64 before mksquashfs...
Replacing /tmp/tmp.zbqqGAmh0q/root/etc/ssh/ssh_host_dsa_key
Replacing /tmp/tmp.zbqqGAmh0q/root/etc/ssh/ssh_host_ecdsa_key
Replacing /tmp/tmp.zbqqGAmh0q/root/etc/ssh/ssh_host_ed25519_key
Replacing /tmp/tmp.zbqqGAmh0q/root/etc/ssh/ssh_host_rsa_key
Generating a RSA private key
.............+++++
.............................................................................................................................+++++
writing new private key to '/tmp/tmp.zbqqGAmh0q/root/etc/ssl/private/ssl-cert-snakeoil.key'
-----
Parallel mksquashfs: Using 6 processors
Creating 4.0 filesystem on /srv/ltsp/images/x86_64.img.tmp, block size 131072.
[=========================================================/] 302360/302360 100%

Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 3743939.84 Kbytes (3656.19 Mbytes)
	41.18% of uncompressed filesystem size (9092607.77 Kbytes)
Inode table size 3293857 bytes (3216.66 Kbytes)
	28.95% of uncompressed inode table size (11379626 bytes)
Directory table size 3236043 bytes (3160.20 Kbytes)
	40.27% of uncompressed directory table size (8034924 bytes)
Xattr table size 46 bytes (0.04 Kbytes)
	38.33% of uncompressed xattr table size (120 bytes)
Number of duplicate files found 27236
Number of inodes 325085
Number of files 260455
Number of fragments 21135
Number of symbolic links  36386
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 8
Number of directories 28236
Number of ids (unique uids + gids) 33
Number of uids 16
	root (0)
	daemon (1)
	lp (7)
	vogtj (1001)
	www-data (33)
	openldap (124)
	speech-dispatcher (112)
	Debian-exim (107)
	lightdm (118)
	colord (109)
	_apt (125)
	man (6)
	mysql (119)
	statd (108)
	systemd-timesync (100)
	proxy (13)
Number of gids 29
	root (0)
	daemon (1)
	dip (30)
	lp (7)
	Debian-exim (116)
	vogtj (1001)
	www-data (33)
	shadow (42)
	lightdm (124)
	openldap (131)
	nslcd (128)
	ssl-cert (112)
	tty (5)
	crontab (107)
	mail (8)
	mlocate (117)
	ssh (118)
	messagebus (109)
	utmp (43)
	staff (50)
	lpadmin (113)
	mysql (125)
	man (12)
	colord (120)
	nogroup (65534)
	sambashare (135)
	systemd-timesync (103)
	adm (4)
	proxy (13)
Running: ltsp kernel /srv/ltsp/images/x86_64.img
Running:  mount -t squashfs -o ro /srv/ltsp/images/x86_64.img /tmp/tmp.yHgjpStOKU/ltsp/
Could not locate vmlinuz and initrd.img in /srv/ltsp/images/x86_64.img

The boot-directory is mounted but in the squashfs image the files in /boot are still not present.

[alkisg]

Ah that's a second problem. It's probably caused because overlayfs doesn't use the "merged view" of the mounts by default, but only a single mount. In LTSP5, we handled that with additional overlay mounts, we should probably do the same here.

I'll need a few days to resolve that; in the mean time, please do something like this as a workaround:

mount --bind / /mnt
cp -a /boot/. /mnt/boot/
umount /mnt
ltsp image /

I.e. copy the contents of the /boot partition to the /boot folder of the / partition, before calling ltsp image /.

[alkisg]

Btw, I just moved this issue from the community issues to ltsp issues as it proved to be a bug.

I do love this "ltsp" vs "community" issues separation!
Something starts as a discussion and it ends up being a bug? We just move it from community/issues to ltsp/issues.
Or the opposite, something starts as a bug but the user just needed advice? We move it to community/issues.

There's no need for users to have two different accounts/interfaces (e.g. launchpad and sourceforge mailing lists for LTSP5), and there's no need to restart the conversation after the move. I hope github won't mind us abusing a bit the issues tracker for discussions, but it's a great tool!

[fwskvogt]

I successfully applied the workaround . I'm seeing forward to your solution. Jürgen Am Samstag, den 12.10.2019, 22:45 -0700 schrieb Alkis Georgopoulos:

Ah that's a second problem. It's probably caused because overlayfs doesn't use the "merged view" of the mounts by default, but only a single mount. In LTSP5, we handled that with [additional overlay mounts]( https://git.launchpad.net/ltsp/tree/server/ltsp-update-image#n149), we should probably do the same here. I'll need a few days to resolve that; in the mean time, please do something like this as a workaround: ```shell mount --bind / /mnt cp -a /boot/. /mnt/boot/ umount /mnt ltsp image / ``` I.e. copy the contents of the /boot partition to the /boot folder of the / partition, before calling `ltsp image /`.

-- FREIE WALDORFSCHULE KASSEL Jürgen Vogt Hunrodstraße 17 34131 Kassel Telefon 0561 935130 Fax 0561 9351316 Email j.vogt@waldorfschule-kassel.de www www.waldorfschule-kassel.de

[alkisg]

Small status update:

the reason I haven't implemented this yet, is that this specific part needs a code redesign.

The old code performs all mounts and submounts and then does a tmpfs/overlay on top of the result. But overlayfs doesn't support multiple lowerdirs in varying subdirs (they all go in a single dst/), so another approach is needed.

With the new approach, a tmpfs will be created first, and then all submounts will be tmpfs/overlays on top of the appropriate subdirs. This will make it impossible to boot a client in "ltsp mode" and not have a tmpfs/overlay for /. If this is ever needed, we'll need another small redesign then.

[enoch85]

Hey, writing here as my issue was closed and referring to this one.

1. The workaround worked.
2. Another issue arise which is that ipxe requires filename.xyz.0. Problem is that the files undionly.kpxe and ltsp.ipxe aren't named .0 at the end. To solve that I simply copied the original files and added .0 after. Now everything works like a charm!

I can add that when installing dnsmasq my DNS went away, so to be able to run the ltsp ipxe command to get all the stuff, I had to purge dnsmasq - maybe that was the issue with not having the .0 files?

[alkisg]

iPXE worked fine without the .0 extension in all the tests I've done, and AFAIK no other LTSP user had that problem.

Can you open another issue about that problem, and mention more details?

Btw when installing dnsmasq, normally it won't start at all because systemd-resolved already has the 53 port open, so in the new issue also mention how you installed dnsmasq etc, i.e. what was different in your case compared to the installation page instructions.

[schweer-guest]

I managed to work around the multiple partion issue using a plain mksquashfs command:

Instead of running ltsp image /
I ran these commands:

cp /usr/share/ltsp/server/image/image.excludes /tmp/excludes
if [ -f /etc/ltsp/image-local.excludes ] ; then
	cat /etc/ltsp/image-local.excludes >> /tmp/excludes
fi
mksquashfs / /srv/ltsp/images/$(uname -m).img -noappend -wildcards -ef /tmp/excludes
rm /tmp/excludes
ltsp kernel

This worked for me however fancy the partitioning has been.

Wolfgang

[alkisg]

Note that this has security implications, e.g. it leaves the server /etc/shadow in the image without cleaning up the server user accounts (no cleanup phase).

[schweer-guest]

Hi Alkis, thanks for the security related warning.

In the case of Debian Edu, the user accounts are kept in LDAP (homedirs are made available using kerberized NFS and autofs-ldap).
Instead of running ltsp image /, this worked for me on both the main server (where LDAP is located) and a workstation (turned into a separate LTSP server).

TEMPDIR=$(mktemp -d)
mkdir $TEMPDIR/etc
cp /etc/shadow* $TEMPDIR/etc
sed -i '/root:/d' /etc/shadow
sed -i '/root:/d' /etc/shadow-
cp /usr/share/ltsp/server/image/image.excludes $TEMPDIR/excludes
if [ -f /etc/ltsp/image-local.excludes ] ; then
	cat /etc/ltsp/image-local.excludes >> $TEMPDIR/excludes
fi
mksquashfs / /srv/ltsp/images/$(uname -m).img -noappend -wildcards -ef $TEMPDIR/excludes
cp $TEMPDIR/etc/shadow* /etc
rm -rf $TEMPDIR
ltsp kernel

I hope that this is the right way to fix the security issue.

[alkisg]

I'm planning to have this fixed for the LTSP 20.01 release; hang on. :)

[schweer-guest]

Great, thank you.

[alkisg]

Fix committed in 147a659.

@schweer-guest, @fwskvogt, or anyone else affected, could you please try the latest build from the proposed PPA and see if it solves your issue without any regressions, so that we upload LTSP 20.01 to Debian? Thanks!

Remember that submounts aren't automatically discovered, so a syntax like the one described here is necessary.

[schweer-guest]

@alkisg
Thanks for fixing this issue.
Test on a system with separate partitions for /boot, /usr and /var has been successful (running ltsp image /,,/boot,subdir=boot,,/usr,subdir=usr,,/var,subdir=var).

There's a regression, though: additional image excludes are no longer evaluated. I had to temporarily add those to /usr/share/ltsp/server/image/image.excludes to get it working after I noticed that directories contained in /etc/ltsp/image-local.excludes were still showing up in the image; explicitly using ADD_IMAGE_EXCLUDES to specify the excludes file in ltsp.conf didn't work either.

[alkisg]

Hi @schweer-guest,

I re-tested ADD_IMAGE_EXCLUDES and it's still working fine for me.
Specific tests:

1. This excludes everything under /usr:

[server]
ADD_IMAGE_EXCLUDES="usr/*"

2. And this does the same:

[server]
ADD_IMAGE_EXCLUDES="/etc/ltsp/image-local.excludes"

...where that file contains the line: usr/*.

Make sure you're putting those under [server], not e.g. under [clients]...

The related changes for image excludes were not done in this recent commit; they were done in e780fbb.

[schweer-guest]

Hi @alkisg,
thanks for the feed back and sorry for the noise: I was misled by this line still contained in /usr/share/ltsp/server/image/image.excludes:

# Create /etc/ltsp/image-local.excludes to add more entries;

I guess the above line should be dropped.

And then, you are right: I placed the ADD_IMAGE_EXCLUDES statement accidently below [clients].
So to summarize: It's all fine with ltsp 20.01-1.
One thing left: Please add Closes: #946946 to the debian/changelog line containing #43, so that the Debian BTS gets notified to close the bug, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=946946

[alkisg]

True, that documentation line needs to go.

I'm reusing the changelog in the github announcements, for example #1 (comment), so Closes: #946946 there would be considered a github issue link... I'm not sure what's the best way to handle that, but for the moment I was trying to completely avoid "Closes". Additionally, I would like to minimize the use of distro-specific bugs as much as possible, in favor of the upstream bug tracker...

I'll rethink that approach in the future when I'll have more free time... maybe then I'll be able to maintain two changelogs for each release, one upstream and a Debian one.

[schweer-guest]

ok, I will close the Debian bug then.
Thanks for your work on LTSP!

[alkisg]

You're welcome, thanks for testing.

I amended the aforementioned comment in 375c9f8.

[alkisg]

Ouch, even with all the delay to find time to do this properly, the fix broke netbooting live CDs.

In live CDs, we want to mount the squashfs image first, then find the overlay module inside it, as it's not otherwise available in the initramfs, and then do the tmpfs/overlay merge.
While this #43 issue required the opposite, to do the tmpfs/overlay first and then to start the mounting process.

...I'll try to find some workaround that doesn't result in completely unreadable code...

[danboid]

LTSP was failing to find my kernel and initrd at the end of creating the image so I ran:

mount --bind / /mnt
cp -a /boot/. /mnt/boot/
umount /mnt
ltsp image /

As advised above and that has fixed the missing kernel problem at least.

The problem here though is that this issue was closed almost a year ago but the problem persists in the latest PPA version of LTSP. If this can't be fixed then shouldn't these extra steps be mentioned in the installation guide?

[alkisg]

Hi danboid,

what you did was not the correct solution. I replied with the correct solution in my first comment in this present issue. I.e. you just need to tell ltsp image that you have a separate boot partition. To be more clear, you should not run ltsp image /, but you should run this:

ltsp image /,,/boot,subdir=boot

[danboid]

I skipped trying that because the person who tried it said it didn't work for them.

It seems to me that command::

ltsp image /,,/boot,subdir=boot

Should be added to the installation guide as most people won't find it in the man page. I expect it applies to everyone who installs Ubuntu using ZFS.

[alkisg]

The correct ltsp image command line didn't work for him because there was a bug then, which is what was resolved in this issue months ago.

But you're right, an example should be included in the ltsp image man page as it's not an uncommon use case to have a separate boot partition.

[danboid]

I'm recommending this command be included within the main installation guide, or at least it should be mentioned that is command is in the man page in the install guide if not.

[gilbertoferreira]

Ah that's a second problem. It's probably caused because overlayfs doesn't use the "merged view" of the mounts by default, but only a single mount. In LTSP5, we handled that with additional overlay mounts, we should probably do the same here.

I'll need a few days to resolve that; in the mean time, please do something like this as a workaround:

mount --bind / /mnt
cp -a /boot/. /mnt/boot/
umount /mnt
ltsp image /

I.e. copy the contents of the /boot partition to the /boot folder of the / partition, before calling ltsp image /.

This works for me.
2022 and still have this bug!

[alkisg]

Hi, this issue has been solved years ago, please read the man page.

https://ltsp.org/man/ltsp-image/#examples

Inform ltsp image that a chrootless installation uses separate /boot and /opt partitions:

ltsp image /,,/boot,subdir=boot,,/opt,subdir=opt

[gilbertoferreira]

Hi, this issue has been solved years ago, please read the man page.

https://ltsp.org/man/ltsp-image/#examples

Inform ltsp image that a chrootless installation uses separate /boot and /opt partitions:

ltsp image /,,/boot,subdir=boot,,/opt,subdir=opt

Oh! I see!
This is not in the main documentation which I was used as a guideline.
I didn't saw that in the "Maintaining a client image" section, so that made me confused.

Sorry