Questions and Bugs

[rodw-au]

When building the installer on Bookworm X86 host, I got this silly error which displays in a window

Building for a Pi 4b 64 bit
The file displayed exists in the root folder complete with the -- prefix

First encountered doing a make all from make menu
Reran again to build a patched preempt_rt kernel (no other changes except adding a patch and changing menuconfig to fully preemptible kernel) and it happened again. I noticed the kernel was built so I tried selecting make bootable file system and the error was immediately displayed again.

Deleted the file and the error persisted.

I don't know how to debug make here. Help would be appreciated.

[pyavitz]

I'll investigate. In the mean time you can also build using make commands

Is it actually producing an img? If so this may just be a dialog error.

[pyavitz]

I just ran native and got this.

That was building for the rootfs only. I have an x86_64 build currently going on all. I'll get back to you.

[rodw-au]

Thanks for a prompt response.
The first time it did build an image, second time it didn't.
I just did this and it seems to just print a file name:
debian@debian:~/devel/rpi-img-builder-lcnc$ make rootfs board=bcm2711
Root Filesystem
256968 -rw-r--r-- 1 root root 263120456 Aug 27 08:45 debian-bookworm-rootfs-arm64.tar.xz

But that file date is too early for it to be my RT build and I would have expected to see RT in the file name

[rodw-au]

So I might start again with:
make purge-all
make all board=bcm27111
and report back in a few hours

[pyavitz]

Nah. It is just telling you the rootfs tarball is already there. I have the builder setup, so if it finds the tarball, it skips the build process. Since it is time and bandwidth consuming.

If you wanna really purge everything, you need to remove the tarball ur self. sudo rm -f *tar*

[rodw-au]

NOw its really crazy. when I say purge all, it starts to build the kernel!
I might reclone

[rodw-au]

Recloned. Here is another bug.
When you set MENUCONFIG=1 in build config, the data is wrong. You get this and an error later and menuconfig does not load.

MENUCONFIG="1"01

[rodw-au]

Cloned again so Brand new
Does this look right? (loaded su so sudo does not time out)
/install.sh
make config
sudo su
make all board=bcm2711

RT patch from kernel.org in userpatches
deleted all kernel deb files from /output

[pyavitz]

Are you back spacing? dialog is a bit silly. So you need to make sure you delete the values listed before entering new ones.

Came right up for me?

[pyavitz]

Cloned again so Brand new Does this look right? (loaded su so sudo does not time out) /install.sh make config sudo su make all board=bcm2711

RT patch from kernel.org in userpatches deleted all kernel deb files from /output

Yes, looks good to me.

[rodw-au]

Thanks, building now. The patches seemed to be applied and menuconfig came up.
I am hoping to fork this for linuxcnc so they can put it in their buildbot. In the past they used a Pi which failed about once a month. Will be compiling linuxcnc from source. I think I know how to make it do that.

The dialog actually caught me out every time so I check it in a text editor to get it right.

[pyavitz]

I'm thinking that menuconfig bit might have something to do with this piece of code.

# userdata-txt
if [[ -f "userdata.txt" ]]; then
	sed -i 's/=0/="0"/g' userdata.txt
	sed -i 's/=1/="1"/g' userdata.txt
	sed -i 's/LLVM_IAS="1"/LLVM_IAS=1/g' userdata.txt
	echo -e "userdata file created."
	exit 0
fi

It doesn't know what you entered and is just putting quotes around what it is looking for.

[rodw-au]

It actually flagged it as invalid data and said it was ignoring it. First time through, I had no idea what that meant :)

[pyavitz]

I just saw what you wrote. And I'm not sure how make purge-all would try and build anything?

Craziness. I even tried to make it happen, which shouldn't work and didn't.

[rodw-au]

I just saw what you wrote. And I'm not sure how make purge-all would try and build anything?

Craziness. I even tried to make it happen, which shouldn't work and didn't.

Yes, I think by then, it was totally screwed! I must have broken it....
Anyway, a fresh start now I know what I am doing should work... :)

[rodw-au]

Hmm,
adduser: Please enter a username matching the regular expression
configured via the NAME_REGEX configuration variable. Use the
`--allow-bad-names' option to relax this check or reconfigure
NAME_REGEX in configuration.
user and PW were both CNC in the userdata.txt

[rodw-au]

OK, must all be lower case..I didn't know that.
Ubuntu only accepts usernames based on the following rules:
Must start with a lowercase letter
May only contain lowercase letters, underscore (_), and dash (-)
May optionally end with a dollar sign ($)

So I changed the userdata.txt to my name and reran

make image board=bcm2711
and I got:
adduser: The user `rodwebster' does not exist.
make: *** [Makefile:136: image] Error 1

[pyavitz]

Hmm, adduser: Please enter a username matching the regular expression configured via the NAME_REGEX configuration variable. Use the `--allow-bad-names' option to relax this check or reconfigure NAME_REGEX in configuration. user and PW were both CNC in the userdata.txt

That is interesting, I also didn't know this. I normally don't get complicated with my user information though.

This worked fine for me.

NAME="CNC"
USERNAME="cnc"
PASSWORD="cnc"

Last I checked the only variable you really need to be careful with here is NAME="".

GOOD

NAME="Marvin"

FAIL

NAME="Marvin Martian"

It doesn't like blank spaces.

[rodw-au]

Thanks I finally got success! I set name = user = password = cnc
I won't get a chance to test it until tomorrow but I have every confidence. :)

[rodw-au]

Finally was able to attempt to boot this image. I got a fail as shown below. I figured the boot loader might be old so upgraded it but still got the same error. It appears to not be able to find a FAT32 boot partition.

I noticed in custom.txt there is the following setting
`

Root Filesystem Types: ext4 btrfs xfs

FSTYPE="ext4"
`
But I don't see an option to use FAT. (Is that actually for the the Boot partition?)

Have you got any ideas here?
Using a Pi 4B about 4 years old (pre covid) and a 15gb SD card.
I'm a bit lost on how to resolve this. I am a very occasional Pi user...

[rodw-au]

That wasn't it. There is a FAT16 Boot partition

mmcblk0p1
vfat FAT16 BOOT 0783-4AD5 208.1M 17% /media/debian/BOOT

[pyavitz]

I no longer own a Pi4, but I wonder if me turning off LBA has in some way created an issue?

Let us check and see. You'll need a Linux box for this. Put the SD in the CARD reader and plug it in to ur linux box. Now find the /dev/$node and run: sudo parted --script /dev/$node set 1 lba on

Example:

sudo parted --script /dev/sdd set 1 lba on

You can check and make sure if you like.

sudo fdisk -l /dev/sdd
Disk /dev/sdd: 29.72 GiB, 31914983424 bytes, 62333952 sectors
Disk model: MassStorageClass
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd29ca7e4

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sdd1         2048   526335   524288  256M  c W95 FAT32 (LBA)
/dev/sdd2       526336 62332927 61806592 29.5G 83 Linux

[pyavitz]

Hmm. Apparently this is an issue; https://forums.raspberrypi.com/viewtopic.php?t=278295

I'll investigate. Thanx for the report.

[pyavitz]

Sorry about all that. This should resolve it: f3f48ad

[rodw-au]

A very interesting article even if its annoying! I might try your LBA tweak, then erase the MBR this way
https://www.cyberciti.biz/faq/linux-clearing-out-master-boot-record-dd-command/
I do remember walking a 12 bit FAT 30 years ago but never again!
Seems like I should be trying to write a standard Rpi image to start with in the hope it corrects the MBR

[rodw-au]

Thanks. Just going to bed. so will try tomorrow night.

[rodw-au]

It worked!
PREEMPT_RT kernel installed perfectly!
XFCE installed
LinuxCNC dependencies installed (about 2 Gb)
Building LinuxCNC now. That will take a while....
Once I have it all down pat, it will be time to script it in the installer.
Thanks for all your help. I really appreciate the prompt response and guidance.
Perhaps leave the issue open for a few days and I'll report any other issues..

[pyavitz]

I only ran a basic test, but looks right?

marvin: ~  $ sed 's/net.ifnames=0/net.ifnames=0 processor.max_cstate=1 isolcpus=2,3/g' board.txt
#!/bin/bash

# architecture
ARCH="arm64"
ARCH_EXT="arm64"
DEBARCH=""
CROSS_COMPILE="aarch64-linux-gnu-"
ROOTFS_ARCH="rootfs-${ARCH_EXT}"
STATIC="qemu-aarch64-static"

# menu
PRETTY_BOARD="Raspberry Pi 4/400		BCM2711 / ARM64"

# misc
FAMILY="broadcom"
FAMILY_EXT=""

# default config
LINUX_DEFCONFIG="bcm2711_defconfig"

# partitions
OFFSET="8192"
P_VALUE="p2"

# output
BOARD="bcm2711"
BOARD_EXT="rpi-4"
OUTPUT="output/${BOARD}"
IMG_EXP="${BOARD_EXT}-*.img*"

# cmdline
CONSOLE="console=serial0,115200 console=tty1"
EXTRA="loglevel=5 net.ifnames=0 processor.max_cstate=1 isolcpus=2,3 firmware_class.path=/lib/firmware/updates/brcm"

# packaging
KERNEL_IMG="kernel8.img"
SERIES="bcm2711-${BOARD_EXT}"

[rodw-au]

Sorted. Missing a " in lib/boards. I must have done that before you suggested function0 and deleted too much!
Building again now

[pyavitz]

good good... I was gonna say, I just ran a real test and sed worked just fine.

[rodw-au]

Great, Still building function2 here but far enough along to see that cmdline.txt is perfect!
console=serial0,115200 console=tty1 root=PARTUUID=3adbfbbd-02 rootfstype=ext4 fsck.repair=yes loglevel=5 net.ifnames=0 processor.max_cstate=1 isolcpus=2,3 firmware_class.path=/lib/firmware/updates/brcm rootwait

[rodw-au]

WHere should this go? I don't think it stuck. I had it in function2

#remove sudo no password
sudo rm -f /etc/sudoers.d/010_*-nopasswd.

Hmm, it was not commented out. Do you need to do it on the pi? (works there)

[pyavitz]

If you have it in uscripts you don't need to use sudo.

Try:

rm -f /etc/sudoers.d/010_${USERNAME}-nopasswd

If for some reason that doesn't work, comment out the lines in stage2.
https://github.com/pyavitz/rpi-img-builder/blob/master/scripts/stage2#L169

# Sudoers file
#if [ $USERACCT -eq 0 ]; then
#	echo ""
#	tee /etc/sudoers.d/010_${USERNAME}-nopasswd <<EOF
#${USERNAME} ALL=(ALL) NOPASSWD: ALL
#EOF
#fi

That should be it?

[rodw-au]

I'll see how I go. I am rebuilding without the sudo. I might leave it to the user to decide how he wants to do sudo.. Main thing I want to get sorted is to get more cores building the Linuxcnc debs. They are masssive! I have had one go at it a while ago and it errored. Hoping this will add another 15 threads into play to the build (-j'nproc' failed)
dpkg-buildpackage -b -uc -j$(nproc)

The last image I built failed to boot. Maybe becasue I did not purge-all, so trying again with a clean purge-all
If it fails this time, I'll go back to redo menuconfig. SO easy to get a step wrong

[pyavitz]

Might be easier to build the debs native on one of the units and then install the required depends and pre-built debs inside the chroot of the builder. Personally, I find building native is faster inside the chroot. But I don't have any x86_64's with that many threads. :)

Yes, it is easy for things to go wrong sometimes.

[rodw-au]

Hi again.

There is one thing I don't understand is how to create a defconfig. On a normal kernel build it seems this is done with make savedefconfig.

Does CUSTOM_DEFCONFIG save the defconfig? There is no mention of it being saved in the docs.
If there is no feature for this, could we add one? Maybe it creates one after menuconfig completes and then copies the defconfig to the defconfig folder?

I've had a wierd data corruption here with a 699 byte userdata.txt file that showed no data in the file. That has held me up for a few days but a couple of other linuxcnc users are working with this too. Finally burning what I hope is a good image again right now! It would be good to be able to lock in the defconfig settings.

[rodw-au]

Grr, my build failed again, going to try one someone els has built.

Also, does COMPRESS_IMG compress the image? My image is 6.4 gb and my mates is 2 gb for the same thing!

[pyavitz]

Yes, it compresses the img to an img.xz so it can be easily uploaded or transferred.

As for the defconfig, it looks like I didn't add the save to output/$BOARD directory yet. I'll add the feature and fix the ccache compiles today or tomorrow.

[rodw-au]

Thanks for the new commit. Sounds like it will be a great enhancement.
I'm glad my feedback is finding bugs and improving your solution!
It seems like I have a problem with mi Pi. Known good imgs are not running on my hardware. I just have not got a lot of time these next few days to check it out.

Thanks again for working with me.

[rodw-au]

One more question. my stage2 script builds the linuxcnc debs in /tmp and installs them.
Last night I copied them to the user's home directory so they don't get overwritten in /tmp

How can I save them into the output directory? I thought it was outside of the chroot. That is the most logical place to put them.

[pyavitz]

Thanks for the new commit. Sounds like it will be a great enhancement. I'm glad my feedback is finding bugs and improving your solution! It seems like I have a problem with mi Pi. Known good imgs are not running on my hardware. I just have not got a lot of time these next few days to check it out.

Thanks again for working with me.

I tend to neglect this builder, as I also have another one, which is where most of my focus goes. But yes, you have found a couple bugs I wasn't aware of, which I appreciate and have prompted me to back port features from the other builder into this one.

One more question. my stage2 script builds the linuxcnc debs in /tmp and installs them.
Last night I copied them to the user's home directory so they don't get overwritten in /tmp

How can I save them into the output directory? I thought it was outside of the chroot. That is the most logical place to put them.

Technically, you could copy them out before the img is unmounted using the stage1 script.
https://github.com/pyavitz/rpi-img-builder/blob/master/scripts/stage1#L184
The "." being the /root/ of the builder, so cp -fr p2/tmp/*.deb output/${BOARD}/ I thinks should work.

Super generic way to do it.

if [[ `ls p2/tmp/*.deb` ]]; then
	cp -fr p2/tmp/*.deb output/${BOARD}/
	rm -f p2/tmp/*.deb
fi

[rodw-au]

Thanks,
You may regret telling me about your other builder!
I have an Odroid N2+ which I wouild like to get Linuxcnc running on. Several others have tried other Armbian supported boards.

I have worked with Armbian, with limited sucess with the RT kernel even with the help of an Armbian RT guy.

Should we use the other builder for the Pi? I tackled this specific board becasue Linuxcnc has an existing but quite broken 32 bit Pi image.

PS I have another builder too. For linuxcnc on X86 hardware. Its a lot easier as there is buildbot for all the debs. I need to finish this one and get back to it.
https://github.com/rodw-au/linuxcnc-live-build

[pyavitz]

I don't use RT kernels, but with the other builder, it should be a lot easier to get that compiled.

I focus most of my attention on Amlogic and the current AML patch set I'm using seems pretty solid. At least on a headless level... I don't mess around with GUI's on ARM all that much.

As for Pis and that builder. No, not really. Any personal Pi builds I create for my self, I use this builder as the other one is primarily mainline centric.

[rodw-au]

Back at it again.
After merging your upstream changes, I went to build with menuconfig I got an error stating:
No rule to make menuconfig (I tried a couple of time)
So I ran it on the mainline with make kernel and that opened the menu
Restarted and ran it with make commit and it had a rule to make menuconfig
But when I built the kenrel I got V 6.1.51 not 6.1.46
So I am stuck here.
I noticed this error after extracting archive
gzip: stdin: unexpected end of file tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now lib/function/rpi-commit: line 85: cd: linux-rpi-6.1.y: No such file or directory

Is there anything I need to do to enable caching on future runs?

[rodw-au]

looks like rpi-commit is opening the wrong folder. Should be the commit number.

[pyavitz]

Should be fixed... Enable ccache ->

[rodw-au]

Thanks, that seems to be working now thanks.
I'll see how I go with some function 1 scripts.
In an automated build, is there a way to pipe "1" etc into bash to accept your continue/exit menu option?

I will say I'm easilly confused tonight. A friend who is building with this flat out calls himself Cornholio on our forum and you are C0rnelius on your emails. I think I should just call myself Confused :)

[pyavitz]

Thanks, that seems to be working now thanks. I'll see how I go with some function 1 scripts. In an automated build, is there a way to pipe "1" etc into bash to accept your continue/exit menu option?

I will say I'm easilly confused tonight. A friend who is building with this flat out calls himself Cornholio on our forum and you are C0rnelius on your emails. I think I should just call myself Confused :)

Why would you want too? All that does is save the defconfig(s) to output/$BOARD dir and give you the option to continue on or quit. If you actually wanna use the defconfig in future builds going forward you need to enable it in the userdata file and put the defconfig in the defconfig dir.

[rodw-au]

One more question. my stage2 script builds the linuxcnc debs in /tmp and installs them.
How can I save them into the output directory? I thought it was outside of the chroot. That is the most logical place to put them.

Technically, you could copy them out before the img is unmounted using the stage1 script. https://github.com/pyavitz/rpi-img-builder/blob/master/scripts/stage1#L184 The "." being the /root/ of the builder, so cp -fr p2/tmp/*.deb output/${BOARD}/ I thinks should work.

Super generic way to do it.

if [[ `ls p2/tmp/*.deb` ]]; then
	cp -fr p2/tmp/*.deb output/${BOARD}/
	rm -f p2/tmp/*.deb
fi

Is this a Stage 1 function?
ref: https://github.com/rodw-au/rpi-img-builder-lcnc/blob/linuxcnc/files/userscripts/uscripts#L10
I build linuxcnc in Stage 2
Should that be built in stage 1 too?
Or can this suggested script be moved to stage 2?
I'm abit confused about the stages

In an automated build, is there a way to pipe "1" etc into bash to accept your continue/exit menu option?

The ultimate goal is to have a fully automated process. Once we have run menuconfig once and created a custom defconfig (saved in my repo), we don't need to touch anything .
Debian 12 is on kernel 6.1. Its probably going to be a long time (if ever) before a new PreemptRT patch is issued. So our commit will stay the same. There is no need for users to then touch any settings. Plus if we want to run on a linuxcnc buildbot, it needs to be hands off..

[rodw-au]

Don't worry about the automated build. It does what I want now.

I will try using stage1 scripts. I finally get the difference in file paths between the stages :)

[pyavitz]

One more question. my stage2 script builds the linuxcnc debs in /tmp and installs them.
How can I save them into the output directory? I thought it was outside of the chroot. That is the most logical place to put them.

Technically, you could copy them out before the img is unmounted using the stage1 script. https://github.com/pyavitz/rpi-img-builder/blob/master/scripts/stage1#L184 The "." being the /root/ of the builder, so cp -fr p2/tmp/*.deb output/${BOARD}/ I thinks should work.
Super generic way to do it.

if [[ `ls p2/tmp/*.deb` ]]; then
	cp -fr p2/tmp/*.deb output/${BOARD}/
	rm -f p2/tmp/*.deb
fi

Is this a Stage 1 function? ref: https://github.com/rodw-au/rpi-img-builder-lcnc/blob/linuxcnc/files/userscripts/uscripts#L10 I build linuxcnc in Stage 2 Should that be built in stage 1 too? Or can this suggested script be moved to stage 2? I'm abit confused about the stages

In an automated build, is there a way to pipe "1" etc into bash to accept your continue/exit menu option?

The ultimate goal is to have a fully automated process. Once we have run menuconfig once and created a custom defconfig (saved in my repo), we don't need to touch anything . Debian 12 is on kernel 6.1. Its probably going to be a long time (if ever) before a new PreemptRT patch is issued. So our commit will stay the same. There is no need for users to then touch any settings. Plus if we want to run on a linuxcnc buildbot, it needs to be hands off..

OK. So you run menuconfig and when done it saves the configs in output/$BOARD and then either continues building the kernel with the new config you made or gives you the option to quit. From that point forward "if you are happy with the defconfig you made" you place either the FULL or MINIMAL defconifg inside the defconfig/ directory.

Now edit the userdata.txt file nano userdata.txt and tell the builder to use the defconfig.

### CUSTOM
CUSTOM_DEFCONFIG="1"
MYCONFIG="bcm2711_defconfig"

NOTICE: The file must end in _defconfig. The kernel doesn't accept bcm2711_defconfig.config or .txt or anything like that. So you can name the file whatever you want linuxcnc_defconfig as long as it ends in _defconfig.

[pyavitz]

As an aside you can also ask questions or report bugs on IRC or Discord.

Libera; #arm-img-builder
https://discord.gg/mypJ7NW8BG