Converting to NOOBS format

[procount]

I am trying to convert your image to be installable by NOOBS/PINN.
If I remove the /boot/autoexpand_root_partition sentinel file as recommended then the raspberrypi64 password does not get set. In this case what is the default password for root to login?

As a workaround, I was modifying the etc/local.d/autoexpand_root_partition.startso that the password and graphical login would be set without partition resizing. But for some reason my root partition always ends up being r/o so that the script will not be effective. I'm still investigating that, but any ideas why this partition should end up readonly would be welcome.

[sakaki-]

The default password is supposed to be raspberrypi64 anyway, but this was causing issues for some people (see #1) so I added the belt-and-braces setting to the autoexpand_root_partition.start file. In any event, you can always do sudo su --login root as demouser to get a root prompt (given the /etc/sudoers setup on the image, this requires no password, not even demouser's), then set your own password.
No idea why your / should be ro though. Is your root partition ext4, as set up in /boot/cmdline.txt?

[sakaki-]

One thought - on Gentoo under OpenRC, the file /etc/init.d/root is what brings the root filesystem to a writeable state (it is ro in very early boot). This init script may not be be being executed (although in this case you'd expect the system to fail to come up), or your / may have failed an fsck and been retained as ro for that reason... have you tried checking the / filesystem (on a PC, when not mounted)?

[procount]

I'm using a standard procedure to convert the image to NOOBS format. It works on countless other OSes, so I just need to identify which bits are different for gentoo.
Here is what I am doing...

1. I extract the image file and loop mount the 2 partitions inside it.
2. I change /boot/autoexpand_root_partition to /boot/autoexpand_root_none
3. I replace /etc/local.d/autoexpand_root_partition.sh to that in PR Modify autostart for NOOBS #14
4. I tar up both partitions and compress with xz
5. NOOBS will create a FAT and an ext4 partition (nominally /dev/mmcblk0p6 and /dev/mmcblk0p7)
6. NOOBS expands the compressed tar.xz files into those partitions.
7. NOOBS will adjust /boot/cmdline.txt to indicate the root is on /dev/mmcblk0p7
8. NOOBS will adjust /etc/fstab to mount the correct partitions (mmcblkp6 & 7)
9. NOOBS will now boot gentoo.

When it boots, autoexpand.... fails to set the default passwords and fails to add xdm, so I am stuck at the login prompt where the default root/demouser logins of raspberrypi64 do not work at all. (The updated script works if I simply apply it to the default image).

So I use my fork of NOOBS (PINN) to change the password of gentoo and reboot into it. Now I can login, but I find the rootfs is r/o, whcih is why the autoexpand script is failing. So something has gone wrong earlier.

Thanks for the above pointers. I will investigate those ideas to see why it is r/o. (I couldn't do sudo su --login root because I can't even login as demouser at that stage)

[sakaki-]

I have merged your PR, thanks, it'll be on the next release (which I am working on at the moment).
When you use your NOOBS fork to boot, and have the ro filesystem, are there any hints about what went wrong in dmesg?

[procount]

I finally managed to log in again, without it immediately requesting me to reset my password.
As gentoo boots it gets to the
* Checking local filesystems ... part
The boot partition checks out fine.
But then it checks the rootfs...
* Remounting root filesystem read/write ...
MOUNT: / not mounted or bad option
...
Root filesystem could not be mounted read/write

The fstab looks fine to me. I only replaced mmcblk0p2 with mmcblk0p7 for root, and similarly for boot.

/dev/mmcblk0p6          /boot   vfat    defaults                0       2
/dev/mmcblk0p7          /       ext4    defaults,noatime        0       1
/var/cache/swap/swap1   none  swap sw    0   0

The only thing I spotted in dmesg was:
[ 8.966408] EXT4-fs (mmcblk0p7): Unrecognized mount option "freely." or missing value.
I couldn't find any other logs (because /var is read-only?)

[sakaki-]

/etc/fstab looks OK, but that dmesg is odd. What is in /boot/cmdline.txt after your script has modified it?

[procount]

dwc_otg.lpm_enable=0 root=/dev/mmcblk0p7 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait

[sakaki-]

Looks OK but something may be at issue elsewhere - what does cat /proc/cmdline show once you are booted (on the ro root system)?

[procount]

8250.nr_uarts=0 cma=256M@256M dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1680 bcm2708_fb.fbheight=1050 bcm2709.boardrev=0xa02082 bcm2709.serial=0x7882aa2b smsc95xx.macaddr=B8:27:EB:82:AA:2B bcm2708_fb.fbswap=1 bcm2709.uart_clock=48000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000 dwc_otg.lpm_enable=0 root=/dev/mmcblk0p7 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait

Maybe it's the way I created the rootfs partition?
I used mkfs.ext4 with option -O ^huge_file and labeled the partition root_gen

[sakaki-]

Looks OK. Must be a filesystem issue then, since the kernel is unchanged.

Possibly you have run out of inodes? Gentoo requires more than most distros because of the from-source nature. Try running df -i (when booted to the ro system) and see what you get for IUse% on /. If full then you probably have a corrupt root filesystem. It can happen with EXT systems on small (e.g. sd-card) partitions under Gentoo (not an issue on large hard drives with the standard mkfs.ext4 settings).
As suggested here, you could try using

mkfs.ext4 -i 8192 -t ext4 /dev/mmcblk0p7

to create a filesystem with a higher inode count. Also try leaving off the huge_file option, at least initially. The partition label should not matter.

[procount]

/ has IUse%=22%
Inodes 1937712
IUsed 414794
Ifree 1522918

[sakaki-]

OK, not that then, but must be something to do with the root filesystem - the kernel is the booting OK and I assume there is nothing weird going on in config.txt, and the full (/proc/cmdline) kernel command line matches a normal startup. The dmesg output

[ 8.966408] EXT4-fs (mmcblk0p7): Unrecognized mount option "freely." or missing value.

is definitely indicating something is wrong with root. Perhaps if you try making /dev/mmcblk0p7 without huge_file?

One last thing you could try is, when the ro system comes up and you are logged in, to try to manually remount the / filesystem read-write - you might get a better diagnostic from that as to what it thinks is wrong:

pi64 ~ # mount -v -o remount,rw /

Sorry this is chewing up so much of your time! If you still can't get it to start after this then if you can send me an image of your boot and root filesystems (/dev/mmcblk0p6 and /dev/mmcblk0p7), I'd be happy to look at it this end for you.

PS I assume you have rebuilt the container image since the original-sentinel version of autoexpand_root_partition.start
ran? The beginning of that file contains:

ROOTDRIVE="/dev/mmcblk0"
ROOTPART="${ROOTDRIVE}p2"

and then it attempts to (given the original sentinel setup):

sfdisk --no-reread -N 2 <<<", +" "${ROOTDRIVE}"

and then

resize2fs -f "${ROOTPART}"

either of which is going to have a potentially nasty effect on /dev/mmcblk0p2, whatever that happens to be... (I appreciate your PR'd version doesn't do this, but the damage may already be done to the partition table, not sure whether you rebuild the full partition structure each test.)

[procount]

I already tried without Huge file - no difference. I also tried mounting root manually (with -n instead of -v) but with the same error. I will try again with -v. Actually I'm sorry this is taking up so much of YOUR time. This is normally a straightforward process. In fact I have a script to do it all for me for 16 or so different OSes. I must be missing something obvious. Rest assured I started with a clean image with SENTINEL3 setfor PR, so none of the resizing happened. I do have to keep restarting from scratch, but the installation only takes 10 mins or so. It's if I have to change my tar files from the original image that it takes the time. I appreciate the offer of looking at my images. It may come to that, but I have another couple of tricks to try first. I'll let you know how they go. (Why is my return key not working..... ;-).

[procount]

I tried -v and it still comes up with the same error. I've still got to try -i 8192 plus some other things....

Assuming I do get this to install properly, I could do with a logo for Gentoo (40x40 pixel png file) plus a few images (about 2-4 png files of size ~384x288 pixels) to be shown as a slideshow during installation, illustrating the main features/benefits of Gentoo. 😄

[sakaki-]

OK I'll see what I can put together...
Do let me know if there's anything else I can do to help debug this boot issue.

[procount]

No rush, especially as I am still struggling. I even extracted the rootfs partition from your image and DD'd it directly to /dev/mmcblk0p7 to eliminate any changes to the format / partitioning, and it was STILL r/o.

I will prepare an image for you so you can have a look at it.
All I am essentially trying to do is to move the boot and root partitions to other partition numbers on the SD card.
Can you think of any other files that might reference /dev/mmcblk0p2?
I am moving them into logical partitions. Would that make a difference? Maybe I will try locating them in different primary partitions next...

[procount]

Finally some good news - I managed to get PINN to boot gentoo64!
To do this, I installed gentoo64 to primary partitions 2&3. I'm not sure if this is the only reason, but now I have a working image, I will try installing them to logical partitions.

[procount]

And now it works on partitions 6&7 as well.
So I think it's something to do with the original image rootfs failing fsck or something. I had to do an fsck on it prior to writing it to the SD card. The rootfs seems to pass an fsck on ubuntu, but then failed it under gentoo (?) But it's looking promising at last. I just need to make the procedure repeatable!

[sakaki-]

Good news - well done.

[sakaki-]

Are there examples of the slideshow images for other O/S installs I could take a quick look at to see the kind of thing you're after?
Various formats of the Gentoo logo itself are available here.

[procount]

You can have a look here for a list of OSes.
https://sourceforge.net/projects/pinn/files/os/
In each OS folder, have a look in the slides_vga subfolder. Some are better than others :) Thanks.

[sakaki-]

Hi - haven't forgotten about this, but have been busy putting the 1.1.0 release out, which has a number of significant improvements (inclusion of packages like libreoffice thunderbird & gimp, binary kernel package support, easier updating etc.)

Would you mind trying to see if the new image works for your installer? I have migrated your PR #14 into an OpenRC service on the image (visible here) but the logic remains the same - you should be able to use autoexpand_root_none rather than autoexpand_root_partition sentinel files.

Now I have some bandwidth, I'll put some slides together for you over the next few days, with screenshots from the new image. Thanks!

[procount]

Hey, no worries! I've been busy myself. Will try the new version shortly. Hopefully it will be easier than last time :) thanks.

[sakaki-]

OK I have posted six first-cut png files here; is that the sort of thing you need? I can make the background white easily if that works better. (PS the benchmarks quoted are from these results).
I am going to be putting out a 1.1.1 release, hopefully today, addressing some minor issues with 1.1.0 (the overscan cursor alignment issue #17 being the most annoying); so it'd be worth using that version if you could.
Also, I intend switching the root (and boot) filesystem to PARTUUID naming in the cmdline.txt and /etc/fstab (I've had a request to do this for people not booting the image from a microSD card); I guess this shouldn't matter too much to you, as you modify these in your installer anyway?

[procount]

Looking good. Thanks!
And I'd just started converting v1.1.0..... :) OK. I'll hold off til the next release.
Yes, I switch mountpoint references during install and use PARTUUID if installing to a USB drive, otherwise I think it just uses the normal partition references, so it shouldn't make any difference. I'll check it when I convert v1.1.1

[sakaki-]

Hi - v.1.1.1 is out now. Things to look at that might possibly cause breakage on your side:

• cmdline.txt now has root=PARTUUID=7A0C8BB0-02 in there, in place of root=/dev/mmcblk0p2;
• similarly, the autoexpand-root service now looks up the root= entry in /proc/cmdline, and then (if a PARTUUID) canonicalizes it; however, this should only happen for the standard sentinel types; the logic should not run with an autoexpand_root_none sentinel;
• lastly, the /etc/fstab now reads (last three lines):

PARTUUID=7a0c8bb0-01	/boot		vfat    defaults          0       2
PARTUUID=7a0c8bb0-02	/		ext4    defaults,noatime  0       1
/var/cache/swap/swap1   none		swap	sw		  0	  0

Please let me know if you encounter any issues trying to get the image prepared for your installer, or if there is anything else that you need.

thanks!

[procount]

Thanks. Downloading now.

[procount]

interim report Just had a chance to test the v1.1.1 PINN installation.
It installed first time and allowed me to enter the correct password for demouser at the cmdline login prompt. However, the rootfs was r/o again, so I'll have to apply some magic....

[sakaki-]

OK, thanks. Strange about the r/o thing. I have added explicit fsck -f checking to the end of the image prepare script, so both partitions should be clean when shipped (and obviously the vanilla image boots up OK from microSD).

One other thing you could try - because the Pi has no RTC, the image uses Gentoo's swclock service; this saves the last shutdown time (in /lib64/rc/cache/shutdowntime) and restores this in early boot (eventually the chronyd NTP client sets the correct time). Large superblock time offsets (last write more than a day in the future, according to the cached shutdown time) will force an fsck to happen during boot. The edits you are making to the two partitions when loop mounted will probably have this effect.

So, you could try touching the file ${YOURLOOPMOUNT}/lib64/rc/cache/shutdowntime as the last step when editing the image, to see if this makes any difference.

[procount]

I still haven't quite got to the bottom of why it goes read-only, but I think it is something to do with the timestamps of modified files, as you suggested. If PINN is installing gentoo offline, it may not have a sense of the time either, so it uses a similar fake clock which will again be different (maybe in the past or future relative to the last real gentoo fs access time.) This maybe giving different results from when PINN is connected to a network with the real time.

However, I managed to hand-hold the installation through, and with a few manual tweaks, PINN managed to load Gentoo64 v1.1.1, from a USB boot (not SD card), complete with a nice icon and shiny slideshow!

I'm going to have to take a break from this for a while, but I'll do some more testing and make my scripts more robust to avoid this read-only problem. Then I hope to release it in a week or so (time permitting) 😄

BTW, the cursor no longer disappears when the screen blanks. I guess this is an artefact of the use of the new software cursor?

[sakaki-]

Nearly done with this now, had a few package update issues over the last few days. But should have 1.1.3 images to upload shortly.

Incidentally, I've added onboard to the repo as a simple onscreen keyboard for use with the touchscreen. If you'd like to try it out (on the current image), just do:

demouser@pi64 ~ $ sudo emaint sync --repo=rpi3
demouser@pi64 ~ $ sudo emerge -v app-accessibility/onboard

Binary packages are uploaded so it shouldn't take long to install. Start it with Applications→Accessories→Onboard

If it works OK for you (my loaner touchscreen has gone back, unfortunately) I'll add it into the release. Thanks!

[procount]

Ok I might give it a go but will need to reinstall gentoo again first. Maybe tomight or tomorrow. I could do with an onscreen keyboard in PINN too for touchscreen and CEC but it would need to work with Qt in buildroot.

…

Sent from my Samsung device

-------- Original message -------- From: sakaki <notifications@github.com> Date: 13/09/2017 01:18 (GMT+00:00) To: sakaki-/gentoo-on-rpi3-64bit <gentoo-on-rpi3-64bit@noreply.github.com> Cc: procount <kevin@procount.co.uk>, Author <author@noreply.github.com> Subject: Re: [sakaki-/gentoo-on-rpi3-64bit] Converting to NOOBS format (#13) Nearly done with this now, had a few package update issues over the last few days. But should have 1.1.3 images to upload shortly. Incidentally, I've added onboard to the repo as a simple onscreen keyboard for use with the touchscreen. If you'd like to try it out (on the current image), just do: demouser@pi64 ~ $ sudo emaint sync --repo=rpi3 demouser@pi64 ~ $ sudo emerge -v app-accessibility/onboard Binary packages are uploaded so it shouldn't take long to install. Start it with Applications→Accessories→Onboard If it works OK for you (my loaner touchscreen has gone back, unfortunately) I'll add it into the release. Thanks! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread. {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/sakaki-/gentoo-on-rpi3-64bit","title":"sakaki-/gentoo-on-rpi3-64bit","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/sakaki-/gentoo-on-rpi3-64bit"}},"updates":{"snippets":[{"icon":"PERSON","message":"@sakaki- in #13: Nearly done with this now, had a few package update issues over the last few days. But should have 1.1.3 images to upload shortly.\r\n\r\nIncidentally, I've added `onboard` to the repo as a simple onscreen keyboard for use with the touchscreen. If you'd like to try it out (on the current image), just do:\r\n```console\r\ndemouser@pi64 ~ $ sudo emaint sync --repo=rpi3\r\ndemouser@pi64 ~ $ sudo emerge -v app-accessibility/onboard\r\n```\r\nBinary packages are uploaded so it shouldn't take long to install. Start it with \u003ckbd\u003eApplications\u003c/kbd\u003e\u0026rarr;\u003ckbd\u003eAccessories\u003c/kbd\u003e\u0026rarr;\u003ckbd\u003eOnboard\u003c/kbd\u003e\r\n\r\nIf it works OK for you (my loaner touchscreen has gone back, unfortunately) I'll add it into the release. Thanks!"}],"action":{"name":"View Issue","url":"https://github.com/sakaki-/gentoo-on-rpi3-64bit/issues/13#issuecomment-329020855"}}}

[sakaki-]

The 1.1.3 releases are up now. kodi, clementine and openJDK v8 ('icedtea') are now bundled, as is onboard. twofing is also present and active by default. And the NetworkManager fix of course!
If you get a chance, please have a play with the image before releasing it via PINN.

Thanks! sakaki

[procount]

Cool! I've just been updating all my OSes and added the network manager fix to gentoo64 1.1.2 myself as a test. But I'll download 1.1.3 now and have a play.
Cheers.

[procount]

Downloaded, converted and installed with PINN. It all seems to work, although my touchscreen mouse was very unreliable, often clicking in the wrong place. How can I disable/enable TwoFing for testing?

[sakaki-]

You should just be able to disable it via Applications→Settings→Session and Startup→Application Autostart tab; untick the "Start twofing gesture daemon" entry and reboot.

Strange though, it didn't seem to have a accuracy problem for me, but perhaps something has changed due to the latest release fixes. I'll see if I can get the loaner system back and retest here over the next few days. Are you running with the screen rotated (lcd_rotate=2 in /boot/config.txt)?

[procount]

Thanks.
I only did a quick test last night and have not eliminated everything yet.
No, screen is not rotated.

[sakaki-]

When I tried it, I did have lcd_rotate=2 set.

[procount]

It seems to work fine tonight. Maybe it was a bad connection or something. 🤷‍♂️

[sakaki-]

Great, looks like we're dangerously close to closing this issue ^-^

Sorry it's eaten up so many of your cycles getting the image available on PINN! (but Gentoo tends to... do that)

One last thought: it might be an idea to post an announcement about the installer on the sticky Gentoo forums thread for this project, since there are (obviously) quite a few potential users there (who might like to try some of your other rpi3 images too)?

[procount]

I'm just finishing updating the other 20ish OSes in between testing my next version of PINN and updating the docs. Then I'll release and announce.

Thanks for the ride and your support along the way. It's been fun and I know a little bit more about gentoo now too!

[procount]

Finally uploaded all 28 OSes. Just got to check them all now....
How's this for a proposed annoucement:

The latest version v1.1.3 of Gentoo64 and Gentoo64pt (for the PiTop) are now installable via the PINN multi-boot manager.

What is PINN?
PINN is a fork of the RaspberryPi NOOBS installer that includes some additional advanced features.
It allows easy installation of many different operating systems using a FAT32 formatted SD card, so it is easy to use for beginners coming from Windows, or more advanced users alike.
It can install several operating systems to the same SD card or USB drive and includes a boot manager to select which OS will be executed on boot.
Operating systems can be downloaded from the internet, or for offline use they can be installed from the SD card or a separate USB stick.

[sakaki-]

Looks good! If posting to the Gentoo forums, you might add a link (usual bb-code [url=https://foo]text[/url] syntax) to the PINN sourceforge page, and also a link to the PINN source on GitHub would probably be appreciated.

[sakaki-]

Saw your Gentoo forums post, looks great. Just one thing:

Many thanks to Sakaki for his support in helping me convert Gentoo64 to the PINN environment.

I'm a she not a he (but I'll take it as a compliment ^-^)

[procount]

Oops! Sorry, I'll fix it. Or would you prefer 'their' to be gender neutral? I guess your avatar is more life-like than I had assumed.

It took me a while because I thought I was already registered. So, many failed logins later I registered again, activated my email and immediately got banned! It seems my email address was caught in their spam filter so I had to jump through a few hoops to get unbanned.

[sakaki-]

No worries - "she/her" is fine by me.

[procount]

Already done 😄

Neddy suggested I move my bit about PINN to a separate topic and link back to your thread, so I may do some adjustments. I think he is concerned about any PINN chat filling up your thread...

BTW, I'm nearly ready to release the next version of PINN. Most testing is complete now. I just have to figure out how I can simulate a mouse press on another desktop application so I can improve my HDMI CEC interaction....

[sakaki-]

Closing now. I'll put a reference to PINN on the main page for those who might prefer to use it rather than writing the image directly. Best, sakaki

[sakaki-]

Hi, a v1.2.0 of the image has just been released. This is in response to Gentoo changing their 'profile' (global build settings) to have gcc build position independent executables by default. If you get a chance it'd be good to get this new image (for regular and Pi-Top RPi3s) onto PINN.

[procount]

Ok. I'll find some time to update it. Thanks.

[sakaki-]

Perfect, many thanks.

[procount]

Done!

[sakaki-]

Wow that was fast! Thanks ^-^

[sakaki-]

For these new images, did you by any chance pull the files from https://isshoni.org/pinn/? Just realized that directory still holds the old version tarballs ><

[procount]

No. I wasn't aware of that url. Did you create them yourself or are they copies of mine?
I created my images directly from https://github.com/sakaki-/gentoo-on-rpi3-64bit/releases/tag/v1.2.0, mostly automated now, apart from having to update the release notes.

[sakaki-]

Thanks, that's great. Just noticed that the old files were up there when doing a clean-up of the web server, thought I might have sent the URL to you at some point. But if you are creating the images directly from the release tarballs, then no problem.

If I could ask for one minor tweak: the version number has rolled from 1.1.x (the 13.0 profile releases) to 1.2.x (the 17.0 profile releases), and this affects the first slide of the marketing deck. As this might confuse people (given the accompanying release notes, forum posts etc.), would it be possible to upload the new Slide1.png, which has the correct version? The updated slide is available here. The other 5 slides are fine to stay as is. Thanks again ^-^

[procount]

Yes, no problem.
If you ever want to host any of these meta-files (so you can update them with new releases, like the slides or release notes), just let me know and I can point the urls to your site instead.

[procount]

Slides should be updated now.

[sakaki-]

Many thanks for that.
I'll look at moving the hosting for the meta files to isshoni.org next time I refresh the server; I'll drop you a line when I do.