Patch/rocko3 #75

andi8086 · 2018-01-30T11:31:34Z

No description provided.

The npm class of rocko tries to use "npm install -g" but stumbles over --no-registry. Install our own method, which is the one morty used, and switch our recipes to it. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Rocko switched to 4.12 for the headers, but we don't want to pull another kernel source tree. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Use normal install and drop rsync which also wasn't properly listed as build-time dependency. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

This lifts the BSP layer to Yocto 2.4 (rocko). Moving to that version allows (and requires) us to drop the initrd installation workaround. This temporarily breaks the example layer. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Enable the example image over (upcoming) Yocto 2.4.1. This brings a number of changes: - meta-intel-iot-middleware is dead; mraa and upm were moved to meta-oe - we can drop bbappends for mraa and upm due to the newer versions in meta-oe - the new mraa recipe requires to ask for nodejs binding and the installation of mraa-utils explicitly - the newer version of mraa provides now Python 3 bindings (because that's the primary Python version on the target) - without Python 2 mraa, we need to port iot2000setup and setledcolor to Python 3 - we can drop our Python 2 libnewt due to this - changes are required to uclibc recipes due to new recipe-specific sysroots of poky Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Allow color names as well. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Already a long time ago, poky stopped pulling in grub this way. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

While this will not bring us all the I/O of the IOT2000, we can still use QEMU to test basic image features. Set the required QB variables in the machine config, specifically defining the RAM size (1G, that of the IOT2040) and ensuring that a second UART is present which the image uses as console. We also need a virtio-rng device due to lacking entropy in the virtual target. Tune the kernel config to carry virtio block, net, and hw-random support. As compensation, we can safely drop hw-random drivers that make no sense on the IOT2000. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Change iot2000-example-image to include meta-efibootguard in its kas file. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

In machine config set EFI_PROVIDER to efibootguard. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

For wic to create a GPT, gptfdisk is required. Make the image depend on it. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Tell wic that we want a GPT. The partition layout must have an EFI partition, two config partitions, two root partitions and a persistent partition for update images at least. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Import official meta-swupdate layer. meta-swupdate needs some minor changes to be compatible with our update process and configuration. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add needed variables to WICVARS via a local.conf entry to have this variable available in the wic source plugins. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Efibootguard's tool is needed to create the config partition Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add bg_setenv/bg_readenv and swupdate to the example image. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

cpio creation in swupdate.bbclass needs an ext4.gz image in the image deploy dir. This must be added before the live-image.bbclass is inherited otherwise the live-image mechanism breaks. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add sw-description as an update example and an example recipe to build an update image. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add a kas yml to generate a base image with an update artifact with the CIP RT kernel. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

do_image_wic requires files that do_bootimg generates. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

* Add error handling. * Add logging. * Do not assume we only have three partitions. Always expand the last partition of the root device. * Do not delete and re-create but use resize command of parted. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add information about how swupdate is integrated and explain the example recipes. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

jan-kiszka · 2018-03-05T07:31:28Z

I suppose I now want to try out patch/rocko4 rather than this version, right?

jan-kiszka

Who needs this? Not described in the commit log.

Also, can't we define this inside some config file that is related to the user of that feature and is pulled into the example image?

jan-kiszka

We should rather make sure that the update image is building into ext4.gz and is NOT using wic.

jan-kiszka

This commit only contains the -rt kas file. Either move the non-rt one from the previous patch or fold both patches into one.

jan-kiszka

Missing reason.

Note that oe-core removed the dependency in http://git.openembedded.org/openembedded-core/commit/?id=af52d878940babb88bc8a2650e8c0115b6ccaf9f

jan-kiszka · 2018-03-05T08:08:49Z

Sorry, for duplicate comments, but github is highly counterintuitive /wrt how to comment on a particular commit in a PR.

jan-kiszka · 2018-03-05T08:14:49Z

OK, rocko4 is broken due to changes in there referenced swupdate repo (hash no longer present). Sticking with this series.

jan-kiszka · 2018-03-05T08:49:47Z

meta-iot2000-example/scripts/lib/wic/canned-wks/example-image.iot2000.wks

 part swap --ondisk mmcblk0 --size 512 --fstype=swap --label swap --align 1024
+part --size 1024 --extra-space 0 --overhead-factor 1 --ondisk mmcblk0 --label persistent --align 1024 --fstype=ext4


This leaves the expansion space unusable for the normal user. It should be mounted, maybe as home partition or /persistent.

jan-kiszka · 2018-03-05T08:54:39Z

meta-iot2000-example/scripts/lib/wic/canned-wks/example-image.iot2000.wks

@@ -1,3 +1,14 @@
+# short-description: Create an Galileo Gen 1/2 disk image
+# long-description: Creates a partitioned EFI disk image for Intel Galileo Gen 1/2,


This is not a Galileo.

jan-kiszka · 2018-03-05T09:45:19Z

We probably want to disable the resizer script in the update image. I suspect it stumbles while calling parted, causing a help dump of that tool during first boot after the update.

jan-kiszka · 2018-03-05T09:47:34Z

meta-iot2000-example/README.SWUPDATE.md

+```
+swupdate -i <filepath>.swu
+```
+


Then I need reboot (swupdate tells me, but better state this again). But how do I declare the update successful after the reboot? Without that, I will fall back into v1 again.

bg_setenv -c is what I want, I suppose.

jan-kiszka · 2018-03-05T12:01:02Z

Further meta comment: I noticed that the Linux watchdog daemon starts early and does not monitor if we manage to get to the final startup script. We probably want a simple "validate-update.sh" script that runs last, starts the watchdog and also confirms the update (if that hasn't happened before). That script could carry some comment like "add your custom validation steps here" so that users know where to put further tests for connectivity or the execution of critical services.

andi8086 · 2018-03-05T12:36:16Z

On Mon, Mar 05, 2018 at 12:01:03PM +0000, Jan Kiszka wrote: Further meta comment: I noticed that the Linux watchdog daemon starts early and does not monitor if we manage to get to the final startup script. We probably want a simple "validate-update.sh" script that runs last, starts the watchdog and also confirms the update (if that hasn't happened before). That script could carry some comment like "add your custom validation steps here" so that users know where to put further tests for connectivity or the execution of critical services.

Be aware that the hardware watchdog on the quark is limited to 133 seconds. This is quickly too little time if waiting for the latest init script thus it might not be possible to guard the whole boot process, but only the entry into user space for sure. Especially the huge delay for random pool initialization might prevent this.

…

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.*

-- Andreas Reichel Dipl.-Phys. (Univ.) Software Consultant Andreas.Reichel@tngtech.com +49-174-3180074 TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring Geschäftsführer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Müller Sitz: Unterföhring * Amtsgericht München * HRB 135082

jan-kiszka · 2018-03-05T12:49:24Z

The random pool issue - at least /wrt tcf-agent - is fixed in current jan/rocko (your baseline is older).

If the watchdog is limited in duration, we can add single triggers along the boot process. But the daemon shall not start that early.

jan-kiszka · 2018-03-06T06:49:16Z

meta-iot2000-example/recipes-example/iot2000setup/files/expandfs.sh

 	exit 1
 	;;
 esac

-START_BLOCK=$(parted $ROOT_DEVICE -ms unit s p | grep "^$PART_NUMBER" | cut -f 2 -d: | sed "s/s$//")
+log "Fixing backup GPT position"
+parted ${ROOT_DEVICE} print Fix


Left over debug print? But "Fix" is not a known parameter to parted.

jan-kiszka · 2018-03-06T06:58:03Z

meta-iot2000-example/recipes-example/iot2000setup/files/expandfs.sh

+partprobe ${ROOT_DEVICE}
+
+log "Resizing file system on ${EXPAND_PARTITION}"
+resize2fs ${EXPAND_PARTITION}


For whatever reason, this just refused to work:

rc: Resizing /dev/mmcblk0p7 to maximum rc: Resizing file system on /dev/mmcblk0p7 resize2fs 1.43.5 (04-Aug-2017) Please run 'e2fsck -f /dev/mmcblk0p7' first.

Adding that check here seems reasonable.

jan-kiszka · 2018-03-06T07:07:46Z

meta-iot2000-example/recipes-example/iot2000setup/files/expandfs.sh


-update-rc.d -f expandfs.sh remove
+disable


BTW, not a regression: parted also gets stuck on lacking entropy. Possibly because it pulls in some uuid lib which first of all wants to initialized some randomness.

I'm checking if we can avoid that. Alternatively, we could send the whole thing into background so that the first boot is not delayed. At least we need to think about the watchdog in case we no longer start it prematurely during early boot.

jan-kiszka · 2018-03-06T07:49:27Z

meta-iot2000-example/scripts/lib/wic/canned-wks/example-image.iot2000.wks


-include common.iot2000.wks.inc


And now we are missing the initrd with ACPI fixups. While I hope we can overcome that soon, we require it for now. I suppose that explains why you patches didn't need some of the fixes of Farhat...

andi8086 · 2018-03-06T10:16:23Z

On Mon, Mar 05, 2018 at 11:49:29PM -0800, Jan Kiszka wrote: @jan-kiszka commented on this pull request. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ In meta-iot2000-example/scripts/lib/wic/canned-wks/example-image.iot2000.wks: > -include common.iot2000.wks.inc And now we are missing the initrd with ACPI fixups. While I hope we can overcome that soon, we require it for now. I suppose that explains why you patches didn't need some of the fixes of Farhat... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.*

Sorry, I must have overlooked the initrd, since our yocto builds never had one. But that can easily be fixed by just appending initrd= and let it load by the kernel. I will fix all that you have found of course.

…

-- Andreas Reichel Dipl.-Phys. (Univ.) Software Consultant Andreas.Reichel@tngtech.com +49-174-3180074 TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring Geschäftsführer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Müller Sitz: Unterföhring * Amtsgericht München * HRB 135082

andi8086 · 2018-03-06T10:19:29Z

On Tue, Mar 06, 2018 at 06:49:19AM +0000, Jan Kiszka wrote: @jan-kiszka commented on this pull request. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ In meta-iot2000-example/recipes-example/iot2000setup/files/expandfs.sh: > exit 1 ;; esac -START_BLOCK=$(parted $ROOT_DEVICE -ms unit s p | grep "^$PART_NUMBER" | cut -f 2 -d: | sed "s/s$//") +log "Fixing backup GPT position" +parted ${ROOT_DEVICE} print Fix Left over debug print? But "Fix" is not a known parameter to parted.

Wrong, this is needed! This is the only way, you can answer the question of parted if it shoul dfix the wrongly aligned GPT backup copy. This is also no debug print, the Fix question only arises when you print it before... It took me a long time to figure that out.

…

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.*

-- Andreas Reichel Dipl.-Phys. (Univ.) Software Consultant Andreas.Reichel@tngtech.com +49-174-3180074 TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring Geschäftsführer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Müller Sitz: Unterföhring * Amtsgericht München * HRB 135082

andi8086 · 2018-03-06T10:34:11Z

On 2018-03-06 11:19, Andreas Reichel wrote: On Tue, Mar 06, 2018 at 06:49:19AM +0000, Jan Kiszka wrote: > @jan-kiszka commented on this pull request. > > ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ > > In meta-iot2000-example/recipes-example/iot2000setup/files/expandfs.sh: > >> exit 1 > ;; > esac > > -START_BLOCK=$(parted $ROOT_DEVICE -ms unit s p | grep "^$PART_NUMBER" | cut -f 2 -d: | sed "s/s$//") > +log "Fixing backup GPT position" > +parted ${ROOT_DEVICE} print Fix > > Left over debug print? But "Fix" is not a known parameter to parted. > Wrong, this is needed! This is the only way, you can answer the question of parted if it shoul dfix the wrongly aligned GPT backup copy. This is also no debug print, the Fix question only arises when you print it before... It took me a long time to figure that out.

Is there no proper non-interactive tool that can fix the issue without requiring this hack? Or doesn't "echo Fix | gparted ..." work? Then at least silence this messy output (>/dev/null), specifically when there is nothing to fix anymore. Jan

franzhoepfinger · 2018-04-07T05:32:12Z

How can i build this pull request to Test?

franzhoepfinger · 2018-04-07T05:32:39Z

When do you expect this Request to be integrated?

jan-kiszka · 2018-04-09T07:07:48Z

We are just releasing 2.2.0 and will then refresh and merge the rocko support. This PR depends on rocko being upstream first. There are also a couple of smaller issues to be addressed with a follow-up PR, and Andreas will need some time to work on them, so don't expect a quick resolution here.

Regarding test builds: There is a readme included, and you basically just need to build the example image with the known kas file. For update images, there is currently a separate kas file (one of the smaller things I'd like to change prior to merging).

jan-kiszka · 2018-11-02T12:06:04Z

Superseded by #102.

jan-kiszka and others added 22 commits January 4, 2018 17:26

example: Prepare for installing npms manually

1c8da62

The npm class of rocko tries to use "npm install -g" but stumbles over --no-registry. Install our own method, which is the one morty used, and switch our recipes to it. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

example: Provide own linux-libc-headers_4.4 recipe

fcad25e

Rocko switched to 4.12 for the headers, but we don't want to pull another kernel source tree. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

example: Avoid rsync dependency in galileo-target installation

1b3f5a8

Use normal install and drop rsync which also wasn't properly listed as build-time dependency. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

bsp: Port to rocko

718b5cf

This lifts the BSP layer to Yocto 2.4 (rocko). Moving to that version allows (and requires) us to drop the initrd installation workaround. This temporarily breaks the example layer. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

example: Rework setledcolor command line interface

e7c44d9

Allow color names as well. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

bsp: Remove obsolete MACHINE_ESSENTIAL_EXTRA_RDEPENDS override

8823915

Already a long time ago, poky stopped pulling in grub this way. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>

Add meta-efibootguard to kas file

870425a

Change iot2000-example-image to include meta-efibootguard in its kas file. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

meta-iot2000-bsp: Change EFI_PROVIDER

8c00033

In machine config set EFI_PROVIDER to efibootguard. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add gptfdisk for wic

9bbf171

For wic to create a GPT, gptfdisk is required. Make the image depend on it. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Change partition layout and bootloader params

59ef398

Tell wic that we want a GPT. The partition layout must have an EFI partition, two config partitions, two root partitions and a persistent partition for update images at least. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Use official meta-swupdate layer

3c9f280

Import official meta-swupdate layer. meta-swupdate needs some minor changes to be compatible with our update process and configuration. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

wic: Export needed bitbake variables to wic

cf26973

Add needed variables to WICVARS via a local.conf entry to have this variable available in the wic source plugins. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add native bg_setenv to the image dependencies

872ca86

Efibootguard's tool is needed to create the config partition Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add efibootguard tools and swupdate to example image

1abad60

Add bg_setenv/bg_readenv and swupdate to the example image. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add update-image recipe and sw-description

6947b47

Add sw-description as an update example and an example recipe to build an update image. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add kas yml for update image generation

b649176

Add a kas yml to generate a base image with an update artifact with the CIP RT kernel. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Be sure do_bootimg ends before do_image_wic starts

1038f13

do_image_wic requires files that do_bootimg generates. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

Add some information about swupdate

4b90604

Add information about how swupdate is integrated and explain the example recipes. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>

andi8086 force-pushed the patch/rocko3 branch from b1e7234 to 4b90604 Compare February 7, 2018 14:44

jan-kiszka requested changes Mar 5, 2018

View reviewed changes

jan-kiszka reviewed Mar 5, 2018

View reviewed changes

jan-kiszka reviewed Mar 6, 2018

View reviewed changes

jan-kiszka mentioned this pull request Oct 17, 2018

Ubuntu Core image? #101

Closed

jan-kiszka closed this Nov 2, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patch/rocko3 #75

Patch/rocko3 #75

andi8086 commented Jan 30, 2018

jan-kiszka commented Mar 5, 2018

jan-kiszka left a comment

jan-kiszka left a comment

jan-kiszka left a comment •

edited

Loading

jan-kiszka left a comment

jan-kiszka commented Mar 5, 2018

jan-kiszka commented Mar 5, 2018

jan-kiszka Mar 5, 2018 •

edited

Loading

jan-kiszka Mar 5, 2018

jan-kiszka commented Mar 5, 2018

jan-kiszka Mar 5, 2018 •

edited

Loading

jan-kiszka commented Mar 5, 2018

andi8086 commented Mar 5, 2018 via email

jan-kiszka commented Mar 5, 2018

jan-kiszka Mar 6, 2018

jan-kiszka Mar 6, 2018 •

edited

Loading

jan-kiszka Mar 6, 2018

jan-kiszka Mar 6, 2018

andi8086 commented Mar 6, 2018 via email

andi8086 commented Mar 6, 2018 via email

andi8086 commented Mar 6, 2018 via email

franzhoepfinger commented Apr 7, 2018

franzhoepfinger commented Apr 7, 2018

jan-kiszka commented Apr 9, 2018

jan-kiszka commented Nov 2, 2018

		part swap --ondisk mmcblk0 --size 512 --fstype=swap --label swap --align 1024
		part --size 1024 --extra-space 0 --overhead-factor 1 --ondisk mmcblk0 --label persistent --align 1024 --fstype=ext4

		@@ -1,3 +1,14 @@
		# short-description: Create an Galileo Gen 1/2 disk image
		# long-description: Creates a partitioned EFI disk image for Intel Galileo Gen 1/2,

Patch/rocko3 #75

Patch/rocko3 #75

Conversation

andi8086 commented Jan 30, 2018

jan-kiszka commented Mar 5, 2018

jan-kiszka left a comment

Choose a reason for hiding this comment

jan-kiszka left a comment

Choose a reason for hiding this comment

jan-kiszka left a comment • edited Loading

Choose a reason for hiding this comment

jan-kiszka left a comment

Choose a reason for hiding this comment

jan-kiszka commented Mar 5, 2018

jan-kiszka commented Mar 5, 2018

jan-kiszka Mar 5, 2018 • edited Loading

Choose a reason for hiding this comment

jan-kiszka Mar 5, 2018

Choose a reason for hiding this comment

jan-kiszka commented Mar 5, 2018

jan-kiszka Mar 5, 2018 • edited Loading

Choose a reason for hiding this comment

jan-kiszka commented Mar 5, 2018

andi8086 commented Mar 5, 2018 via email

jan-kiszka commented Mar 5, 2018

jan-kiszka Mar 6, 2018

Choose a reason for hiding this comment

jan-kiszka Mar 6, 2018 • edited Loading

Choose a reason for hiding this comment

jan-kiszka Mar 6, 2018

Choose a reason for hiding this comment

jan-kiszka Mar 6, 2018

Choose a reason for hiding this comment

andi8086 commented Mar 6, 2018 via email

andi8086 commented Mar 6, 2018 via email

andi8086 commented Mar 6, 2018 via email

franzhoepfinger commented Apr 7, 2018

franzhoepfinger commented Apr 7, 2018

jan-kiszka commented Apr 9, 2018

jan-kiszka commented Nov 2, 2018

jan-kiszka left a comment •

edited

Loading

jan-kiszka Mar 5, 2018 •

edited

Loading

jan-kiszka Mar 5, 2018 •

edited

Loading

jan-kiszka Mar 6, 2018 •

edited

Loading