Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for Yocto 5.0 "scarthgap" #2099

Draft
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

TheYoctoJester
Copy link
Contributor

This is a proof of concept to prepare for compatibility with the Yocto 5.0 release scarthgap. It is based on the prepare_master_for_scarthgap branch and not fit for direct merge, but provides a known good starting point.

Changelog: Title
Ticket: MEND-7157

@kacf
Copy link
Member

kacf commented Apr 22, 2024

Started an "early feedback" CI run here.

@kacf
Copy link
Member

kacf commented Apr 23, 2024

Started another one here (first one failed because of operator error 🙂).

@kacf
Copy link
Member

kacf commented Apr 24, 2024

And yet another one here.

@kacf
Copy link
Member

kacf commented Apr 25, 2024

@TheYoctoJester: The error I'm now getting in the CI is of this nature:

ERROR: /builds/Northern.tech/Mender/meta/recipes-core/packagegroups/packagegroup-rust-cross-canadian.bb: Please ensure that your setting of VIRTUAL-RUNTIME_init_manager (systemd) matches the entries enabled in DISTRO_FEATURES

Does it ring a bell for you? We do have this section, which looks related, but it should be active in your build as well, so I don't know why you don't see it.

Are you using INHERIT += "mender-full" in your build?

@TheYoctoJester
Copy link
Contributor Author

@kacf strange. I have systemd enabled (https://github.com/mendersoftware/meta-mender-community/blob/scarthgap/kas/include/mender-base.yml#L31), and INHERIT += mender-full for all non-UBI cases (https://github.com/mendersoftware/meta-mender-community/blob/scarthgap/kas/include/mender-full.yml#L8).

Is there a way to see the effective DISTRO_FEATURES on that build?

@ColinFinck
Copy link
Contributor

For the past few days, I have been successfully running Mender on Yocto Scarthgap with my branch at https://github.com/enlyze/meta-mender/tree/feature/scarthgap-support-for-master

Feel free to cherry-pick what is still missing.
I particularly recommend enlyze@bb75c83 or smooth upgrades from Yocto 3.1 Dunfell to Yocto 5.0 Scarthgap won't be possible.

@ColinFinck
Copy link
Contributor

@TheYoctoJester:

Is there a way to see the effective DISTRO_FEATURES on that build?

You are probably looking for running bitbake on your target image with the -e flag. That will print a full list of variables, along with their history and their final value. Among them is DISTRO_FEATURES

@TheYoctoJester
Copy link
Contributor Author

@TheYoctoJester:

Is there a way to see the effective DISTRO_FEATURES on that build?

You are probably looking for running bitbake on your target image with the -e flag. That will print a full list of variables, along with their history and their final value. Among them is DISTRO_FEATURES

@ColinFinck heh yeah that one is obvious. The problem is, it works in my builds but seems to fail in the automated test runs that the team runs on GitLab. So the question is about seeing the effective DISTRO_FEATURES there, where the build is not interactively accessible.

@kacf
Copy link
Member

kacf commented May 2, 2024

I think I found the issue here, our build files are using the old _append syntax for this specific assignment. I guess it still worked on kirkstone as a backwards compatibility thing, but no more on scarthgap. I don't have time now, but I'll try another build tomorrow after fixing it.

@kacf
Copy link
Member

kacf commented May 3, 2024

I think I found the issue here, our build files are using the old _append syntax for this specific assignment. I guess it still worked on kirkstone as a backwards compatibility thing, but no more on scarthgap. I don't have time now, but I'll try another build tomorrow after fixing it.

Nope, I was way off here, we are using the right syntax.

But in any case, I did figure out the reason, and I took the liberty of pushing to this branch. @TheYoctoJester, can you check the commit I added, which has some implications, and see if you agree with my reasoning?

@kacf
Copy link
Member

kacf commented May 3, 2024

New full CI pipeline here.

@TheYoctoJester
Copy link
Contributor Author

But in any case, I did figure out the reason, and I took the liberty of pushing to this branch. @TheYoctoJester, can you check the commit I added, which has some implications, and see if you agree with my reasoning?

Hi @kacf, interesting catch. From the pure Yocto point of view, this approach is definitely to be preferred, as it requires users to make conscious decisions instead of "hidden" magic.

TheYoctoJester and others added 12 commits May 3, 2024 11:43
When building from git master, it is necessary to also fetch
the vendored submodules. Switching the recipe to the gitsm://
fetcher fixes this process.

Changelog: Title
Ticket: None

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
…psule

The existing mender artifact class can only be used for generating
system/rootfs update artifacts. For the UEFI capsule, we need a custom
module image artifact with specific flags/conditions enabled

These are the following changes:
* Add new class, this has a dependency on the UEFI capsule class from
  other layers which generates the required UEFI firmware capsule
* Add new variables
* Generate the `provides` and `depends` in the below format:
  `uefi-firmware.<guid>.name:<name>`
  `uefi-firmware.<guid>.version:<version>`
  by using `MENDER_ARTIFACT_CAPSULE_PROVIDES` and
  `MENDER_ARTIFACT_CAPSULE_DEPENDS` variables

Changelog: Title
Ticket: None

Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
Signed-off-by: Vikas Katariya <vikas.katariya@arm.com>
Changelog: None
Ticket: None

Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
Changelog: Title
Ticket: None

Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
Signed-off-by: Vikas Katariya <vikas.katariya@arm.com>
Bump the `meta-mender-core` and `meta-mender-demo` compatibility to
`langdale`

Signed-off-by: Vikas Katariya <vikas.katariya@arm.com>
To facilitate ongoing build testing on the master/main branches,
release compatibility needs to be declared.

Changelog: Title
Ticket: None
Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
Recent bitbake/Yocto releases check for the Upstream-Status
of all patches they apply, in order to track them.
The Mender-specific patches are considered inappropriate
for U-Boot upstream and annotated as such.

Changelog: Title
Ticket: None

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
The generic bootlimit patch was labeled as experimental
and not being used anywhere, so can be removed for moving forwards.

Changelog: Title
Ticket: None

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
This is a refresh of the u-boot board integration patch
for the Raspberry Pi family of boards.

Changelog: Title
Ticket: None

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
The patch which added a header to fix build breakage is not
required anymore, as upstream has been fixed.
The other u-boot patches have been renamed accordingly.

Changelog: Title
Ticket: None

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
Changelog: Title
Ticket: None

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
Adjust the LAYERSERIES_COMPAT of all layer.conf files to
scarthgap. Where multiple releases were supported, those are
removed.

Changelog: Title
Ticket: None

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
TheYoctoJester and others added 2 commits May 3, 2024 11:52
The patch is refreshed to the 2024.01 release of u-boot, including
upstream state set to Inappropriate.

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
PreetiSachan and others added 4 commits May 3, 2024 12:22
Setting hardcoded path "/lib/systemd/system" to
SYSTEMD_UNIT_DIR variable, is causing following failure:
| do_package: Didn't find service unit 'mender-updated.service',
| specified in SYSTEMD_SERVICE:mender-update

Set SYSTEMD_UNIT_DIR to bitbake env variable ${systemd_system_unitdir}.
When usrmerge DISTRO_FEATURE is enabled, ${root_prefix} points to
${exec_prefix} otherwise to ${base_prefix}.

Ref:
https://git.openembedded.org/openembedded-core/commit/?id=700848c6ebd03bf3105d09a41d758883ab875618

Signed-off-by: Preeti Sachan <preeti.sachan@intel.com>
(cherry picked from commit 12a5b99)
The existing setting "/bin/bash" breaks with the usrmerge of
the scarthgap release, where bash is located at /usr/bin/bash.

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
The meta-mender-raspberrypi and meta-mender-raspberrypi-demo
layers are move to the meta-mender-community repository to
decouple board integration from Mender product development
and maintenance.

Changelog: Title
Ticket: None

Signed-off-by: Josef Holzmayr <jester@theyoctojester.info>
It does not look like it's possible to do it safely anymore. The old
approach with `DISTRO_FEATURES_BACKFILL` will not work on scarthgap
and later, because there is a separate variable `INIT_MANAGER`, which,
if set to `sysvinit` (the fault), it starts competing with us, using
`DISTRO_FEATURES_BACKFILL_CONSIDERED`.

The next logical step would then be to use `INIT_MANAGER` instead, but
this does not work either because it eventually leads to loading
`init-manager-${INIT_MANAGER}.inc`. This particular step is sensitive
to load ordering, and therefore our assignment comes to late to have
any effect, leading to the `sysvinit` variant being loaded, even if we
have set it to `systemd`. The mentioned load step comes from
`bitbake.conf`, and the only files loaded before that are `local.conf`
and `layer.conf` files. We can not (reasonably) utilize any of these
files for what we want to do, because feature selection comes much
later in the load order.

So we pick the safest variant, and ask the user to set their
`INIT_MANAGER` variable correctly instead. We could have made it
adaptive instead, and simply disabled the `mender-systemd` feature if
that `INIT_MANAGER` is not used. But since we have poor support for
this, and it's likely to break daemon mode, default to an error
message instead. `mender-systemd` can still be disabled manually,
which will get rid of the message.

Changelog: Enabling systemd as init manager is no longer done
automatically by the meta-mender layer. In order to re-enable it, put
`INIT_MANAGER = "systemd"` in your `local.conf` file. Alternatively,
use `MENDER_FEATURES_DISABLE = "mender-systemd"`, but be aware that
this can break daemon mode.

Ticket: None

Signed-off-by: Kristian Amlie <kristian.amlie@northern.tech>
@kacf
Copy link
Member

kacf commented May 3, 2024

Yet another one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants