Introduction of board and SoC scheme v2.

[tejlmand]

This PR introduces a board and SoC scheme v2.

Fixes #51831

The benefits of board and SoC scheme v2 are:

• Inherit BOARD setting from CMake
  • boards: set CONFIG_BOARD default based on devicetree #49260 (comment)
  • Kconfig: add CONFIG_BOARD_REVISION #44654 (comment)
  • This aligns how BOARD is passed from CMake --> Kconfig to the way BOARD_REVISION is handled
  • this removes any inconsistency in names
  • Removes the need to keep names in sync, and updating them in two places.
• Remove the need for prompts on SoCs
  • In zephyr/main today then the SoC is set in a config file instead of being selected directly by the board.
    This can be confusing to new-comers, as they might wonder if an SoC can be changed because all SoCs in Zephyr appears in a choice menu in menuconfig
    Changing an SoC without a pristine build is not supported or tested, so no reason to expose such options directly to users.
• Board v2 uses select of SoC instead of a configuration file
  • This makes it clearer and conforming to rest of Kconfig tree, where a given setting will generally select other settings that it requires and not expect all those settings being defined in config files.
• The SoC v2 Kconfig tree has specific rules that can be described and thus tested in compliance (pending task)
• The SoC v2 Kconfig tree is independent, allowing it to be re-used, for example by sysbuild
  • Re-usability of SoC tree outside Zephyr build system #43488.

Note, there is no requirement for existing boards or SoCs to be updated to v2, but if a board and SoC want to benefit from above, then they can of course be updated.

• Create board v2 infrastructure in CMake and Kconfig
• Create SoC v2 infrastructure in CMake and Kconfig
• Document board v2 requirements
• Document SoC v2 requirements
• Update compliance check to verify a board / SoC v2 conforms to the rules

Draft PR until all above requirements has been implemented.

[andyross]

Still outside my area of expertise, but this scheme I like much better than N-thousand-line PRs!

[mbolivar-nordic]

@tejlmand it would be nice if "board scheme v2" didn't require a boards/ARCH/boardname hierarchy so that we wouldn't have to hack around the zephyr file system standard for targets with multiple architectures on a single SoC/PCB.

[tejlmand]

@tejlmand it would be nice if "board scheme v2" didn't require a boards/ARCH/boardname hierarchy so that we wouldn't have to hack around the zephyr file system standard for targets with multiple architectures on a single PCB.

A very good point indeed, and I assume same comment applies to SoC tree.
At first I tried to stay close to current scheme, but with the v2, reconsidering the arch hierarchy would make a lot of sense.

To others reading this, please give a 👍 if you think scheme v2 should actually get rid of ARCH as suggested by @mbolivar-nordic.

This would especially make sense for boards with multiple SoCs of different archs, or multicore SoCs where different cores might be of different archs.

[mbolivar-nordic]

I assume same comment applies to SoC tree.

It does.

[nashif]

for those who have been long enough here, you will remember the days we had to pass ARCH and BOARD to make to build a board. I personally have no issues with that, although it was always easier to find the boards I was looking for, given that some of us are invested in more than one architecture and not necessarily ARM.
Let me undersdtand though, how is that going to change anything for multi-arch platforms? I mean both the SOC and BOARD would still be tied to an architecture, is the intention here to have a 2 cores of different architectures in the same silicon be on the same level? Is it only about having one level less and a shorter path or are you thinking about something else supporting this type of hardware?

[mbolivar-nordic]

both the SOC and BOARD would still be tied to an architecture

No, that's what I'm saying should be changed for the v2 hardware scheme. We have known this has been a problem for years:

0d811b9#diff-cc856e9b347fddf6d5a472d1330e1e71e805aee2e66a48b9251b528af7f390f8R290

Decoupling "board" and "soc" from "arch" is exactly what I am suggesting. An SoC can have CPU clusters with as many different architectures as the vendor wants to cram in there, so any target board can too.

[mbolivar-nordic]

0d811b9#diff-cc856e9b347fddf6d5a472d1330e1e71e805aee2e66a48b9251b528af7f390f8R290

Link is not working well; I'm trying to reference this part of the old getting started:

.. [#board_misnomer]
   This has become something of a misnomer over time. While the target can be,
   and often is, a microprocessor running on its own dedicated hardware
   board, Zephyr also supports using QEMU to run targets built for other
   architectures in emulation, targets which produce native host system
   binaries that implement Zephyr's driver interfaces with POSIX APIs, and even
   running different Zephyr-based binaries on CPU cores of differing
   architectures on the same physical chip. Each of these hardware
   configurations is called a "board," even though that doesn't always make
   perfect sense in context.

[tejlmand]

Link is not working well; I'm trying to reference this part of the old getting started:

Maybe use this link to the old docs instead: https://docs.zephyrproject.org/1.14.1/getting_started/index.html#board-misnomer

[tejlmand]

Decoupling "board" and "soc" from "arch" is exactly what I am suggesting. An SoC can have CPU clusters with as many different architectures as the vendor wants to cram in there, so any target board can too.

@mbolivar-nordic While i'm agreeing a lot with your statement, I also like the structure of multiple levels, as a form of filtering.
This means a user can look only at arm, riscv, x86 based boards.

Have you considered if everything should be at top-level or we should have another structure ?
Vendor based, like boards/intel/<intel-boards>.
That may make sense for some vendors, but we risk n-vendor folders just containing a single board, which also leads to mess.
And what if you're having a board with an intel chip, but from vendor b, then in some cases it could be nicer to filter on soc-vendor rather than board vendor.

One requirement for the v2 scheme is to be self-contained.
A board selects a SoC / SoC core, and a SoC core selects the architecture (done indirectly today through the folder hierarchy).

Now, because the v2 scheme must be self-contained, then that also means we could easily extend west boards, so that we simply build the board / SoC / arch Kconfig tree, and thus can be able to support:
west boards --> list all boards
west boards --arch arm --> list all boards where the has a SoC, where as minimum one core is arm based.
west boards --vendor intel --> list all boards where the board vendor is intel.
west boards --soc-vendor nordic --> list all boards where the soc vendor is nordic.

Such a possibility would imo make it acceptable to have all boards and SoCs at a single level, as there is now an option to easily filter boards.

Note 1: This was not an original use-case, but a use-case that will be possible in the v2 scheme, but I will need to take a look at the arch v2 scheme, as that has not initially been part of this work.

Note 2: Today addressing a specific core on a SoC is defined as a independent board, like nrf5340dk_nrf5340_cpuapp vs. nrf5340dk_nrf5340_cpuapp. Should we consider addressing this in scheme v2 as well.
Formalize how cores are addressed, for example as: nrf5340dk_nrf5340@app and nrf5340dk_nrf5340@net
And then a west boards --vendor nordic could present:

$ west boards --vendor nordic
...
board: nrf52840dk_nrf52840, SoC: nrf52840 (1 core): core 0: arm
board: nrf5340dk_nrf5340,   SoC: nrf5340 (2 cores): core 0: arm, core 1: arm
...

[tejlmand]

Update compliance check to verify a SoC v2 conforms to the rules

This is tested here, #50356, with an invalid defined board v2, and the result:

warning: undefined symbol NATIVE_APPLICATION:

- Referenced at boards/arm/nrf52840dk_nrf52840/Kconfig.v2.nrf52840dk_nrf52840:4:

config BOARD_NRF52840DK_NRF52840
	bool
	default y
	select SOC_SERIES_NRF52X
	select SOC_NRF52840_QIAA
	select NATIVE_APPLICATION
ERROR   : Test KconfigBoardV2 failed: Undefined Kconfig symbols:

warning: undefined symbol NATIVE_APPLICATION:

[nashif]

An SoC can have CPU clusters with as many different architectures as the vendor wants to cram in there, so any target board can too.

in zephyr we are using SoC in the wrong way. The way it is now, it represents just one IP that is tied to one architecture. An SoC with two different IP blocks with different architectures would be a superset in this case, a system....
If we are looking at craming multiple IPs under SoC to represent a system, we might be looking at more than just a change to Kconfig..

[tejlmand]

Toolchain WG summary:

• Current structure makes it confusing to users on multi-soc and multi-core-soc boards
  • example: Up Squared which has multi-core SoC (Celeron / Pentium core (x86) + audio DSP (xtensa)) and altera FPGA MAX10 SoC.
  • nRF9160dk contains both a nRF9160 and a nRF52840 SoC (same arch)
• The current structure also results in build system needing to setup a generic toolchain for dts pre-processing and later switch the toolchain to a target toolchain. This is because arch (and thus toolchain arch) is not known until after DTS.
  A v2 scheme could benefit the build system, so that board, SoC, and arch can be determined before DTS invocation, allowing build system to decide toolchain early
  • This has the extra benefit of allowing further cleanup of cmake/modules/*.cmake in areas such as:
    • toolchain handling
    • arch and soc CMake

[nashif]

thought more about all of this. In the ideal world and looking down the path of supporting products with multi-arch SoCs, an SoC (as we have it today) would be just contained and part of a Multiprocessor system on a chip (mSoC) and we would have one board definition that builds all parts of the mSoC and generates artifacts for those using different toolchains if applicable (basically what sysbuild does) in a multi-stage build process.....

[tejlmand]

would be just contained and part of a Multiprocessor system on a chip (mSoC) and we would have one board definition that builds all parts of the mSoC and generates artifacts for those using different toolchains if applicable (basically what sysbuild does) in a multi-stage build process.....

correct, and i'm trying to design this, including thoughts on scalability.

We basically want a common tree (to make maintaining boards / SoC easier and cleaner), and in the highlevel system (sysbuild) be able to specify just the board (system).

The highlevel (sysbuild) will then pass specific information down to the individual build stages, for example build Zephyr sample N with toolchain X, for core A, build sample M, toolchain Y, core B, etc.

But it should still be possible to directly request a build of sample N, toolchain X, core A.
Also each build of Zephyr must still be able to identify exactly for which arch / core its being built, as to ensure correct settings (and defaults) are available.

Therefore we want common knowledge shared in the tree, and not scattered around.

[mbolivar-nordic]

would be just contained and part of a Multiprocessor system on a chip (mSoC) and we would have one board definition that builds all parts of the mSoC and generates artifacts for those using different toolchains if applicable (basically what sysbuild does) in a multi-stage build process.....

correct, and i'm trying to design this, including thoughts on scalability.

... and I'm trying to help on the devicetree side using system devicetree, which I'll send an RFC about soon.

[mbolivar-nordic]

On Mon, Sep 19 2022, Torsten Tejlmand Rasmussen wrote: > Decoupling "board" and "soc" from "arch" is exactly what I am suggesting. An SoC can have CPU clusters with as many different architectures as the vendor wants to cram in there, so any target board can too. @mbolivar-nordic While i'm agreeing a lot with your statement, I also like the structure of multiple levels, as a form of filtering. This means a user can look only at arm, riscv, x86 based boards. Have you considered if everything should be at top-level or we should have another structure ? Vendor based, like `boards/intel/<intel-boards>`. That may make sense for some vendors, but we risk n-vendor folders just containing a single board, which also leads to mess.

I agree, it's a tricky problem to solve. I just think we should not enforce this false ARCH-based hierarchy in v2.

[fabiobaltieri]

@tejlmand may not be a bad idea having some smartness in west boards to print full names if multiple identifier are present for a board, for example right now the fact that some boards have multiple revisions is rather opaque.

[fabiobaltieri]

i'm very open to any proposal, not happy about v2 myself, but lack a better idea.

Is it unreasonable to mix v1 and v2 up under top root boards/ and soc/? We'd have to live with a fair bit of chaos during the transition but at least we'd keep the top level directories.

[gmarull]

i'm very open to any proposal, not happy about v2 myself, but lack a better idea.

Is it unreasonable to mix v1 and v2 up under top root boards/ and soc/? We'd have to live with a fair bit of chaos during the transition but at least we'd keep the top level directories.

I'd personally migrate all socs/boards while in the collab branch, and make main fully v2 in one go (so no need for v2 folders). Otherwise, it will be a mess.

[fabiobaltieri]

I'd personally migrate all socs/boards while in the collab branch, and make main fully v2 in one go (so no need for v2 folders). Otherwise, it will be a mess.

Frankly that'd be ideal if the two don't have to coexist. Would it be an option to migrate all upstream boards in one swipe and keep supporting existing out-of-tree boards with no changes? I'm assuming this will use the two-release deperacation timeline right?

[tejlmand]

@tejlmand may not be a bad idea having some smartness in west boards to print full names if multiple identifier are present for a board, for example right now the fact that some boards have multiple revisions is rather opaque.

I think too much smartness in west boards is colliding with the --format option, cause anything printed with default formatting should also be possible to print with a specific --format="{default_fmt}" string.

I've added a identifiers format option, and if identifiers is added the boards in new format will print like this:

$ west board --format="{name},{identifiers}"
mps3,['mps3/an547', 'mps3/an547/ns']
rzt2m_starter_kit,['rzt2m_starter_kit']

Boards in old format will have an empty identifiers string, but I don't think we should care to much for those, especially not if we'll make the transition for all boards in the collab branch.

So only thing left to decide is the default format string to use.

Currently that is:

$ west boards -h
...
usage: west boards [-h] [-f FORMAT] [-n NAME_RE] [--arch-root ARCH_ROOTS] [--board-root BOARD_ROOTS] [--soc-root SOC_ROOTS] [--board BOARD]

Display information about boards

optional arguments:
...
  -f FORMAT, --format FORMAT
                        Format string to use to list each board; see FORMAT STRINGS below.
...
FORMAT STRINGS
--------------

Boards are listed using a Python 3 format string. Arguments
to the format string are accessed by name.

The default format string is:

"{name}"

The following arguments are available:

- name: board name
- identifiers: board identifiers
- arch: board architecture
- dir: directory that contains the board definition

[tejlmand]

Is it unreasonable to mix v1 and v2 up under top root boards/ and soc/?

it is, especially as some Kconfig files in old mode are sourced inside choices.

If we do soc / board transition for everything in Zephyr tree (and allow old format in oot repos), then we can move everything from soc/v2 --> soc and boards/v2 --> boards in one commit in the collab branch before merging to main.