Ensure that *.csolution.yml projects are usable

[ReinhardKeil]

The Problem To Be Solved
It is unclear if a project example that is submitted in the *.csolution.yml format is correctly structured.

Also it would be good to obtain information about the toolchain, board, and/or device that is the target.

Proposed Solution

The csolution convert command can be used to verify the correct structure of an project example. In particular it validates:

• the correct format against a schema
• the semantics of the project, i.e. that required packs are installed, source files are provided, etc.
• generates additional information files (one file *.cbuild-idx.yml and for each context *.<context>.cbuild.yml).

The exit code 0 indicates that conversion is successful. Any other exit code indicates warnings or errors that are output to the console and could be displayed to the user.

The format of *.cbuild-idx.yml and *.<context>.cbuild.yml is defined by schema files. While there will be further extensions to these files, the nodes and the key/value pairs will remain compatible.

To extract the information about toolchain, board (optional), and device use this process:

From *.cbuild-idx.yml the list of *.<context>.cbuild.yml files is provided by: cbuilds: - cbuild: with a relative path.
Each *.cbuild-idx.yml specifies exactly one device:, one board: that is optional, and exactly one compiler:.

It should be possible to either use this information directly or write a simple data converter that exports it.

[ReinhardKeil]

Would #1186 affect you much?

[mcgordonite]

#1186 would have some impact on the tools reading the cbuild files, since it is a breaking change. Tools that need to support every version of Toolbox 2.x would need a small amount of additional logic to fall back to cbuild-set.yml if cbuild-set.yaml could not be found.

[ReinhardKeil]

@mcgordonite agree this is the small amount of work. On the other hand we would make the tools more consistent. Currently VS Code extensions use *.yaml whereas CMSIS-Toolbox uses *.yml as extension. As a user this might be confusing. So the question is could you prepare for this work?

[acabarbaye]

@ReinhardKeil this approach should work for us in order to have a sustainable way to extract information about csolution projects. we will base our data extracting tools on the generated *.cbuild-idx.yml and *..cbuild.yml files. I presume in our case, we will have to disable the schema check

[acabarbaye]

@ReinhardKeil @brondani Unless, I missed it the following two fields (defined in a project/csolution https://github.com/Open-CMSIS-Pack/cmsis-toolbox/blob/main/docs/YML-Input-Format.md#solution) cannot be retrieved from the *.cbuild.yml files

field	_	description
created-for:	Optional	Specifies the tool for building this solution, i.e. CMSIS-Toolbox@2.2.0
description:	Optional	Brief description text of this solution.

We are following the process described here to determine all information about projects in a robust way and so, it would be good if those fields could be present in the cbuild.yml files

the two fields are related to the ask described in #1161

[ReinhardKeil]

Currently the information created-for: and description: is not processed by the CMSIS-Toolbox as the intent was to use it just in the IDE. If there is something missing here, we should understand the overall flow.

[willlordarm]

The real world impact of not processing descriptions can be seen on keil.arm.com.

Take this example - https://github.com/Arm-Examples/Hello_FRDM-K32L3A6.

In the csolution.yaml, the description is provided but is not rendered out on keil.arm.com and is instead replaced with a generic placeholder.

This is probably workable for a period of time but to aid discovery developers should be able to view a description of the project. This text can also be indexed for search.

I would still like to see a better convention for naming examples (with e.g. a title in the csolution schema) because the current approach (trimming csolution.yaml) isn't very scalable and will end with title that are hard to parse by humans.

[ReinhardKeil]

@willlordarm I don't understand your request. It should be possible to process description: on other places, but the CMSIS-Toolbox is not using it.

This page https://www.keil.arm.com/boards/nxp-frdm-k32l3a6-989d2e5/projects/ shows descriptions that are provided as part of the pack *.PDSC file. Optionally, the description of the *.csolution.yml could be used.

[acabarbaye]

@ReinhardKeil The problem is for examples coming from other sources than packs such as anything on Arm-Examples. The rationale behind the issue discussed here was that *cbuild.yml files are now the source of truth for any data related to a csolution project: a machine readable interface working across csolution schema versions was indeed required and since the schema of *cbuild.yml is less likely to change, it was agreed that those files would become this necessary interface. This was mentioned here as well as #1161 and #1159.

[ReinhardKeil]

OK, then missing is the description in the *.cbuild.yml files? Or what exactly needs to change?

[acabarbaye]

yes. #1177 (comment). In short, any metadata about projects we want to gather/display need to be in those files or cbuild-idx.yml. This is true for description and created-for

[ReinhardKeil]

@acabarbaye I created #1383 - I hope this captures your problem.

Regarding created-for: this is kind of minium version information. IMHO we should issue a warning if an older version of the CMSIS-Toolbox is used. This could be done by the CMSIS-Toolbox.

However I struggle to see the value of this information in *.cbuild-idx.yml as it contains already version number of the CMSIS-Toolbox that was used to process the solution.

Also created-for: might be mis-understood by users as they believe that exactly this version must be used.

A side note: we should perhaps not use a "closed" issue as it really gets confusing of what is missing. Instead, you could refer to a previous issue when required. Are you OK to close this now in light of #1383?