Manifest file documentation

[andreaskern74]

This is a first draft of a manifest file.

See also #34.

[heuerfin]

As you mentioned the specific call /usr/bin/python3 is linux/unix specific. But however an user on windows can easily change it to C:\ProgramFiles\Python\bin\python3.exe. Or in a more generic case python3 (I think win sub-process will resolve python3 -> python3.exe).
I would generally not advise to use the attribute name. I would suggest something like program="/usr/bin/python3".

[andreaskern74]

"My windows" 😆 resolves python3 to the Microsoft App Store, which I cannot connect to 🙈 . python resolves to a Python 3.x version of the python.exe.

Anyway, I change name --> program

[MatteoRagni]

Manifest are inherently installation specific, and I would go for a simplification. While installing a checker bundle, the installation procedure can put the correct location for the executable program in the manifest itself (e.g., starting from a manifest template). This is a developer specific implementation. In general, this will simplify installation of checker bundle that runs in virtualenv.

Some notable examples:

• for python packages installed with pip, a CusomInstall.run command can be included in setup.py files to generate the manifest file. System location for manifest file may be read from env variable (the same one used by the framework to scan for manifest files)
• for linux apt based project, manifest may be generated in postinst phases
• for windows installers based on Inno Setup, it is a file created with a custom Pascal function (yikes!)

Under this condition, we should only specify the final form of the manifest, which in general can be OS agnostic.

---

Additionally, I would go with the concept of <Interpreter> for languages that require an interpreter instead of being execute directly.

[heuerfin]

"My windows" 😆 resolves python3 to the Microsoft App Store, which I cannot connect to 🙈 . python resolves to a Python 3.x version of the python.exe.

Anyway, I change name --> program

Yeah thats a common "issue"... You can change that annoying gehavior via "Manage app execution aliases": https://stackoverflow.com/questions/58754860/cmd-opens-windows-store-when-i-type-python

[heuerfin]

Manifest are inherently installation specific, and I would go for a simplification. While installing a checker bundle, the installation procedure can put the correct location for the executable program in the manifest itself (e.g., starting from a manifest template). This is a developer specific implementation. In general, this will simplify installation of checker bundle that runs in virtualenv.

Some notable examples:

• for python packages installed with pip, a CusomInstall.run command can be included in setup.py files to generate the manifest file. System location for manifest file may be read from env variable (the same one used by the framework to scan for manifest files)
• for linux apt based project, manifest may be generated in postinst phases
• for windows installers based on Inno Setup, it is a file created with a custom Pascal function (yikes!)

Under this condition, we should only specify the final form of the manifest, which in general can be OS agnostic.

Additionally, I would go with the concept of <Interpreter> for languages that require an interpreter instead of being execute directly.

Yeah I think thats right. This will depend on installation or a development configuration.
From the frameworks perspective, we shouldn't consider how this is filled, only that this will be somehow defined (installation wise or by the users definition).

Have you any opinion on maybe changing the control flow from qc-framework searches for manifests to checker bundles register at qc-framework? (e.g. via EnvVar).
From my point of view this would only have some issues if qc-framwork is later installed than the checker bundles. (even though every checker could have some code like

if ENV_VAR not present
    ENV_VAR <= "/some/user/accessible_directory/"
fi
place manifest in ENV_VAR folder

But this will rely on the point, that each checker will do this stuff.

[heuerfin]

To mitigate all the OS stuff we could probably introduce something like:
<Executable program="python3" os="linux"> or <Executable program="python3" os="windows">. Then the qc-framework should determine the OS and run the matching <Executable> definition (could also be set in qc-framework configs/parameters in contrast to auto determination).

[heuerfin]

A general question: does an checker-library gets called with some qc-framework parameters by default? E.g.
Executable definition /path/to/my_checker (no further args)
Will lead to a default call of, for example /path/to/my_checker --xqar-manifest /path/to/the/manifest.xml?

Can the checker library rely on some default arguments? (I wouldn't suggest it, since the arguments will be given in the file).

• If default arguments are provided we should guarantee, that these parameters will be appended (at the end) to the list of arguments in the <Executable><Arg></Executable> definition. Otherwise, something could be messed up in the case of python/script languages, e.g. with no order guarantee: python3 --xqar-manifest /path/to/the/manifest.xml checker.py --fancy-param=True

Also, can the checker library rely on being called in the directory of the manifest file (so that it can find the manifest in the current directory)?

[andreaskern74]

Info about the current implementation: https://github.com/asam-ev/qc-framework/blob/main/doc/manual/writing_user_defined_modules.md

There are three ways to call a Checker Bundle

• Start with configuration file (official approach from the framework)
• Create "empty result file" --> this is the manifest for the Configuration GUI, see also: https://github.com/asam-ev/qc-framework/blob/main/doc/manual/using_the_framework.md#add-self-implemented-checkerbundles-and-reportmodules
• Start with default configuration (without configuration file, for simple batch processing, when called without the framework. I did that many times when showing demo, calling CheckXodr.bat)

[MatteoRagni]

I would go with placeholder arguments, such as <Arg>{CONFIG}</Arg>. We just need to document them, and eventually include in schema the presence for them when we consider them mandatory.

[heuerfin]

I mean if the placeholder / resolvable aguments just resemble the same as, e.g., <Param name="CheckerBundleParameter" value="0.1"/> , I would not include them since it will be unclear where to derive the current configuration from (CLI vs File).
My understand was more, that the <Arg> will represent executable specific arguments like main.py for an interpreted Python file.

But as @andreaskern74 mentioned in the docs, there is a way to call the checker only via CLI args: Check a Single File With One Single CheckerBundle
So probably, the placeholder stuff will be necessary.

However, while checking the docs I came across another "difficulty":

> checker_bundle_exe configuration.xml

> checker_bundle_exe track.xodr [-o1 -o2 ...]

Both calls will look from the checkers perspectiv identically. Determining whether the specified parameter is a configuration or the file to be checked is therefore more difficult than it should be. I know reading the file and determining if it is a configuration is probably not that hard, but still it would be much easier if, we just introduce a slightly different call for each version of these, e.g.,:

> checker_bundle_exe -c configuration.xml
> checker_bundle_exe --config configuration.xml

or

> checker_bundle_exe --check track.xodr [-o1 -o2 ...]

However, this discussion should be probably shifted to another place.

[andreaskern74]

However, this discussion should be probably shifted to another place.

👍

[heuerfin]

And to be honest with the introduction of these <Executables>, the location requirements of executables regarding to relations-between-terms--definitions become irelevant. A checker library can now at any place, this configuration will define the locations of them.

[andreaskern74]

What do you mean with "location requirements"? Did you understand the diagram, that the Checker Library should be in a sub directory? IMHO not!

In the current implementation, everything needs to be in the same directory. There was no concept of different Checker Libraries. And yes, with this concept, we have at least two different installation directories:

• ASAM QC Framework
• e.g. Checker Library for ASAM OpenSCENARIO XML

And maybe

• My internal Checker Library

And that is IMHO exactly how it should work. We "only" need to solve the problem, that the framework need to find the executables/modules from the Checker Library.

That's the issue we are talking about: #11

[MatteoRagni]

I would force the execution for loacations in subdirectories of the library. It is a little restrictive, but may be a security requirements (e.g., the approach for Dockerfiles).

[heuerfin]

What do you mean with "location requirements"? Did you understand the diagram, that the Checker Library should be in a sub directory? IMHO not!

In the current implementation, everything needs to be in the same directory. There was no concept of different Checker Libraries. And yes, with this concept, we have at least two different installation directories:

• ASAM QC Framework
• e.g. Checker Library for ASAM OpenSCENARIO XML

And maybe

• My internal Checker Library

And that is IMHO exactly how it should work. We "only" need to solve the problem, that the framework need to find the executables/modules from the Checker Library.

That's the issue we are talking about: #11

I assumed from the diagram that at least Checker Bundle A1.exe, Checker Bundle A2.exe, and Checker Bundle A3.exe must be in the directory Checker Library A Directory. Is that wrong? Because the contains was quiet clear for me that these will be inside there.

However, from my point of view with the definiton of exetuables this will no longer be necessary since one executable can be at /path/a/Checker Bundle A1.exe and a possible second one at /path/to/somewhere/else/Checker Bundle A2.exe

[heuerfin]

would force the execution for loacations in subdirectories of the library. It is a little restrictive, but may be a security requirements (e.g., the approach for Dockerfiles).

Yeah from a security perspectives this makes probably sense. But, to be honest, if we think that programs outside the current directory can't be trusted, we probably can't trust executable files in the current directory either :D

And when we exclude executables outside the current directory we loose all the benfits of calls like

• /usr/bin/python
• conda
• /home/user/.venv/.../bin/python

and so on. So I would also allow all this.
However, to mitigate this security stuff, we should probably consider the registration again. Because, in that case we can ensure at some point only "privileged" processes (e.g. installation processes) can place configurations (which are confirmed by the user with his installation) in the qc-frameworks manifest registration directory. So only privileged users can temper with these configurations.
This way, we can use executables outside a specific directory and also have a lower security risk (yeah I know there are also arguments against it, since the executable can also be modified - but to be honest - the installation should make sure that only privilged users can temper also these executables).

[heuerfin]

We should keep in mind that we need in the XSD schema at least 1..* cardinality: minOccurs="1". On the other hand it should be possible to define multiple <Executable> for probably multiple OS definitions. Logicaly each <CheckerBundle> should only have one <Executable>, so we could also use 1..1 cardinality: minOccurs="1" maxOccurs="1" and some other XML structure:

<Executable>
    <Program os="linux">/usr/bin/python3</Program>
    <Program os="windows">C:\Program\Python\bin\python3.exe</Program>
    <Arg>my_fancy_script.py</Arg> <!-- Mandatory arguments, followed by the configuration.xml or input file-->
    <Arg>--fancy-param=True</Arg>
 </Executable>

Now the <Program> will have 1..* cardinality and <Arg>should be in all cases 0..*

[heuerfin]

I think either, it should be possible to define 0..* directories upon calling qc-framework which will be recursivly scanned for these manifests. Or we could have like a registering procedure for checker libraries which will place the manifests somewhere in a qc-framework "checker-directory" probably defined by an environment variable like QC_FRAMEWORK_CHECKER_DIR.

[heuerfin]

I agree that it is the same thing in this sense. However, I would also be fine if we wanted to distinguish between the two for the sake of readability for users.

[heuerfin]

I think this question is simply delegated to the operating system. We do not consider what the user wrote in the attribute program="...". It will be just processed by OS -> sub process. This will perform the priority of relative/absolute path, executable on $PATH / user_local_path and so on.

[andreaskern74]

When this is delegated to the OS, then the OS needs to find it. And for a "normal" checker_bundler.exe, the Checker Library binary path needs to be in $PATH (don't like).

So I propose something like

1. Try to find program="..." in the Checker Library directory
2. If not found, delegate to OS

[heuerfin]

Hmm... I didn't mean it must be on $PATH. I agree that this shouldn't be mandatory.

However, the manifest will be defined by the checker library, right? So there is something like "configuration time". During this time the library knows verry well its location (via installation and automatic determination or via user's definition). Thus, it doesn't need to be on PATH, it is still defined (via absolute path), and we can delegate everything to OS.

I'm thinking about three ways:

1. Developer:
   E.g., I develop xosc_check.py. To test it I need to define the manifest. So I know verry well the configuration, e.g. <Executable program="/home/user/.venv/dev/bin/python3"><Arg>/home/user/dev_project/xosc_check.py</Arg></Executable>
   This manifest will be somehow available to the framework (e.g. a search based process or registration).
2. Installation
   The installation process will know where it placed exectuables, so the manifest is clear:
   <Executable program="python"><Arg>/usr/local/install_dir/xosc_check.py</Arg></Executable>
3. The common "gity" way:
   git clone https://github.com/my-fancy-checker
   The checker should include a "deployment/installation" script or documentation:
   ./install.sh -> will place the executables and know the configuration.
   or "Create a manifest in /x/y/z with the following content"

So at any time during this "configuration time" the execution method & call is clear and can be specified. Then the qc-framework has it easy at execution time since the configuration contains all this stuff.

[MatteoRagni]

I was confused by my first read, and I just noticed that things are becoming for me a little too blurred or mixed up. I'll give my two cent on the argument, while summarizing what was raised in the discussion.

We must separate two topics:

1. Manifests are the way for checker libraries to register themself in a framework install, working more on a system / installation level and to tell the framework how contained bundles are intended to be executed (where and how)
2. parameters for a checker bundle can be obtained via Result File Format during some sort of dry run, thus working at application execution. IMHO this is also the stage that informs the framework of the checked rules (UID list), information that is not contained in the manifest (also because, as a developer, I would want to be able to create this list dynamically, and not to maintain it in a separate file)

While I can understand the usage of xqar for the second, I would argue against using it for Manifests for the following reasons:

• SOLID and in particular interface segregation principle violation, which implies:
  • including manifest stuff in result file schema will make the format confusing to operate, which can have a negative impact on project adoption
  • manifest language evolves with different speed with respect to results - i.e., with adoption of checker implementations (today executables and python, tomorrow containers or other languages)
• there is a strong probability that user specific install requires intervention on the content of the manifest file for customizations: readability is important and xml should be avoided (e.g., a developer of a python bundle using a virtualenv for a specific test install)

Now focusing the discussion on Manifest:

Since manifest are a way to inform the framework which executables / check-bundle are included, we can identify the following properties:

• they are operating system specific, and may be generated on the fly during checker library install or manually created / edited by the user
• as @heuerfin said: they register themself: manifests must be installed in a directory known to the framework (e.g., on linux platform the standard the facto is to have a directory like /etc/asam/qc/libraries.conf.dand all manifest must be .conf file in that directory). Additionally, framework can be aware of a sequence of well-known directories (e.g., on linux /etc/asam/qc/libraries.conf.dand $HOME/.config/asam/qc/mainfest.conf.d) and a path-set of directory provided via a specific environment variable (e.g. ASAM_QC_LIBRARIES_PATH)
• optionally, they may be combinable, i.e. one manifest includes (or anchors) the others: this is important to enforce DRY, and is particularly powerful to create complex custom configuration sets (e.g., define only in the root manifest which is my reference python binary or my reference virtualenv). In order to do so, inclusion of manifests (discussed at the previous point) must be a mechanism which is explicit of the manifest language (i.e., some include keyword) and not implicit (i.e., in the code of the framework implementation)

Regarding execution, we should focus on the following argument and their impact on the manifest definition:

• execution type: we always considered executables and python, but maybe we should consider executables and interpreted. If we want to be future proof, other categories to consider may be containers and remote (even if I would suggest that the last one is implementable as a specialization of the others)
• piping: simply put, redirection of stdin / stdout
• environment: definition of specific env variables and definition of the current working directory of execution, if in a new login shell and which account, mangle the PATH for dependencies, etc.
• file management: how do we want to coordinate the location for input file and for output / log files (trivially: how to robustly avoid that two rogue bundles overwrite each other results)
• security: unfortunately, the framework may be part of automated pipelines and this can create some problems... Registration approach at installation, as suggested by @heuerfin will help us quite a lot, but still we may want to avoid to eavesdrop a shadow-file 😅 . Considering a checksums db handled by framework can be discussed.

(Now I'll go to sleep I'll try to get some time to write down a more concrete proposal tomorrow)

[heuerfin]

@MatteoRagni Yes, I think you're probably right.

1. We should use different description format for results and manifests
   Probably, using the same argument, we could/should seperate the results from the dry-run generating the checkers parameters?
2. The proposed registration options seem to be very good! We should use default config directory, well-known config directories and environment variable option.
3. And yes, from my point of view too, the two options for execution types are executable files or interpreters. However, by using the concept of "program" and "arguments", we can support executable programs and interpreters.
   I think, a possible container type will also just fit in this concept: program=/usr/bin/docker + args=["run", "--rm", "asam/xosc-checker:1.23.1", "-c", "/path/to/config", "-v", "/local/conf/path:/path/to/config"]
   And as you already stated remote will also be just a special case.

[heuerfin]

One thing I'd like to discuss in general is what you think is really the most likely way of installing/deploying the framework (and checker libraries)?

• "Classic" installation processes, e.g., Windows: setup, winget, linux: package manager?
• Downloading components: wget https://remote/release/1.0.0/qc_framework.tar.gz, wget https://remote/release/1.0.0/xodr_checker_library.tar.gz, wget https://remote/release/1.0.0/xosc_checker_library.tar.gz (or use git git clone repo@remote)
  With an manual way or included deployment scripts?
• Bundled builds: e.g. defining piplines bundling qc-framewok + xodr-checker + xosc-checker and provide it as asam-qc-linux.tar.gz / asam-qc_windows.zip

To be honest, I would think that the last option is the way to go. The first option means a lot of extra work for the provision of setup programs/installers. The second option means extra work for the users. I think in the end it should be really easy to do these checks for a single file and low "deployment" overhead.
So a workflow like this would be my prefered way:

• go to the qc-framework release website (https://github.com/asam-ev/qc-framework/releases) and download the latest bundled version with predefined ASAM checks (or version independent link https://github.com/asam-ev/qc-framework/releases/lates/asam-qc-bundled-win.zip)
• just click framework.exe or run ./framework.

What do you think?

[MatteoRagni]

@MatteoRagni Yes, I think you're probably right.

1. We should use different description format for results and manifests
   Probably, using the same argument, we could/should seperate the results from the dry-run generating the checkers parameters?

If we design the system from scratch I would argue that we should. I can understand we have an existing implementation, which includes the parameters and rules information in the result file, thus I can understand its usage in some sort of dry run scenario. Dry run is used to know how to use the bundle and what the bundle does, there is a requirement to have the same set of information also for normal runs.

I only want to make sure that everyone acknowledge that with this approach, while a bundle may want to provide a "schema" in the dry run (list of parameters, their limits, cardinality, etc.), while a result file will contain a "realization" of that schema (the actual parameter values). To merge both behavior in a single file format may again violate good design principle, but I can live with that.

I would push to make the execution of the dry-run explicit though (e.g., bundle.exe --bundle-schema or similar)

3. And yes, from my point of view too, the two options for execution types are executable files or interpreters. However, by using the concept of "program" and "arguments", we can support executable programs and interpreters.

I can agree with that (including env-var in the argument). Looking at the Statista used programming languages just for fun, we may consider the following as relevant for us just to understand if we are future-proof:

• javascript / typescript: there are approaches with npm and yarn to run with node-packages from a different directory using command line arguments (npm --prefix <path> run <command>). Ok
• python: we can use the path of the interpreter to encapsulate virtual env. I'm not experienced with conda though, but I think it must be explicitely activated before running. This may requires us to include the concept of before_script, something to run before the actual command, in the same process (stolen from gitlab-ci). Investigate
• bash / shell: Ok
• powershell: I think we must consider some issues with Execution-Policy, but I'm not an expert. Investigate
• java: if I remember correctly we must only provide a way to set the env-var JAVA_HOME, or distribute them as compiled executable. Ok, but Review please
• C#: the behavior is different between linux and windows (the first must define the dotnet interpreter, while on windows you will get an executable to call directly). Ok
• PHP: (please no)
• Ruby: the most applied approach I think is bundler and a reference to a Gemfile via envvar BUNDLE_GEMFILE. Ok
• Lua: or by setting the LUA_PATH env var. There is also something similar to python virtual env, called vert but I don't know its adoption. In any case it requires to activate the environment, similar to before_script. Investigate
• Matlab: envvar for MATLABPATH and MATLABROOT are enough? Ok, but Review please

I think, a possible container type will also just fit in this concept: program=/usr/bin/docker + args=["run", "--rm", "asam/xosc-checker:1.23.1", "-c", "/path/to/config", "-v", "/local/conf/path:/path/to/config"]

It can be a good enough initial solution, still leave open the door for a future more explicit configuration (e.g., attaching to networks or existing volumes may require structure that is more complex).

And as you already stated remote will also be just a special case.

Agreed.

[MatteoRagni]

One thing I'd like to discuss in general is what you think is really the most likely way of installing/deploying the framework (and checker libraries)?

• "Classic" installation processes, e.g., Windows: setup, winget, linux: package manager?
• Downloading components: wget https://remote/release/1.0.0/qc_framework.tar.gz, wget https://remote/release/1.0.0/xodr_checker_library.tar.gz, wget https://remote/release/1.0.0/xosc_checker_library.tar.gz (or use git git clone repo@remote)
  With an manual way or included deployment scripts?
• Bundled builds: e.g. defining piplines bundling qc-framewok + xodr-checker + xosc-checker and provide it as asam-qc-linux.tar.gz / asam-qc_windows.zip

I think the correct solution depends on how the checker library is implemented.

• C/C++ using cmake, makes a compelling argument in using CPACK to create redistributable packages (Windows installer or linux repository packages) with little to no overhead.
• Python using setuptools and distribution commands is again something easy to put in place (I'm a little biased because I actually do not see a real use case for single scripts files, but still the single script can be treated as an artifact)

Once package is created distribution depends again on technical choices: github/gitlab release pages, artifacts repository (binaries, winget, apt, rmp, pypi, etc.) really depends on the IT infrastructure of the developer of the library, more than a question to be discussed in the framework. Packaging provides you usually a structured way to install the manifest, which can be beneficial to the overall argument.

• go to the qc-framework release website (https://github.com/asam-ev/qc-framework/releases) and download the latest bundled version with predefined ASAM checks (or version independent link https://github.com/asam-ev/qc-framework/releases/lates/asam-qc-bundled-win.zip)
• just click framework.exe or run ./framework.

What do you think?

If we are talking about ASAM implementations for quality checker and checker bundle, which are intended to be open source, an initial release like the one you described is in my opinion good enough.

A successful project should start to use public repository when it becomes mature (mainly apt, rmp, winget and pypi, given the selected technology). Furthermore I'm envisioning a docker image distribution with all the ASAM checker executable and libraries installed, set and ready to go.

[MatteoRagni]

Here an example of what I intended with manifest (this is the too-complex-version-to-be-implemented-right-now-but-a-possible-candidate-for-the-future one)

---
# /etc/asam/qc/manifest.conf
# Root manifest file, its location is really well known by the framework executable
# There is nothing that prevents us to use it as a "database" file and to include some 
# utilities in the framework gui application that helps the end user to modify this file
# such as adding new search directory.

# I'm trying to do a very complex example. Most of the thing I'm suggesting can be "dumbled down"
# for a simpler implementation, and left for the future. **Furthermore, I'm using yaml for simplicity
# but that does not mean we must use this language**, even if using some of its features like anchoring may
# be really useful.

$schema: "asam.org/schemas/manifests/1/checker-framework.schema"
defaults: # An object to define defaults value to be used troughout. Default is a private name that 
          # cannot be used for other libraries
  variables:
    VARIABLE_NAME: value
    # While here we have a env var set explicitely, framework may decide to set several implicitely
    # and well document them. For example, the path for configuration file that has to be sent to
    # a bundle, instead of being part of the arguments of the executable, is defined in an envvar.
    # We will see later an application example. We can use those implicit variables throughout the
    # manifest.
    #
    # Some implicitely predefined variables may be, for example:
    # ASAM_FRAMEWORK_CHECK_UUID: set a unique id for the current checker bundle run
    # ASAM_FRAMEWORK_CHECK_CWD: set the current working directory intended 
    # ASAM_FRAMEWORK_CHECK_CONFIGURATION: the input cponfiguration, a bundle can set it in its own
    #                                     input argument string
    # ASAM_FRAMEWORK_INPUT_FILES: path separated list of input files may be requested in the future (e.g., xosc + xodr)
    # ASAM_FRAMEWORK_LOG_LEVEL: this may be an indication in order to raise the current log level for all the bundle.
    #                           But it is up to the bundle if onor this or not.
    
  executable: &executables # Sets the default configuration for running an executable (&executables syntax is used later)
    cwd: "$ASAM_FRAMEWORK_CWD"
    stdout: "$ASAM_FRAMEWORK_CWD/stdout.bin" # Where do we redirect the output?
    stderr: "__PIPE__" # There can be special viariable for piping and streaming information between framework and 
                       # checker bundle process, for example for showing dat adirectly in a user interface.
    variables: # Additional variables? Why not
      ANOTHER_VARIABLE: value

  interpreters: # Please notice that even if we defining an interpreter, we should the same options provided
                # in the executables. This can be done in two way: explicitly, by using supporting language conventions
                # in order to include defaults, or implicitly by programming the framework in order to do so. In this
                # particular example, python is shown as implicit, ruby as explicit. We should select one of the two approaches
                # and this can be constrinaed by the supporting language.
    python: &python # (this is an anchor, used later for an example)
      executable: /usr/bin/python3
      stdout: "__PIPE__" # just to show that we can override some configuration on the executable

    ruby: &ruby 
      <<: *executables  # This example uses anchor and pointers to merge complex structures, instead of doing it implicitly software side
      executable: "/usr/bin/bundler exec"  # Yes, I prefer to use strings in place of list of arguments... We can split it at the end.
      variables:
        BUNDLE_GEMFILE: "$HOME/.local/Gemfile" 
      
      
includes: # This part will tell the manifest where to search for other manifests explicitely. Each line is considered a path separated
          # list of path. For example on linux, the pathsep character is `:`. Manifest shall end with a well known suffix name
  - /etc/asam/qc/manifest.conf.d
  - $HOME/.config/asam/qc/manifest.conf.d
  - $ASAM_FRAMEWORK_MANIFEST_PATHS # This will be expanded from the env variable, and will become a list of path.

---
# /etc/asam/qc/manifest.conf.d/example-1.conf
# This is a manifest for a checker library, which contains a couple of bundles.

my-awesome-checker-library:  # Forcing unique name for libraries
  $schema: "asam.org/schemas/manifests/1/checker-library.schema"
  
  name: My Awesome Checker Library
  version: "1.0.0"
  description: <<
    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor 
    incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
  
  variables: 
    AWESOME_CHECK_LIBRARY_INSTALL_PATH: $HOME/awesome-library/bin
  
  bundles:
    my-executable-bundle: # ...and forcing unique name for bundles
      name: My Executable Bundle
      description: <<
        Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor 
        incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
      
      type: executable
      executable: "$AWESOME_CHECK_LIBRARY_INSTALL_PATH/executable-bundle.bin"
      args:
        schema: --get-parameters-schema  
        execute: --config $ASAM_FRAMEWORK_CHECK_CONFIGURATION  # It is up to the bundle to decide if the configuration is something that
                                                               # comes from command line or it is read from the env variables. We don't even 
                                                               # need to check if it is used explicitly.
                                                               
    my-python-bundle:
      name: My Python Bundle
      description: <<
        Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor 
        incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
      
      type: interpreter.python # This can be defined in this way, but if use anchors, we can also use something like a pointer...
      # <<: *python
      
      args:
        schema: "-m awesome_python_bundle.schema"  # In this case developer chose to use empty argument
        execute: "-m awesome_python_bundle" # In this case the configuration is read via env file...
        
    my-virtualenv-python-bundle:  # A very simple example with venv
      name: My Virtualenv Python Bundle
      description: <<
        Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor 
        incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
      
      <<: *python # This is the alternative approach
      
      executable: "$HOME/.venv/bin/python"
      
      args:
        schema: "-m awesome_python_bundle.schema"  # In this case developer chose to use empty argument
        execute: "-m awesome_python_bundle" # In this case the configuration is read via env file...

[andreaskern74]

@MatteoRagni / @heuerfin : Thanks for all this info/discussions! So we have multiple topics

• framework need to find checker bundles of checker libraries after installation
• framework need to find the required interpreter of the checker bundles in system
• seems that there need to be a easy/generic process to update/adapt the manifest file to the current system during installation
• need for standardized way to set parameters of a checker bundle via command line? in our current implement this is not standardized, although the doc has written it. The only thing which has all checker bundles in common is that they can be called with standard setting bundle.exe track.xodr (no --check here). Setting parameters is done for each different, because of different developers preferences 😕
• distinguish result file format from manifest file format

If the current architecture/implementation is a big impediment here, then we need to adapt it. This topic is one of the most important one for the success of the framework. I think we are on a good way regarding result/reporting/rule relations/... So it's no blocker for the service provider to solve in the short run.

And yes - this is the wrong place for discussion. This PR was a starting point and should not be merged!

[MatteoRagni]

We moved the discussion in a ASAM tenant Teams chat, in order to create a first design draft to propose to the workgroup. If interested in join the discussion, just DM me on Teams or drop me an email.

[andreaskern74]

https://asamev.sharepoint.com/:w:/r/sites/qualitychecker/Freigegebene%20Dokumente/QC_Framework/Design/Manifest%20File%20for%20Quality%20Checker%20and%20Check%20Libraries.docx?d=w0ae18496d676420cb9cde57d66462cc5&csf=1&web=1&e=WFInW8