Skip to content

Show requirements statistics and test results in documentation#77

Open
eelcoem wants to merge 9 commits intoeclipse-score:mainfrom
etas-contrib:eelcoem/add_test_results_to_docs
Open

Show requirements statistics and test results in documentation#77
eelcoem wants to merge 9 commits intoeclipse-score:mainfrom
etas-contrib:eelcoem/add_test_results_to_docs

Conversation

@eelcoem
Copy link
Contributor

@eelcoem eelcoem commented Feb 16, 2026

No description provided.

Empting Eelco (ETAS-ECM/XPC-Fe2) added 2 commits February 16, 2026 08:48
@eelcoem eelcoem temporarily deployed to workflow-approval February 16, 2026 08:50 — with GitHub Actions Inactive
@eelcoem eelcoem temporarily deployed to workflow-approval February 16, 2026 08:50 — with GitHub Actions Inactive
@github-actions
Copy link

github-actions bot commented Feb 16, 2026

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.4.2) and connecting to it...
INFO: Invocation ID: fb2d99d6-b18a-41fc-a43b-f0cb40f0be9a
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
WARNING: For repository 'score_tooling', the root module requires module version score_tooling@1.1.0, but got score_tooling@1.1.2-RC in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'score_rust_policies', the root module requires module version score_rust_policies@0.0.3, but got score_rust_policies@0.0.5 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //:license-check (40 packages loaded, 9 targets configured)

Analyzing: target //:license-check (86 packages loaded, 9 targets configured)

Analyzing: target //:license-check (140 packages loaded, 1963 targets configured)

Analyzing: target //:license-check (148 packages loaded, 5178 targets configured)

Analyzing: target //:license-check (152 packages loaded, 7758 targets configured)

Analyzing: target //:license-check (160 packages loaded, 7817 targets configured)

Analyzing: target //:license-check (160 packages loaded, 7817 targets configured)

Analyzing: target //:license-check (160 packages loaded, 7817 targets configured)

Analyzing: target //:license-check (164 packages loaded, 9829 targets configured)

INFO: Analyzed target //:license-check (165 packages loaded, 9955 targets configured).
[12 / 16] checking cached actions
[14 / 16] JavaToolchainCompileBootClasspath external/rules_java+/toolchains/platformclasspath.jar; 0s disk-cache, processwrapper-sandbox
[15 / 16] Building license.check.license_check.jar (); 0s disk-cache, multiplex-worker
INFO: Found 1 target...
Target //:license.check.license_check up-to-date:
  bazel-bin/license.check.license_check
  bazel-bin/license.check.license_check.jar
INFO: Elapsed time: 26.047s, Critical Path: 2.47s
INFO: 16 processes: 12 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 16 total actions
INFO: Running command line: bazel-bin/license.check.license_check ./formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

@github-actions
Copy link

The created documentation from the pull request is available at: docu-html

@eelcoem eelcoem temporarily deployed to workflow-approval February 16, 2026 08:53 — with GitHub Actions Inactive
@eelcoem eelcoem temporarily deployed to workflow-approval February 16, 2026 08:53 — with GitHub Actions Inactive
@eelcoem
Copy link
Contributor Author

eelcoem commented Feb 16, 2026

@ Reviewers: After merging, it will look like this: https://eelcoem.github.io/lifecycle/main/statistics.html

@eelcoem eelcoem temporarily deployed to workflow-approval February 16, 2026 12:19 — with GitHub Actions Inactive
@eelcoem eelcoem temporarily deployed to workflow-approval February 16, 2026 12:19 — with GitHub Actions Inactive
@eelcoem eelcoem temporarily deployed to workflow-approval February 16, 2026 12:24 — with GitHub Actions Inactive
@eelcoem eelcoem temporarily deployed to workflow-approval February 16, 2026 12:24 — with GitHub Actions Inactive
Copy link
Contributor

@paulquiring paulquiring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

MODULE.bazel Outdated
Comment on lines 35 to 40

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not having the required properties inside your tests will result in errors / crashes in future versions if I remember correctly.
For the 2.3.3 you using here I think this might still be fine. But you will not have any links.
The links are only generated through 'PartiallyVerifies' & 'FullyVerifies' attributes.
If those are not at your tests we can't link it, just so you are aware.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, yeah I think the upgrade to score_docs_as_code 3.0.0 should be a separate PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to 3.0.1 already and adapted accordingly

@@ -28,10 +28,16 @@ bazel_dep(name = "googletest", version = "1.17.0.bcr.1")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to not upgrade further?
As for example Platform & Tooling are also not on newest version

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't upgrade score_docs_as_code to v3 as that would break the build, but the bump to 2.3.3 was necessary because 2.3.0 didn't allow linking test results in the pipeline. As for the other requirements, I think there is no reason not to bump them, I'll have a look

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bumped versions. Also updated to docs-as-code 3.0.1 and adapted requirements as necessary

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename file to test_and_docs, as workflow now does both

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -29,7 +29,12 @@ on:
types: [checks_requested]

jobs:
unit-tests:
uses: ./.github/workflows/tests.yml
secrets: inherit
build-docs:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a job to do docs check, take a look how it is done in docs-as-code:
https://github.com/eclipse-score/docs-as-code/blob/main/.github/workflows/test_and_docs.yml

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added docs-verify step

docs/conf.py Outdated
Comment on lines 72 to 76
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

take a look into rglob method from pathlib, no need to mix it with os https://docs.python.org/3/library/pathlib.html#pathlib.Path.rglob

You can simplify it into something like

from itertools import chain

for log_file in chain((ws_root / "bazel-testlogs").rglob("test.log"), (ws_root / "tests-report").rglob("test.log")):
   ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the tip, changed as suggested

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

general comment to all cpp tests:

If its requirements-based why there is no mapping to the requirement that is tested by this testcase?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All tests that are suppose to be parsed by the testlinker are required to either have 'PartiallyVerified' or 'FullyVerified' as properties that links existing requirements

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now there are only dummy requirements, so I didn't link any yet. I planned to either link requirements or adapt the TestType as soon as we have requirements, should I do it differently?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there are no real requirements just go with explorative-testing derivation technique and update test type accordingly.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to test then if docs-as-code breaks in the version you have.
Not sure anymore if it is demanding the correct signature there already or only in 3.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to explorative-testing and interface-test



def setup(app):
app.add_directive("display-test-logs", DisplayTestLogs)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be really slow at some point depending on how large the logs are and how many of these directives you have.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I would kick this out as soon as it slows down the doc build significantly, but I think it's useful to have as long as it's feasible

@pawelrutkaq
Copy link
Contributor

This will imho have to wait a bit for a resolution on the general issues doc + cov - > eclipse-score/cicd-workflows#63

"@score_platform//:needs_json",
"@score_process//:needs_json", # This allows linking to requirements (wp__requirements_comp, etc.) from the process_description repository.
],
source_dir = "docs",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have expected some like deps = [":test_logs"], here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I suppose it will work either way, decided to go with this approach as it is already done like this in persistency

@eelcoem eelcoem temporarily deployed to workflow-approval February 18, 2026 13:28 — with GitHub Actions Inactive
@eelcoem eelcoem temporarily deployed to workflow-approval February 18, 2026 13:28 — with GitHub Actions Inactive
@eelcoem eelcoem temporarily deployed to workflow-approval February 18, 2026 13:36 — with GitHub Actions Inactive
@eelcoem eelcoem temporarily deployed to workflow-approval February 18, 2026 13:36 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

Comments