[IsaacLab CI] upload omni-github test-result artifacts from CI jobs - #6234
Conversation
Greptile SummaryThis PR adds a new
Confidence Score: 5/5Safe to merge; all new logic is additive CI instrumentation with graceful fallbacks that cannot block existing test reporting. The converter correctly handles passed, skipped, failed, and error test states. The composite action gracefully skips upload when the JUnit file is missing, the schema validation venv is self-contained, and callers all set .github/actions/upload-omni-github-test-results/action.yml — two URL inputs bypass the env-var pattern used by all other inputs in the same run block. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Job as CI Job (install-ci-run / run-tests)
participant JUnit as Upload JUnit XML actions/upload-artifact@v7
participant Comp as Upload Comparison Images actions/upload-artifact@v7
participant Convert as Convert JUnit XML (bash + python3)
participant Schema as Schema Validation (jsonschema venv)
participant Upload as Upload omni-github artifact actions/upload-artifact@v7
participant OmniGH as omni-github dashboard
Job->>JUnit: results.xml
JUnit-->>Convert: artifact-url (junit-log-url)
Job->>Comp: comparison-images/ (run-tests only)
Comp-->>Convert: artifact-url (comparison-images-url)
Convert->>Convert: derive app_platform from RUNNER_OS/RUNNER_ARCH
Convert->>Convert: python3 junit_to_omni_github_results.py
Convert->>Schema: pip install jsonschema in venv
Schema->>Schema: validate test_results.json vs result-json.schema.json
alt "validation passes and tests > 0"
Schema-->>Convert: "upload=true"
Convert->>Upload: artifact_dir
Upload-->>OmniGH: "artifact name = prefix--v1-{repo_id}-{run_id}-{attempt}-{check_run_id}"
else no JUnit file or empty suite or schema fail
Schema-->>Convert: "upload=false warning emitted"
end
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Job as CI Job (install-ci-run / run-tests)
participant JUnit as Upload JUnit XML actions/upload-artifact@v7
participant Comp as Upload Comparison Images actions/upload-artifact@v7
participant Convert as Convert JUnit XML (bash + python3)
participant Schema as Schema Validation (jsonschema venv)
participant Upload as Upload omni-github artifact actions/upload-artifact@v7
participant OmniGH as omni-github dashboard
Job->>JUnit: results.xml
JUnit-->>Convert: artifact-url (junit-log-url)
Job->>Comp: comparison-images/ (run-tests only)
Comp-->>Convert: artifact-url (comparison-images-url)
Convert->>Convert: derive app_platform from RUNNER_OS/RUNNER_ARCH
Convert->>Convert: python3 junit_to_omni_github_results.py
Convert->>Schema: pip install jsonschema in venv
Schema->>Schema: validate test_results.json vs result-json.schema.json
alt "validation passes and tests > 0"
Schema-->>Convert: "upload=true"
Convert->>Upload: artifact_dir
Upload-->>OmniGH: "artifact name = prefix--v1-{repo_id}-{run_id}-{attempt}-{check_run_id}"
else no JUnit file or empty suite or schema fail
Schema-->>Convert: "upload=false warning emitted"
end
Reviews (16): Last reviewed commit: "skip upload on empty tests" | Re-trigger Greptile |
…or-nvidia/IsaacLab into mataylor/omni-github-test-results
ba6ba0e to
245ebd1
Compare
| # Validating with a local pinned copy of the schemas is recommended by the omni-github team. | ||
| schema_venv="${RUNNER_TEMP}/omni-github-jsonschema-venv" | ||
| python3 -m venv "$schema_venv" | ||
| "$schema_venv/bin/python" -m pip install -q jsonschema |
There was a problem hiding this comment.
polluting isaac_sim venv with jsonschema dep causes errors
Description
Add
upload-omni-github-test-resultscomposite action to convert JUnit XML into the omni-github test-result artifact contract.Wire the upload step into install-ci, package test, and Docker test composite actions.
Use IsaacLab upstream repository ID
567038244in artifact identities for omni-github registration.Installation Tests workflow uploads
pytest-results-*--v1-567038244-*artifactsomni-github registration PR can reference real sample artifact names from this run
link to slack discussion:
https://nvidia.slack.com/archives/C0B16FFDDBP/p1782216333267729
link to artifact schema:
https://github.com/NVIDIA-Omniverse/omni-github/blob/main/docs/test-results/clients/artifact-upload/artifact-schema.md
reference to junit format: https://github.com/testmoapp/junitxml#structure
link to test results dashboard:
https://omni-github-stg.horde-aks.nvidia.com/test-results/repository-test-summary?repository=isaac-sim%2FIsaacLab&days=30&prMode=include&groupBy=job_name&groupValue=isaaclab+%28core%29+%5B3%2F3%5D
example artifact from most recent pipeline run:
rendering correctness test: test_results.json
asset test: https://github.com/isaac-sim/IsaacLab/actions/runs/28196987278/artifacts/7890469555
corresponding golden images are uploaded with
Fixes # (OMPE-98494)
Type of change
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there