add InterUSS uss_qualifier qualification pipeline#112
Merged
Conversation
🧠 Code Complexity ReportAverage cyclomatic complexity: A (2.206657420249653) High-complexity blocks (grade C and above)Low maintainability files |
🚨 Bandit results0 issues found () Show details |
🛡️ pip-audit results✅ No vulnerabilities found. |
📊 Test Coverage Report🔴 Overall coverage: 48.6% Files with lowest coverage (<50%)
|
🐳 Trivy Container Scan🚨 Total: 38 (MEDIUM: 29, HIGH: 7, CRITICAL: 2) Show details |
Adds GitHub Actions workflow and local run script for testing Flight Blender against InterUSS uss_qualifier for ASTM F3548-21 (SCD) and F3411-22a (NetRID). Infrastructure added: - .github/workflows/interuss-qualification.yml — CI workflow triggered on push to main/develop, PRs to main, and on-demand dispatch - testing/interuss/docker-compose.yml — Flight Blender test stack (postgres, valkey, celery, flight-blender with hostname flight-blender.localutm) - testing/interuss/.env.interuss — environment variables for test stack - testing/interuss/configs/f3548_flight_blender.yaml — uss_qualifier config for ASTM F3548-21 Strategic Conflict Detection - testing/interuss/configs/netrid_v22a_flight_blender.yaml — uss_qualifier config for ASTM F3411-22a Remote ID v22a - testing/interuss/scripts/run_interuss_tests.sh — local replica of CI workflow - testing/interuss/scripts/report_to_summary.py — parse raw report JSON and emit GitHub Step Summary Markdown (pass/fail per scenario and participant) Verified locally: both F3548 and NetRID v22a test suites run to completion. Failures are expected (dummy-oauth JWT incompatibility) and captured in the report. Known incompatibilities captured as test failures: - dummy-oauth JWTs lack 'kid' header field required by Flight Blender - dummy-oauth does not expose /.well-known/jwks.json required by Flight Blender
Three bugs fixed in the InterUSS qualification workflow: 1. Network subnet conflict: interop_ecosystem_network was created with --subnet=172.27.0.0/16, conflicting with dss_internal_network which also uses that subnet. Fixed by creating interop_ecosystem_network as a plain bridge network (no subnet). 2. Broken dummy-oauth health check: checked /.well-known/jwks.json which returns 404 on dummy-oauth, causing 'curl -sf' to always fail. After 60 retries (180s) the step exited 1 — this was the cause of the 3m 29s failure. Fixed to check any HTTP response on port 8085 (same pattern as Flight Blender health check). 3. Wrong artifact upload paths: referenced flat .json/.html files but actual uss_qualifier output is nested directories (output/f3548/report.json, output/f3548/f3548_requirements/, output/f3548/sequence/, etc.).
Adds a step that posts (or updates) a PR comment with the full uss_qualifier report after each run. The comment is generated by the same Python script used for the Step Summary. On re-runs, the existing bot comment is replaced rather than adding duplicates.
atti92
force-pushed
the
feat/issue-95-interuss-qualification-pipeline
branch
from
May 29, 2026 22:13
1c9e575 to
388dc45
Compare
There was a problem hiding this comment.
Pull request overview
Adds an InterUSS uss_qualifier qualification setup for Flight Blender to support ongoing compatibility checks against ASTM SCD and NetRID suites.
Changes:
- Adds a new GitHub Actions workflow and local script to run InterUSS qualification tests.
- Adds InterUSS-specific Docker Compose, environment, and uss_qualifier configuration files.
- Updates Django from 5.2.13 to 5.2.14 and removes the dependency-update workflow.
Reviewed changes
Copilot reviewed 9 out of 11 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/interuss-qualification.yml |
Adds CI workflow for InterUSS qualification runs and report publishing. |
.github/workflows/dependency-update.yml |
Removes scheduled dependency update automation. |
.gitignore |
Ignores generated InterUSS qualification output. |
pyproject.toml |
Bumps Django dependency version. |
uv.lock |
Updates locked Django package metadata. |
testing/interuss/.env.interuss |
Adds test-only environment settings for InterUSS runs. |
testing/interuss/docker-compose.yml |
Adds local test stack for Flight Blender, Postgres, Redis, and Celery. |
testing/interuss/configs/f3548_flight_blender.yaml |
Adds F3548-21 SCD uss_qualifier configuration. |
testing/interuss/configs/netrid_v22a_flight_blender.yaml |
Adds NetRID v22a uss_qualifier configuration. |
testing/interuss/scripts/report_to_summary.py |
Adds report JSON to Markdown summary conversion. |
testing/interuss/scripts/run_interuss_tests.sh |
Adds local end-to-end runner for InterUSS qualification tests. |
InterUSS Qualification ReportRun #6 | Commit InterUSS uss_qualifier —
|
| Result | Count |
|---|---|
| ✅ Pass | 4596 |
| ❌ Fail | 18 |
| Total | 4614 (100% pass) |
Per-scenario breakdown
| Scenario | ✅ Pass | ❌ Fail | Execution error |
|---|---|---|---|
| Get system versions | 1 | 1 | No |
| ASTM F3548 flight planners preparation | 38 | 0 | No |
| ASTM SCD DSS: Operational Intent Explicit Subscription handling | 45 | 0 | No |
| ASTM SCD DSS: Implicit Subscription handling | 121 | 0 | No |
| ASTM SCD DSS: Operational Intent Reference Simple | 114 | 0 | No |
| ASTM SCD DSS: Constraint Reference Simple | 10 | 0 | No |
| ASTM SCD DSS: Constraint Reference Synchronization | 227 | 0 | No |
| ASTM SCD DSS: USS Availability Synchronization | 37 | 0 | No |
| ASTM F3548-21 UTM DSS Operational Intent Reference State Transitions | 17 | 0 | No |
| ASTM SCD DSS: Subscription and entity deletion interaction | 73 | 0 | No |
| ASTM SCD DSS: Subscription and entity interaction | 102 | 0 | No |
| ASTM SCD DSS: Operational Intent Reference Key Validation | 32 | 0 | No |
| ASTM SCD DSS: Operational Intent Reference Synchronization | 126 | 0 | No |
| ASTM SCD DSS: Interfaces authentication | 182 | 0 | No |
| ASTM SCD DSS: Subscription Simple | 403 | 0 | No |
| ASTM SCD DSS: Subscription Validation | 7 | 0 | No |
| ASTM F3548-21 UTM DSS Operational Intent Reference Access Control | 18 | 0 | No |
| ASTM F3548-21 UTM DSS interoperability | 2 | 2 | No |
| ASTM SCD DSS: Subscription Synchronization | 398 | 0 | No |
| ? | 0 | 0 | No |
| OVN Request Optional Extension to ASTM F3548-21 | 12 | 0 | No |
| ASTM SCD DSS: Report | 2 | 0 | No |
| ASTM Availability DSS: USS Availability Mutation | 4 | 0 | No |
| ASTM SCD DSS: Operational Intent Explicit Subscription handling | 45 | 0 | No |
| ASTM SCD DSS: Implicit Subscription handling | 121 | 0 | No |
| ASTM SCD DSS: Operational Intent Reference Simple | 114 | 0 | No |
| ASTM SCD DSS: Constraint Reference Simple | 10 | 0 | No |
| ASTM SCD DSS: Constraint Reference Synchronization | 227 | 0 | No |
| ASTM SCD DSS: USS Availability Synchronization | 38 | 0 | No |
| ASTM F3548-21 UTM DSS Operational Intent Reference State Transitions | 17 | 0 | No |
| ASTM SCD DSS: Subscription and entity deletion interaction | 73 | 0 | No |
| ASTM SCD DSS: Subscription and entity interaction | 102 | 0 | No |
| ASTM SCD DSS: Operational Intent Reference Key Validation | 32 | 0 | No |
| ASTM SCD DSS: Operational Intent Reference Synchronization | 126 | 0 | No |
| ASTM SCD DSS: Interfaces authentication | 182 | 0 | No |
| ASTM SCD DSS: Subscription Simple | 403 | 0 | No |
| ASTM SCD DSS: Subscription Validation | 7 | 0 | No |
| ASTM F3548-21 UTM DSS Operational Intent Reference Access Control | 18 | 0 | No |
| ASTM F3548-21 UTM DSS interoperability | 2 | 2 | No |
| ASTM SCD DSS: Subscription Synchronization | 398 | 0 | No |
| ? | 0 | 0 | No |
| OVN Request Optional Extension to ASTM F3548-21 | 12 | 0 | No |
| ASTM SCD DSS: Report | 2 | 0 | No |
| ASTM Availability DSS: USS Availability Mutation | 4 | 0 | No |
| Validation of operational intents | 7 | 1 | No |
| Validation of operational intents | 38 | 0 | No |
| Nominal planning: conflict with higher priority | 4 | 1 | No |
| Nominal planning: conflict with higher priority | 20 | 1 | No |
| Nominal planning: conflict with higher priority | 4 | 1 | No |
| Nominal planning: conflict with higher priority | 175 | 0 | No |
| Nominal planning: not permitted conflict with equal priority | 4 | 1 | No |
| Nominal planning: not permitted conflict with equal priority | 36 | 1 | No |
| Nominal planning: not permitted conflict with equal priority | 4 | 1 | No |
| Nominal planning: not permitted conflict with equal priority | 131 | 0 | No |
| Data Validation of GET operational intents by USS | 18 | 1 | No |
| Data Validation of GET operational intents by USS | 53 | 0 | No |
| Awareness of relevant operational intents | 2 | 1 | No |
| Awareness of relevant operational intents | 69 | 0 | No |
| Off-Nominal planning: down USS | 14 | 1 | No |
| Off-Nominal planning: down USS | 29 | 0 | No |
| Off-Nominal planning: down USS with equal priority conflicts not permitted | 15 | 1 | No |
| Off-Nominal planning: down USS with equal priority conflicts not permitted | 20 | 0 | No |
| ASTM F3548 flight planners preparation | 38 | 0 | No |
| ASTM F3548 makeUssReport | 1 | 0 | No |
| ASTM F3548-21 evaluate system versions | 5 | 2 | No |
| ASTM F3548 UTM aggregate checks | 5 | 0 | No |
Failed checks per participant
| Participant | Failed checks | Unique requirements |
|---|---|---|
flight_blender |
14 | 2 |
uss1_dss |
2 | 1 |
uss2_dss |
2 | 1 |
Failed checks (18 total, showing first 18)
| Check | Participants | Requirements | Summary |
|---|---|---|---|
| 🟡 Valid response | flight_blender |
versioning.ReportSystemVersion |
Error querying version |
| 🟡 DSS instance is publicly addressable | uss1_dss |
astm.f3548.v21.DSS0300 |
DSS host dss1.uss1.localutm is not publicly addressable |
| 🟡 DSS instance is publicly addressable | uss2_dss |
astm.f3548.v21.DSS0300 |
DSS host dss1.uss2.localutm is not publicly addressable |
| 🟡 DSS instance is publicly addressable | uss2_dss |
astm.f3548.v21.DSS0300 |
DSS host dss1.uss2.localutm is not publicly addressable |
| 🟡 DSS instance is publicly addressable | uss1_dss |
astm.f3548.v21.DSS0300 |
DSS host dss1.uss1.localutm is not publicly addressable |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🔴 Failure | flight_blender |
interuss.automated_testing.flight_planning.ExpectedBehavior |
Flight planning activity PlanningActivityResult.Failed leaving flight plan Fligh |
| 🟡 Valid response | flight_blender |
versioning.ReportSystemVersion |
Error querying version |
| 🟡 Valid response | flight_blender |
versioning.ReportSystemVersion |
Error querying version |
InterUSS uss_qualifier — netrid_v22a
Codebase: interuss/monitoring/v0.30.0 (commit d1166f192120)
Started: 2026-05-31T18:08:51.430595Z
Ended: 2026-05-31T18:21:02.207997Z
Overall result: ❌ FAIL
Check results
| Result | Count |
|---|---|
| ✅ Pass | 7602 |
| ❌ Fail | 87 |
| Total | 7689 (99% pass) |
Per-scenario breakdown
| Scenario | ✅ Pass | ❌ Fail | Execution error |
|---|---|---|---|
| ? | 0 | 0 | No |
| ASTM NetRID Service Provider operator notification under slow update rate | 32 | 1 | No |
| ASTM NetRID Display Provider behavior | 19 | 0 | No |
| ASTM NetRID networked UAS disconnection | 43 | 45 | No |
| ASTM NetRID Service Provider operator notification on missing fields | 66 | 0 | No |
| ASTM NetRID Service Provider notification behavior | 12 | 1 | No |
| ASTM NetRID nominal behavior | 195 | 40 | No |
| ASTM NetRID SP clients misbehavior handling | 59 | 0 | No |
| ASTM NetRID DSS: Endpoint encryption | 0 | 0 | No |
| ASTM NetRID DSS: Simple ISA | 80 | 0 | No |
| ASTM NetRID DSS: Submitted ISA Validations | 10 | 0 | No |
| ASTM NetRID DSS: ISA Expiry | 6 | 0 | No |
| ASTM NetRID DSS: ISA Subscription Interactions | 30 | 0 | No |
| ASTM NetRID DSS: Subscription Validation | 19 | 0 | No |
| ASTM NetRID DSS: Subscription Simple | 382 | 0 | No |
| ASTM F3411-22a NetRID DSS interoperability | 125 | 0 | No |
| ASTM NetRID DSS: Token Validation | 25 | 0 | No |
| ? | 0 | 0 | No |
| ASTM NetRID DSS: Concurrent Requests | 2910 | 0 | No |
| ASTM NetRID DSS: Endpoint encryption | 0 | 0 | No |
| ASTM NetRID DSS: Simple ISA | 80 | 0 | No |
| ASTM NetRID DSS: Submitted ISA Validations | 10 | 0 | No |
| ASTM NetRID DSS: ISA Expiry | 6 | 0 | No |
| ASTM NetRID DSS: ISA Subscription Interactions | 30 | 0 | No |
| ASTM NetRID DSS: Subscription Validation | 19 | 0 | No |
| ASTM NetRID DSS: Subscription Simple | 382 | 0 | No |
| ASTM F3411-22a NetRID DSS interoperability | 125 | 0 | No |
| ASTM NetRID DSS: Token Validation | 25 | 0 | No |
| ? | 0 | 0 | No |
| ASTM NetRID DSS: Concurrent Requests | 2910 | 0 | No |
| ASTM F3411-22a NetRID aggregate checks | 2 | 0 | No |
Failed checks per participant
| Participant | Failed checks | Unique requirements |
|---|---|---|
flight_blender |
87 | 8 |
Failed checks (87 total, showing first 30)
| Check | Participants | Requirements | Summary |
|---|---|---|---|
| 🟡 Operator notification present | flight_blender |
astm.f3411.v22a.NET0040 |
No operator notification found |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
| 🟡 Disconnected flight is shown as such | flight_blender |
astm.f3411.v22a.NET0320 |
Expected flight not observed |
| 🟡 Missing flight | flight_blender |
astm.f3411.v22a.NET0610, astm.f3411.v22a.NET0710,1, astm.f3411.v22a.NET0340, astm.f3411.v22a.NET0260,Table1,10, astm.f3411.v22a.NET0260,Table1,11 |
Expected flight not observed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #95