Moved scenarios to folders - #110
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends scenario support across the backend and web editor to allow scenarios to live in nested subfolders (e.g., suite_a/scenario_1.yaml), enabling cleaner organization and easier merging across forks.
Changes:
- Backend: list/load/save scenarios using subfolder-relative scenario IDs and update docs lookup for nested paths.
- Tooling/tests: recursively discover YAML scenarios and generate docs into matching subfolder structures.
- Web editor: display scenarios grouped by folder when suite grouping is not available.
Reviewed changes
Copilot reviewed 6 out of 27 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| web-editor/src/components/ScenarioEditor/ScenarioList.tsx | Adds folder-based grouping UI for scenario IDs containing / and adjusts item display name. |
| tests/test_yaml_scenarios.py | Switches YAML discovery to recursive search and improves parametrized test IDs. |
| src/openutm_verification/server/router.py | Updates scenario list/suites endpoints for subfolders; adds {scenario:path} docs route; enables saving scenarios into subfolders. |
| src/openutm_verification/core/execution/scenario_loader.py | Adds path traversal protection + bare-name fallback search across subfolders. |
| scripts/generate_docs.py | Generates markdown docs for scenarios recursively, mirroring subfolder layout. |
| docs/index.md | Adds “SDSP Sensor Failure” to docs index (links still assume flat docs layout). |
| scenarios/standard-scenaios/F5_non_conforming_path.yaml | Adds standard scenario YAML under a “standard-*” folder. |
| scenarios/standard-scenaios/F3_non_conforming_path.yaml | Adds standard scenario YAML under a “standard-*” folder. |
| scenarios/standard-scenaios/F2_contingent_path.yaml | Adds standard scenario YAML under a “standard-*” folder. |
| scenarios/standard-scenaios/F1_happy_path.yaml | Adds standard scenario YAML under a “standard-*” folder. |
| scenarios/standard-scenaios/F1_flow_no_telemetry_with_user_input.yaml | Adds standard scenario YAML under a “standard-*” folder. |
| scenarios/sdsp-f3623/verify_sdsp_metrics.yaml | Adds SDSP scenario YAML under an SDSP subfolder. |
| scenarios/sdsp-f3623/sdsp_verify_sensor_failure_report.yaml | Adds SDSP sensor-failure scenario YAML under an SDSP subfolder. |
| scenarios/sdsp-f3623/sdsp_track.yaml | Adds SDSP track scenario YAML under an SDSP subfolder. |
| scenarios/sdsp-f3623/sdsp_heartbeat.yaml | Adds SDSP heartbeat scenario YAML under an SDSP subfolder. |
| scenarios/geo-fence/geo_fence_upload.yaml | Adds geo-fence scenario YAML under a geo-fence subfolder. |
| scenarios/flight-declarations/bulk_add_flight_declarations_via_operational_intents.yaml | Adds flight-declaration scenario YAML under a flight-declarations subfolder. |
| scenarios/flight-declarations/bulk_add_flight_declarations.yaml | Adds flight-declaration scenario YAML under a flight-declarations subfolder. |
| scenarios/flight-declarations/add_flight_declaration_via_operational_intent.yaml | Adds flight-declaration scenario YAML under a flight-declarations subfolder. |
| scenarios/flight-declarations/add_flight_declaration.yaml | Adds flight-declaration scenario YAML under a flight-declarations subfolder. |
| scenarios/airtraffic-simulations/stream_air_traffic_example.yaml | Adds air-traffic scenario YAML under an airtraffic-simulations subfolder. |
| scenarios/airtraffic-simulations/openutm_sim_air_traffic_data.yaml | Adds air-traffic scenario YAML under an airtraffic-simulations subfolder. |
| scenarios/airtraffic-simulations/opensky_live_data.yaml | Adds air-traffic scenario YAML under an airtraffic-simulations subfolder. |
| scenarios/airtraffic-simulations/bluesky_sim_air_traffic_data_latency_issues.yaml | Adds air-traffic scenario YAML under an airtraffic-simulations subfolder. |
| scenarios/airtraffic-simulations/bluesky_sim_air_traffic_data.yaml | Adds air-traffic scenario YAML under an airtraffic-simulations subfolder. |
| scenarios/airtraffic-simulations/bayesian_sim_air_traffic_data_varying_refresh_rates.yaml | Adds air-traffic scenario YAML under an airtraffic-simulations subfolder. |
| scenarios/airtraffic-simulations/bayesian_sim_air_traffic_data.yaml | Adds air-traffic scenario YAML under an airtraffic-simulations subfolder. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+38
to
+42
| """List all available scenarios, including those in sub-folders.""" | ||
| path = get_scenarios_directory() | ||
| if not path.exists(): | ||
| return [] | ||
| return [f.stem for f in path.glob("*.yaml")] | ||
| return [str(f.relative_to(path).with_suffix("")) for f in sorted(path.rglob("*.yaml"))] |
| else: | ||
| raise HTTPException(status_code=404, detail="Documentation not found") | ||
|
|
||
| with open(file_path, "r") as f: |
Comment on lines
+130
to
+146
| <div | ||
| key={name} | ||
| className={styles.nodeItem} | ||
| onClick={() => handleLoad(name)} | ||
| role="button" | ||
| tabIndex={0} | ||
| title={name} | ||
| style={{ | ||
| cursor: 'pointer', | ||
| opacity: loading ? 0.5 : 1, | ||
| borderColor: name === currentScenarioName ? 'var(--accent-primary)' : 'var(--border-color)', | ||
| backgroundColor: name === currentScenarioName ? 'var(--bg-secondary)' : 'var(--bg-primary)' | ||
| }} | ||
| > | ||
| <FileText size={16} color={name === currentScenarioName ? "var(--accent-primary)" : "#8b949e"} /> | ||
| <span>{displayName.replace(/_/g, ' ').replace(/\b\w/g, l => l.toUpperCase())}</span> | ||
| </div> |
Comment on lines
+185
to
+218
| ) : folderGroups.some(([folder]) => folder !== '') ? ( | ||
| folderGroups.map(([folder, items]) => { | ||
| const isCollapsed = collapsedSuites.has(`__folder__${folder}`); | ||
| const label = folder === '' | ||
| ? 'Root' | ||
| : folder.replace(/\//g, ' / ').replace(/_/g, ' ').replace(/\b\w/g, l => l.toUpperCase()); | ||
| return folder === '' ? ( | ||
| <div key="root" style={{ display: 'flex', flexDirection: 'column', gap: '8px' }}> | ||
| {items.map(renderScenarioItem)} | ||
| </div> | ||
| ) : ( | ||
| <div key={folder} style={{ marginBottom: '4px' }}> | ||
| <button | ||
| type="button" | ||
| className={styles.groupHeader} | ||
| onClick={() => toggleSuite(`__folder__${folder}`)} | ||
| aria-expanded={!isCollapsed} | ||
| style={{ padding: '6px 4px', marginTop: 4, marginBottom: 4, background: 'none', border: 'none', width: '100%' }} | ||
| > | ||
| {isCollapsed ? <ChevronRight size={14} /> : <ChevronDown size={14} />} | ||
| <FolderOpen size={14} /> | ||
| {label} | ||
| <span style={{ marginLeft: 'auto', fontSize: '11px', fontWeight: 400, opacity: 0.7 }}> | ||
| {items.length} | ||
| </span> | ||
| </button> | ||
| {!isCollapsed && ( | ||
| <div style={{ paddingLeft: '8px', display: 'flex', flexDirection: 'column', gap: '8px' }}> | ||
| {items.map(renderScenarioItem)} | ||
| </div> | ||
| )} | ||
| </div> | ||
| ); | ||
| }) |
| * [OpenUTM Sim Air Traffic Data](scenarios/openutm_sim_air_traffic_data.md) | ||
| * [SDSP Heartbeat](scenarios/sdsp_heartbeat.md) | ||
| * [SDSP Track](scenarios/sdsp_track.md) | ||
| * [SDSP Sensor Failure](scenarios/sdsp_verify_sensor_failure_report.md) |
Comment on lines
+92
to
100
| @scenario_router.post("/api/scenarios/{name:path}") | ||
| async def save_scenario(name: str, scenario: ScenarioDefinition): | ||
| """Save a scenario to a YAML file.""" | ||
| path = get_scenarios_directory() | ||
| file_path = (path / name).with_suffix(".yaml") | ||
|
|
||
| # Ensure directory exists | ||
| path.mkdir(parents=True, exist_ok=True) | ||
| # Ensure directory exists (including any sub-folder) | ||
| file_path.parent.mkdir(parents=True, exist_ok=True) | ||
|
|
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds support for organizing YAML scenarios into subfolders so forks can structure scenario sets more cleanly while still being loadable via the API and web editor.
Changes:
- Extend scenario discovery/loading to recurse into scenario subfolders and expose path-relative scenario IDs via the API.
- Update web editor scenario list to display scenarios grouped by folder when suites aren’t present.
- Update docs generation + YAML execution tests to discover scenarios recursively; add multiple scenarios into new subdirectories.
Reviewed changes
Copilot reviewed 6 out of 27 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| web-editor/src/components/ScenarioEditor/ScenarioList.tsx | Groups scenarios by folder for display and shows basename in the list. |
| tests/test_yaml_scenarios.py | Recursively discovers YAML scenarios for execution tests. |
| src/openutm_verification/server/router.py | Updates scenario/suite APIs for subfolder IDs; adds path-based docs route; allows saving to subfolders. |
| src/openutm_verification/core/execution/scenario_loader.py | Loads scenarios by path-relative ID and supports searching subfolders for bare names. |
| scripts/generate_docs.py | Generates scenario docs mirroring the scenarios subfolder structure. |
| docs/index.md | Adds an entry for an SDSP sensor failure scenario doc. |
| scenarios/standard-scenaios/F5_non_conforming_path.yaml | Adds standard scenario YAML under a subfolder. |
| scenarios/standard-scenaios/F3_non_conforming_path.yaml | Adds standard scenario YAML under a subfolder. |
| scenarios/standard-scenaios/F2_contingent_path.yaml | Adds standard scenario YAML under a subfolder. |
| scenarios/standard-scenaios/F1_happy_path.yaml | Adds standard scenario YAML under a subfolder. |
| scenarios/standard-scenaios/F1_flow_no_telemetry_with_user_input.yaml | Adds standard scenario YAML under a subfolder. |
| scenarios/sdsp-f3623/verify_sdsp_metrics.yaml | Adds SDSP metrics verification scenario under a subfolder. |
| scenarios/sdsp-f3623/sdsp_verify_sensor_failure_report.yaml | Adds SDSP sensor failure scenario under a subfolder. |
| scenarios/sdsp-f3623/sdsp_track.yaml | Adds SDSP track scenario under a subfolder. |
| scenarios/sdsp-f3623/sdsp_heartbeat.yaml | Adds SDSP heartbeat scenario under a subfolder. |
| scenarios/geo-fence/geo_fence_upload.yaml | Adds geo-fence scenario under a subfolder. |
| scenarios/flight-declarations/bulk_add_flight_declarations_via_operational_intents.yaml | Adds flight declaration bulk scenario under a subfolder. |
| scenarios/flight-declarations/bulk_add_flight_declarations.yaml | Adds flight declaration bulk scenario under a subfolder. |
| scenarios/flight-declarations/add_flight_declaration_via_operational_intent.yaml | Adds flight declaration scenario under a subfolder. |
| scenarios/flight-declarations/add_flight_declaration.yaml | Adds flight declaration scenario under a subfolder. |
| scenarios/airtraffic-simulations/stream_air_traffic_example.yaml | Adds air-traffic simulation example scenario under a subfolder. |
| scenarios/airtraffic-simulations/openutm_sim_air_traffic_data.yaml | Adds air-traffic simulation scenario under a subfolder. |
| scenarios/airtraffic-simulations/opensky_live_data.yaml | Adds OpenSky scenario under a subfolder. |
| scenarios/airtraffic-simulations/bluesky_sim_air_traffic_data_latency_issues.yaml | Adds BlueSky latency scenario under a subfolder. |
| scenarios/airtraffic-simulations/bluesky_sim_air_traffic_data.yaml | Adds BlueSky scenario under a subfolder. |
| scenarios/airtraffic-simulations/bayesian_sim_air_traffic_data_varying_refresh_rates.yaml | Adds Bayesian varying refresh scenario under a subfolder. |
| scenarios/airtraffic-simulations/bayesian_sim_air_traffic_data.yaml | Adds Bayesian scenario under a subfolder. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces support for organizing scenario YAML definitions into subfolders (path-relative scenario IDs), enabling cleaner merges across forks while keeping the editor and API compatible with nested scenario layouts.
Changes:
- Update backend scenario APIs to list/load/save scenarios using subfolder-relative IDs (
{scenario:path}) and extend docs lookup for nested paths. - Update web editor scenario list UI to group scenarios by folder when suites aren’t present.
- Update tests and docs generation to recurse into scenario subfolders; add new grouped scenario YAMLs and corresponding docs pages.
Reviewed changes
Copilot reviewed 6 out of 42 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| web-editor/src/components/ScenarioEditor/ScenarioList.tsx | Groups scenario list by folder paths and updates item rendering to use <button> with leaf display names. |
| tests/test_yaml_scenarios.py | Recursively discovers YAML scenarios under subfolders and uses relative-path IDs for parametrized test names. |
| src/openutm_verification/server/router.py | Switches scenario routes to {scenario:path}/{name:path}, lists scenarios via rglob, resolves suite scenario IDs, and hardens docs/save path handling. |
| src/openutm_verification/core/execution/scenario_loader.py | Adds traversal protection, supports loading subfolder scenario IDs, and provides bare-name fallback lookup with ambiguity detection. |
| scripts/generate_docs.py | Generates docs for nested scenarios by preserving relative subpaths under docs/scenarios/. |
| scenarios/standard-scenaios/F5_non_conforming_path.yaml | Adds a standard scenario YAML under a subfolder. |
| scenarios/standard-scenaios/F3_non_conforming_path.yaml | Adds a standard scenario YAML under a subfolder. |
| scenarios/standard-scenaios/F2_contingent_path.yaml | Adds a standard scenario YAML under a subfolder. |
| scenarios/standard-scenaios/F1_happy_path.yaml | Adds a standard scenario YAML under a subfolder. |
| scenarios/standard-scenaios/F1_flow_no_telemetry_with_user_input.yaml | Adds a standard scenario YAML under a subfolder. |
| scenarios/sdsp-f3623/verify_sdsp_metrics.yaml | Adds an SDSP scenario YAML under a subfolder. |
| scenarios/sdsp-f3623/sdsp_verify_sensor_failure_report.yaml | Adds an SDSP scenario YAML under a subfolder. |
| scenarios/sdsp-f3623/sdsp_track.yaml | Adds an SDSP scenario YAML under a subfolder. |
| scenarios/sdsp-f3623/sdsp_heartbeat.yaml | Adds an SDSP scenario YAML under a subfolder. |
| scenarios/geo-fence/geo_fence_upload.yaml | Adds a geo-fence scenario YAML under a subfolder. |
| scenarios/flight-declarations/bulk_add_flight_declarations_via_operational_intents.yaml | Adds a flight-declarations scenario YAML under a subfolder. |
| scenarios/flight-declarations/bulk_add_flight_declarations.yaml | Adds a flight-declarations scenario YAML under a subfolder. |
| scenarios/flight-declarations/add_flight_declaration_via_operational_intent.yaml | Adds a flight-declarations scenario YAML under a subfolder. |
| scenarios/flight-declarations/add_flight_declaration.yaml | Adds a flight-declarations scenario YAML under a subfolder. |
| scenarios/airtraffic-simulations/stream_air_traffic_example.yaml | Adds an air-traffic scenario YAML under a subfolder. |
| scenarios/airtraffic-simulations/openutm_sim_air_traffic_data.yaml | Adds an air-traffic scenario YAML under a subfolder. |
| scenarios/airtraffic-simulations/opensky_live_data.yaml | Adds an air-traffic scenario YAML under a subfolder. |
| scenarios/airtraffic-simulations/bluesky_sim_air_traffic_data_latency_issues.yaml | Adds an air-traffic scenario YAML under a subfolder. |
| scenarios/airtraffic-simulations/bluesky_sim_air_traffic_data.yaml | Adds an air-traffic scenario YAML under a subfolder. |
| scenarios/airtraffic-simulations/bayesian_sim_air_traffic_data_varying_refresh_rates.yaml | Adds an air-traffic scenario YAML under a subfolder. |
| scenarios/airtraffic-simulations/bayesian_sim_air_traffic_data.yaml | Adds an air-traffic scenario YAML under a subfolder. |
| docs/scenarios/standard-scenarios/F5_non_conforming_path.md | Adds documentation page for a standard scenario. |
| docs/scenarios/standard-scenarios/F3_non_conforming_path.md | Adds documentation page for a standard scenario. |
| docs/scenarios/standard-scenarios/F2_contingent_path.md | Adds documentation page for a standard scenario. |
| docs/scenarios/standard-scenarios/F1_happy_path.md | Adds documentation page for a standard scenario. |
| docs/scenarios/standard-scenarios/F1_flow_no_telemetry_with_user_input.md | Adds documentation page for a standard scenario. |
| docs/scenarios/sdsp-f3623/verify_sdsp_metrics.md | Adds documentation page for an SDSP scenario. |
| docs/scenarios/sdsp-f3623/sdsp_verify_sensor_failure_report.md | Adds documentation page for an SDSP scenario. |
| docs/scenarios/sdsp-f3623/sdsp_track.md | Adds documentation page for an SDSP scenario. |
| docs/scenarios/sdsp-f3623/sdsp_heartbeat.md | Adds documentation page for an SDSP scenario. |
| docs/scenarios/geo-fence/geo_fence_upload.md | Adds documentation page for a geo-fence scenario. |
| docs/scenarios/flight-declarations/add_flight_declaration_via_operational_intent.md | Adds documentation page for a flight-declarations scenario. |
| docs/scenarios/flight-declarations/add_flight_declaration.md | Adds documentation page for a flight-declarations scenario. |
| docs/scenarios/airtraffic-simulations/openutm_sim_air_traffic_data.md | Adds documentation page for an air-traffic scenario. |
| docs/scenarios/airtraffic-simulations/opensky_live_data.md | Adds documentation page for an air-traffic scenario. |
| docs/scenarios/airtraffic-simulations/bluesky_sim_air_traffic_data.md | (Listed in PR) Air-traffic documentation file presence/update. |
| docs/index.md | Reorganizes docs index into sections and updates links to subfolder docs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR add the ability to add sub-folders to scenario definitions so that forks can have folders and clean merging would be possible.