[#109] Discovery: Add YAML Manifest as a static source of entity discovery #118

bburda · 2026-01-17T13:08:56Z

Pull Request

Summary

Adds YAML manifest-based discovery as a static source for entity discovery, implementing three discovery modes: runtime_only (existing behavior), manifest_only (static YAML definitions), and hybrid (combines manifest definitions with runtime linking).

Changes:

Introduces manifest parser, validator, and manager for YAML-based entity definitions
Adds App and Function discovery models with runtime linking support
Refactors discovery architecture with strategy pattern (RuntimeDiscoveryStrategy, HybridDiscoveryStrategy)
Adds HTTP handlers for /apps and /functions endpoints
Updates API response format for list endpoints to include {items: [], total_count: N}

Issue

Link the related issue (required):

closes Discovery: Add Manifest YAML as production source of truth #109

Type

Bug fix
New feature or tests
Breaking change
Documentation only

Testing

Unit and Integration Testing

Checklist

Breaking changes are clearly described (and announced in docs / changelog if needed)
Tests were added or updated if needed
Docs were updated if behavior or public API changed

Refactored discovery-related code into dedicated discovery/ subfolder. Changes: - Created discovery/ subfolder with modular structure - Implemented DiscoveryStrategy interface (Strategy Pattern) - Extracted RuntimeDiscoveryStrategy from DiscoveryManager - Split models.hpp into discovery/models/{area,component,app,function,common}.hpp - Added new fields to Area and Component models (name, description, tags, etc.) - Created stub App and Function models for future manifest support - Moved area_handlers and component_handlers to http/handlers/discovery/ - Created stub app_handlers and function_handlers - Updated CMakeLists.txt with new file paths - Maintained backward compatibility via include redirects The DiscoveryManager now acts as an orchestrator using the strategy pattern, delegating to RuntimeDiscoveryStrategy for ROS 2 graph-based discovery. Future strategies (ManifestDiscoveryStrategy, HybridDiscoveryStrategy) can be added without changing the interface.

- Add full App struct with RosBinding, runtime state, and resources - Add full Function struct with hosts and depends_on relationships - Implement to_json(), to_entity_reference(), to_capabilities() for all models - Add to_capabilities() method to Area model - Create app.cpp and function.cpp with serialization implementations - Add unit tests for discovery model serialization

- Add ManifestParser for loading YAML manifests into internal structures - Add ManifestValidator with validation rules R001-R011: - R001: Version must be '1.0' - R002-R005: Unique IDs for all entity types - R006: Valid area/parent references - R007: App must reference existing component - R008: depends_on references validation (warning) - R009: hosts references validation (warning) - R010: No duplicate ROS bindings - R011: Circular dependency detection - Add Manifest, ManifestConfig, ManifestMetadata structs - Add ValidationError and ValidationResult for error reporting - Add unit tests covering all validation rules

Add ManifestManager class for thread-safe manifest loading, validation, and entity access: - load_manifest(path, strict) / load_manifest_from_string(yaml, strict) - reload_manifest() / unload_manifest() for lifecycle management - Entity access: get_areas(), get_components(), get_apps(), get_functions() - O(1) lookup by ID via index maps - Relationship queries: get_components_for_area(), get_apps_for_component(), get_hosts_for_function(), get_subareas(), get_subcomponents() - get_status_json() for REST API integration - Thread-safe with std::mutex protection

Implement RuntimeLinker class that binds manifest-declared Apps to actual ROS 2 nodes discovered at runtime: - Exact match: node_name + namespace both match - Wildcard namespace: namespace='*' matches any namespace - Topic namespace: topic_namespace prefix matches node's topics - Enriches linked apps with runtime data (topics, services, actions) - Detects orphan nodes (runtime nodes not in manifest) - Applies orphan policy: ignore/warn/error/include_as_orphan - Provides lookup methods: is_app_online(), get_bound_node(), get_app_for_node()

Implement discovery mode configuration for DiscoveryManager: - DiscoveryMode enum: RUNTIME_ONLY, MANIFEST_ONLY, HYBRID - DiscoveryConfig struct for initialization parameters - initialize() method for deferred configuration - HybridDiscoveryStrategy combining manifest + runtime linking DiscoveryManager now supports: - Entity lookup by ID: get_area(), get_component(), get_app(), get_function() - Relationship queries: get_subareas(), get_subcomponents(), get_components_for_area(), get_apps_for_component(), get_hosts_for_function() - Manifest management: get_manifest_manager(), reload_manifest() - Mode switching with fallback to runtime on manifest load failure HybridDiscoveryStrategy features: - Uses manifest as source of truth for entity IDs - Links Apps to runtime nodes via RuntimeLinker - Enriches manifest components with runtime topics/services/actions - Handles orphan nodes per ManifestConfig policy Backward compatible: default RUNTIME_ONLY mode behaves identically to before.

- Add AppHandlers with 6 endpoint handlers: - GET /apps - list all apps - GET /apps/{id} - get app capabilities - GET /apps/{id}/data - list app topics - GET /apps/{id}/data/{data-id} - get specific data item - GET /apps/{id}/operations - list services and actions - GET /apps/{id}/configurations - list app parameters - Register app routes in RestServer - Include HATEOAS links and capabilities in responses

- Add FunctionHandlers with 5 endpoint handlers: - GET /functions - list all functions - GET /functions/{id} - get function capabilities - GET /functions/{id}/hosts - get apps that host this function - GET /functions/{id}/data - aggregated data from host apps - GET /functions/{id}/operations - aggregated operations from hosts - Register function routes in RestServer - Include HATEOAS links and capabilities in responses

- Add area relationship handlers: - GET /areas/{id}/subareas - list nested areas - GET /areas/{id}/related-components - list components in area - Add component relationship handlers: - GET /components/{id}/subcomponents - list nested components - GET /components/{id}/related-apps - list apps on component - Register 4 new routes in RestServer - Include HATEOAS links in all responses

- Add CapabilityBuilder utility class with Capability enum - Add LinksBuilder fluent API for HATEOAS links - Implement GET /areas/{area_id} endpoint with capabilities - Implement GET /components/{component_id} endpoint with capabilities - Refactor app_handlers and function_handlers to use builders - Add unit tests for capability_builder utilities - All handlers now return SOVD-compliant capabilities arrays

Documentation: - Add user guide: docs/tutorials/manifest-discovery.rst - Add schema reference: docs/config/manifest-schema.rst - Add migration guide: docs/tutorials/migration-to-manifest.rst - Create new docs/config section for configuration references - Update docs/index.rst with Configuration section Example manifests: - turtlebot3_manifest.yaml - Nav2 navigation stack example - minimal_manifest.yaml - Simplest possible template - multi_robot_manifest.yaml - Fleet management with namespaces - demo_nodes_manifest.yaml - Integration testing manifest Also fixes copyright headers to 2026 bburda in new source files

- Fix :doc: references in manifest-schema.rst to use /tutorials/ prefix - Update broken external links in manifest-discovery.rst - Add missing field initializers for ServiceInfo and ActionInfo in tests

- Update schema reference docs to use manifest_version - Update user guide and migration tutorial - Update all example manifest YAML files - Fix doc references and broken links - Fix compiler warnings in test_runtime_linker.cpp The C++ code already used manifest_version internally. This change removes direct SOVD naming from manifest schema.

API Changes: - Changed list endpoints to return SOVD-compliant format: {\"items\": [...], \"total_count\": N} instead of raw arrays - Updated /areas, /areas/{id}/components, /components endpoints Manifest Parser Fixes: - Added recursive subarea parsing with parent_area_id tracking - Fixed Function parsing: use 'hosted_by' field (was 'hosts') - Added Component 'type' field parsing from manifest YAML Manifest Manager Fixes: - get_components_for_area() now includes components from all subareas - Hierarchical lookup collects all descendant area IDs Entity ID Validation: - Allow hyphens in entity IDs (e.g., 'engine-ecu' from manifests) - Validation regex: alphanumeric, underscore, and hyphen Testing: - Added test_discovery_hybrid.test.py for hybrid mode tests - Added test_discovery_manifest.test.py for manifest-only mode tests - Updated all existing tests for new API response format - Made hybrid runtime tests defensive for flaky scenarios Documentation: - Fixed broken external links in tutorials - Removed non-working GitHub examples links

Copilot

Pull request overview

This pull request adds YAML manifest-based discovery as a static source for entity discovery, implementing three discovery modes: runtime_only (existing behavior), manifest_only (static YAML definitions), and hybrid (combines manifest definitions with runtime linking).

Changes:

Introduces manifest parser, validator, and manager for YAML-based entity definitions
Adds App and Function discovery models with runtime linking support
Refactors discovery architecture with strategy pattern (RuntimeDiscoveryStrategy, HybridDiscoveryStrategy)
Adds HTTP handlers for /apps and /functions endpoints
Updates API response format for list endpoints to include {items: [], total_count: N}

Reviewed changes

Copilot reviewed 68 out of 68 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
test_*.cpp (new test files)	Comprehensive unit tests for manifest parser, validator, linker, models, and capability builder
test_discovery_*.test.py (new)	Integration tests for manifest_only and hybrid discovery modes
test_integration.test.py	Updates tests for new API response format with items/total_count wrapper
discovery/manifest/*.cpp	Implementation of manifest parsing, validation, and runtime linking
discovery/discovery_manager.cpp	Refactored discovery manager with strategy pattern
http/handlers/discovery/*.cpp	New handlers for apps, functions, areas, and components with HATEOAS links
handler_context.cpp	Updates entity ID validation to allow hyphens (for manifest entity IDs)
models.hpp	Refactored to use modular model structure from discovery/models/
gateway_node.cpp	Adds discovery mode configuration parameters

src/ros2_medkit_gateway/test/test_discovery_hybrid.test.py

src/ros2_medkit_gateway/test/test_discovery_manifest.test.py

Documentation: - Fix title underline length in manifest-schema.rst Tests: - Rename duplicate test functions in test_discovery_hybrid.test.py to avoid verification.rst ID conflicts - Remove 'component-name' from invalid ID test since hyphens are now allowed in entity IDs (for manifest IDs like 'engine-ecu') Build: - Fix format string warning in fault_manager_node.cpp (%d -> %ld)

src/ros2_medkit_gateway/include/ros2_medkit_gateway/discovery/manifest/manifest_manager.hpp

src/ros2_medkit_gateway/config/examples/minimal_manifest.yaml

src/ros2_medkit_gateway/src/discovery/manifest/manifest_parser.cpp

src/ros2_medkit_gateway/test/test_discovery_hybrid.test.py

App Handlers: - Implement topic sampling for GET /apps/{id}/data/{data_id} - Implement parameter fetching for GET /apps/{id}/configurations when app has bound runtime node - Related apps endpoint already implemented in component_handlers Manifest Parser: - Fix app parsing: use 'is_located_on' field (was 'component') to match SOVD spec and manifest YAML format Tests: - Add test_app_data_item_endpoint for single data item sampling - Add test_component_related_apps for /components/{id}/related-apps - Add test_component_related_apps_empty and _not_found - Update YAML in unit tests: component -> is_located_on

- Add explanatory comments to empty except clauses - Implement strict validation: ERRORs always fail, strict=false only skips WARNINGs - Fix parser to use 'is_located_on' field name for SOVD OpenAPI compliance - Fix parent_component_id field parsing

Test now properly checks _links.is-located-on URL instead of non-existent component_id field in single app response.

bburda self-assigned this Jan 17, 2026

bburda added documentation Improvements or additions to documentation enhancement New feature or request discovery Discovery endpoints or strategies labels Jan 17, 2026

bburda added this to the MS1: Full Discovery & Data Access milestone Jan 17, 2026

bburda changed the title ~~Discovery: Add YAML Manifest as a static source of entity discovery~~ [#109] Discovery: Add YAML Manifest as a static source of entity discovery Jan 17, 2026

bburda force-pushed the feat/discovery/manifest branch from 41957c9 to 81da526 Compare January 18, 2026 08:32

bburda added 14 commits January 18, 2026 08:32

fix: resolve doc references and compiler warnings

67d86d8

- Fix :doc: references in manifest-schema.rst to use /tutorials/ prefix - Update broken external links in manifest-discovery.rst - Add missing field initializers for ServiceInfo and ActionInfo in tests

bburda force-pushed the feat/discovery/manifest branch from 81da526 to 6d66025 Compare January 18, 2026 08:34

bburda marked this pull request as ready for review January 18, 2026 08:34

bburda requested review from Copilot and mfaferek93 January 18, 2026 08:34

Copilot started reviewing on behalf of bburda January 18, 2026 08:34 View session

Copilot AI reviewed Jan 18, 2026

View reviewed changes

mfaferek93 reviewed Jan 18, 2026

View reviewed changes

bburda added 2 commits January 18, 2026 20:49

fix: address PR review comments

1b73fb1

- Add explanatory comments to empty except clauses - Implement strict validation: ERRORs always fail, strict=false only skips WARNINGs - Fix parser to use 'is_located_on' field name for SOVD OpenAPI compliance - Fix parent_component_id field parsing

fix: correct test_app_component_relationship assertion

1e80c06

Test now properly checks _links.is-located-on URL instead of non-existent component_id field in single app response.

mfaferek93 self-requested a review January 19, 2026 07:51

mfaferek93 approved these changes Jan 19, 2026

View reviewed changes

mfaferek93 merged commit bae5ffd into main Jan 19, 2026
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#109] Discovery: Add YAML Manifest as a static source of entity discovery #118

[#109] Discovery: Add YAML Manifest as a static source of entity discovery #118

Uh oh!

bburda commented Jan 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[#109] Discovery: Add YAML Manifest as a static source of entity discovery #118

[#109] Discovery: Add YAML Manifest as a static source of entity discovery #118

Uh oh!

Conversation

bburda commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request

Summary

Issue

Type

Testing

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bburda commented Jan 17, 2026 •

edited

Loading