Feature branch sync - pub/q2_dev to pub/q2_upgrade#4433
Merged
Conversation
Signed-off-by: Kratika Patidar <Kratika.Patidar@dell.com>
Signed-off-by: Kratika Patidar <Kratika.Patidar@dell.com>
Update apply_telemetry_on_upgrade.yml
* Update storage_config.j2 and transform_storage_config.yml * Update storage_config.j2 * Update storage_config.j2 * Ldms changes for upgrade * Admin NIC state validation * Update validation_utils.py * upgrade idrac telemetry replica preserve changes * localrepo upgrade migration Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> * Update ansible-lint.yml * Update pylint.yml * Removed display task from playbook Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> * pulp python content copy fix Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> * Merge pull request #4242 from SOWJANYAJAGADISH123/pub/q2_dev pxe boot utility * appended log directory details to migration msg Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> --------- Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> Co-authored-by: mithileshreddy04 <mithilesh.reddy@dell.com> Co-authored-by: Kratika_Patidar <Kratika.Patidar@dell.com> Co-authored-by: pullan1 <sudha.pullalaravu@dell.com> Co-authored-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> Co-authored-by: sakshi-singla-1735 <sakshi.s@dell.com>
* Update openchami git version (#4251) Co-authored-by: mithileshreddy04 <mithilesh.reddy@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com> * copying input files to gitlab * Update push_ci_files.yml * syntax error * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update push_ci_files.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * removal of update code * Update sync_input_file.yml --------- Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: mithileshreddy04 <mithilesh.reddy@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com>
Domain layer (core/image_group/): - ImageGroupId value object (String(128), non-UUID catalog identifier) - ImageGroupStatus enum (BUILT→DEPLOYING→DEPLOYED→RESTARTING→RESTARTED→ VALIDATING→PASSED/FAILED/CLEANED) with is_terminal() - PipelinePhase enum (BUILD, DEPLOY) - ImageGroup/Image domain entities with lifecycle state machine - Repository interfaces (ImageGroupRepository, ImageRepository) using ABC - Domain exceptions: DuplicateImageGroupError, ImageGroupNotFoundError, ImageGroupMismatchError, InvalidStateTransitionError - State machine guard functions with ALLOWED_TRANSITIONS and STATUS_FLOW ORM model changes (infra/db/models.py): - ImageGroupModel: String(128) PK, UNIQUE job_id FK (1:1 mapping), status with CHECK constraint, timestamps, cascade relationships - ImageModel: UUID PK, FK to image_groups, role+image_name, UNIQUE(image_group_id, role) constraint - JobModel: Added pipeline_phase column (nullable String(10)), singular image_group relationship (uselist=False) - StageModel: Added result_detail JSONB column Infrastructure: - ImageGroupMapper/ImageMapper in infra/db/mappers.py - SqlImageGroupRepository/SqlImageRepository in infra/db/repositories.py (save, find_by_id, find_by_job_id, find_by_job_id_for_update, update_status, list_by_status with pagination, exists, save_batch) - InMemoryImageGroupRepository/InMemoryImageRepository for dev/test - StageType enum extended with DEPLOY, PXE_BOOT, VALIDATE stages - Alembic migration 006: image_groups + images tables, pipeline_phase, result_detail columns - DI container updated (Dev + Prod) with new repository providers - API dependencies: get_image_group_repo, get_image_repo factories Tests (99 new tests, all passing): - Unit: value objects, entities, exceptions, state machine guards - Unit: in-memory repository operations and pagination - Integration: ORM round-trip with SQLite, relationships, constraints, cascade deletes, SQL repository operations, mapper conversions, StageModel.result_detail, JobModel.pipeline_phase Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Update storage_config.j2 and transform_storage_config.yml * Update storage_config.j2 * Update storage_config.j2 * Ldms changes for upgrade * Admin NIC state validation * Update validation_utils.py * upgrade idrac telemetry replica preserve changes * localrepo upgrade migration Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> * Update ansible-lint.yml * Update pylint.yml * Removed display task from playbook Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> * pulp python content copy fix Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> * Merge pull request #4242 from SOWJANYAJAGADISH123/pub/q2_dev pxe boot utility * appended log directory details to migration msg Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> * removing input template * Fix for pulp remote RemoteArtifacts is 0 after repo migration Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> * Feature/ome discovery pxe mapping enhancements (#4245) * feat(discovery): OME static group extraction, PXE mapping IP/SU/parent tag enhancements ome_server_inventory.py: - Fix static group extraction: find 'Static Groups' container by name and select only direct children via ParentId; avoids picking system/nested groups - Emit module.warn() for static groups that exist but have no devices assigned - Fix idrac_hostname: read InstrumentationName/DnsName from DeviceManagement ManagementType==2 entry instead of DeviceName which returns the IP address generate_pxe_mapping.py: - ADMIN_IP: derive from first 2 octets of admin_network.subnet + last 2 of BMC IP - IB_IP: derive from first 2 octets of ib_network.subnet + last 2 of BMC IP - Skip IB_IP/IB_MAC when server has no IB NIC (ib_nic_mac is empty) - Add extract_su_from_hostname() with regex (SU[A-Z]?\d+)(?=R\d+) to parse Scalable Unit from BMC hostname; rejects service-tag-only hostnames (idrac-JCGT033) and falls back to grp0 when no SU pattern is found - Set GROUP_NAME to extracted SU identifier (fallback: grp0) - Post-process rows to assign PARENT_SERVICE_TAG from the service_kube_control_plane_x86_64 node within the same SU group - Remove BMC_HOSTNAME from CSV headers and output rows - Lint: remove dead try/except in calculate_admin_ip/calculate_ib_ip, reuse ib_mac variable, suppress broad-except pylint warning generate_pxe_mapping.yml: - Load network_spec.yml via include_vars - Set admin_subnet and ib_subnet using selectattr on Networks list - Pass both subnets as parameters to the generate_pxe_mapping module defaults/main.yml: - Add admin_subnet and ib_subnet default variables (empty string) provision_validation.py: - Comment out validate_admin_ips_against_network_spec function and its call site; ADMIN_IPs are now derived from subnet octets + BMC IP and will not necessarily fall within primary_oim_admin_ip/netmask_bits range * refactor: rename discovery directory to provision, update network_spec.yml - Renamed discovery/ to provision/ (git detected as rename, no content loss) - Updated input/network_spec.yml with latest network configuration changes * Update discovery.yml * refactor: unify OME credentials into get_config_credentials flow - Added ome_ip, ome_username, ome_password to omnia_credential.j2 template - Added 'discovery' service entry to omnia_credentials in update_config/vars/main.yml - Added 'discovery' to the hardcoded service key trigger list in fetch_credentials.yml - Replaced custom vault logic in get_ome_credentials.yml with unified decrypt_include_encrypt.yml call against omnia_config_credentials.yml - Updated ome_discovery/vars/main.yml to reference omnia_config_credentials_file and omnia_config_credentials_vault_key instead of the separate .vault/ paths - Deleted .vault/ome_credentials.yml and .vault/.vault_password (no longer needed) * chore: update copyright year from 2025 to 2026 in modified files Updated copyright header in all ome_discovery files modified during this feature branch: - library/generate_pxe_mapping.py - library/ome_server_inventory.py - tasks/generate_pxe_mapping.yml - tasks/get_ome_credentials.yml - defaults/main.yml - vars/main.yml * fix: restore discovery_validations role missed during discovery-to-provision rename discovery/roles/discovery_validations/ was accidentally dropped when renaming the discovery/ directory to provision/. Add it back under provision/roles/discovery_validations/ to resolve the PR merge conflict. * chore: update copyright year to 2026 in provision/roles/discovery_validations files * fix: remove duplicate discovery_validations role (provision_validations already exists) provision/roles/provision_validations/ is the correct renamed equivalent of discovery/roles/discovery_validations/. The discovery_validations copy added to provision/ was redundant. * feat: apply upstream telemetry upgrade changes from dell/omnia pub/q2_dev - Replace kubectl command with kubernetes.core.k8s module for iDRAC StatefulSet - Preserve existing replica count during iDRAC StatefulSet upgrade - Add LDMS store daemon check, restart, and readiness wait tasks * fix: quote build_stream_job_id_absent message in provision_validations vars * feat: add discovery/roles/discovery_validations and telemetry files - Add discovery/roles/discovery_validations/vars/main.yml with task definitions for validation flow - Add discovery/roles/telemetry/tasks/apply_telemetry_on_upgrade.yml with upstream telemetry upgrade logic (replica preservation + LDMS store) * fix: wrap long line in fetch_credentials.yml to satisfy yaml[line-length] lint * refactor: move ome_ip from credentials to discovery_config.yml - Create input/discovery_config.yml for non-credential discovery settings (ome_ip, future Magellan config) - Remove ome_ip from omnia_credential.j2 and credential update vars - Load ome_ip via include_vars from discovery_config.yml in get_ome_credentials.yml - Add discovery_config.yml to provision_validations discovery_inputs - Remove redundant ib_subnet/admin_subnet defaults from ome_discovery * fix: add newline at end of ome_discovery/defaults/main.yml * fix: override role_path to absolute path for decrypt_include_encrypt.yml role_path resolves to ome_discovery role path, causing encrypt_files_vars.yml to be looked up incorrectly. Override to playbook_dir dirname (/opt/omnia/omnia). * fix: inline credential loading to avoid role_path resolution issue role_path cannot be overridden in include_tasks vars. Replace the call to decrypt_include_encrypt.yml with direct include_vars using stat checks for encrypted vs unencrypted credential file handling. * fix: skip load-failure rule in ansible-lint to avoid CI false positives ansible-lint fails to resolve role_path relative paths during static analysis in GitHub Actions, causing false load-failure errors for files that exist and work at runtime. * Update ansible.cfg * Update ansible.cfg * refactor: rename discovery references to provision and add discovery_config variable - Rename discover_mapping_nodes.yml to provision_mapping_nodes.yml - Replace "discovery" terminology with "provision" across playbooks, vars, READMEs, and task names in provision roles - Add subnet as required field with IP pattern validation in network_spec schema - Define discovery_config variable in ome_discovery vars and use it in get_ome_credentials.yml (consistent with provision_config pattern) - Rename discovery_inputs to provision_inputs in validation vars - Rename discovery_mech_mapping to provision_mech_mapping - Update user-facing messages to reference provision.yml Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: credential rules, vault handling, GROUP_NAME validation, and discovery playbook improvements - Add ome_username and ome_password validation rules to credential_rules.json - Add 'discovery' tag to prepare_oim omnia_run_tags so OME credentials are prompted - Fix vault-encrypted credential loading in get_ome_credentials.yml (use decrypt-include-reencrypt pattern instead of unsupported vault_password_file) - Add include_input_dir.yml import to discovery.yml so input_project_dir is set - Accept SU1-SU100 (case-insensitive) in addition to grp0-grp100 for GROUP_NAME - Fix Magellan message to use list format (avoids \n in debug output) - Remove escaped quotes from discovery usage examples Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: extend SU group name support to build_image validation and schemas - Add build_aarch_image tag to input_file_inventory so build_image_aarch64.yml runs provision_config validation (was missing, causing no validation to run for aarch64 builds) - Update GROUP_NAME patterns in functional_groups_config.json and omnia_config.json schemas to accept SU1-SU100 format alongside grp0-grp100 - Update INVALID_GROUP_NAME_MSG to reflect both accepted formats Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Signed-off-by: Sujit Jadhav <sujit_jadhav@dell.com> Co-authored-by: Super User <root@testbed.omnia.test> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Cleanup discovery roles: move library modules, remove unused roles (#4261) * Cleanup discovery roles: move library modules, remove unused roles - Move ome_server_inventory.py and generate_pxe_mapping.py from discovery/roles/ome_discovery/library/ to common/library/modules/ so they are shared via the common module search path already configured in discovery/ansible.cfg - Remove unused discovery/roles/telemetry/ directory - Remove unused discovery/roles/discovery_validations/ directory - Load discovery_config.yml at playbook level in discovery.yml (consistent with how build_stream_config.yml is loaded in provision.yml) - Fix discovery_complete_msg formatting for readable Ansible output Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Remove unused discovery_validations role Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Super User <root@testbed.omnia.test> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix for set_pxe_boot.yml when custom inventory given (#4260) * Update generate_bmc_inventory.yml Signed-off-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> * Update pre_checks.yml Signed-off-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> * lint issue Signed-off-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> --------- Signed-off-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> * Delete examples/input_template directory Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> * Delete discovery/roles/discovery_validations/vars/main.yml Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> * Delete discovery/roles/telemetry/tasks/apply_telemetry_on_upgrade.yml Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> * Update pre_checks.yml Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> * Update pulp_repo_name_migration.py Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> --------- Signed-off-by: pullan1 <sudha.pullalaravu@dell.com> Signed-off-by: Sujit Jadhav <sujit_jadhav@dell.com> Signed-off-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: mithileshreddy04 <mithilesh.reddy@dell.com> Co-authored-by: Kratika_Patidar <Kratika.Patidar@dell.com> Co-authored-by: pullan1 <sudha.pullalaravu@dell.com> Co-authored-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> Co-authored-by: sakshi-singla-1735 <sakshi.s@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Co-authored-by: snarthan <narthan.s@dell.com> Co-authored-by: Sujit Jadhav <sujit_jadhav@dell.com> Co-authored-by: Super User <root@testbed.omnia.test> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Abhishek S A <abhishek.sa3@dell.com>
…mage Data Model (#4265) * Fix all the integration tests * S1-3: Implement ImageGroup/Image data model for Release 2 Domain layer (core/image_group/): - ImageGroupId value object (String(128), non-UUID catalog identifier) - ImageGroupStatus enum (BUILT→DEPLOYING→DEPLOYED→RESTARTING→RESTARTED→ VALIDATING→PASSED/FAILED/CLEANED) with is_terminal() - PipelinePhase enum (BUILD, DEPLOY) - ImageGroup/Image domain entities with lifecycle state machine - Repository interfaces (ImageGroupRepository, ImageRepository) using ABC - Domain exceptions: DuplicateImageGroupError, ImageGroupNotFoundError, ImageGroupMismatchError, InvalidStateTransitionError - State machine guard functions with ALLOWED_TRANSITIONS and STATUS_FLOW ORM model changes (infra/db/models.py): - ImageGroupModel: String(128) PK, UNIQUE job_id FK (1:1 mapping), status with CHECK constraint, timestamps, cascade relationships - ImageModel: UUID PK, FK to image_groups, role+image_name, UNIQUE(image_group_id, role) constraint - JobModel: Added pipeline_phase column (nullable String(10)), singular image_group relationship (uselist=False) - StageModel: Added result_detail JSONB column Infrastructure: - ImageGroupMapper/ImageMapper in infra/db/mappers.py - SqlImageGroupRepository/SqlImageRepository in infra/db/repositories.py (save, find_by_id, find_by_job_id, find_by_job_id_for_update, update_status, list_by_status with pagination, exists, save_batch) - InMemoryImageGroupRepository/InMemoryImageRepository for dev/test - StageType enum extended with DEPLOY, PXE_BOOT, VALIDATE stages - Alembic migration 006: image_groups + images tables, pipeline_phase, result_detail columns - DI container updated (Dev + Prod) with new repository providers - API dependencies: get_image_group_repo, get_image_repo factories Tests (99 new tests, all passing): - Unit: value objects, entities, exceptions, state machine guards - Unit: in-memory repository operations and pagination - Integration: ORM round-trip with SQLite, relationships, constraints, cascade deletes, SQL repository operations, mapper conversions, StageModel.result_detail, JobModel.pipeline_phase Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * S1:4 Align Parse-Catalog and Image-Build API to new Images and ImageGroup data models --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Update openchami git version (#4251) Co-authored-by: mithileshreddy04 <mithilesh.reddy@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com> * copying input files to gitlab * Update push_ci_files.yml * syntax error * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update push_ci_files.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * Update sync_input_file.yml * removal of update code * Update sync_input_file.yml * Update pre_checks.yml * debug * Update pre_checks.yml * Update pre_checks.yml * removal of debug task * Restart api * Create test_restart_api.sh * Fix auth credentials in restart test script Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: mithileshreddy04 <mithilesh.reddy@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Delete examples/input_template directory
Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com>
* Delete discovery/roles/discovery_validations/vars/main.yml
Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com>
* Delete discovery/roles/telemetry/tasks/apply_telemetry_on_upgrade.yml
Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com>
* Update pre_checks.yml
Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com>
* Update pulp_repo_name_migration.py
Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com>
* Add upload API endpoint for configuration file uploads
- Add upload router to API with PUT /jobs/{job_id}/upload endpoint
- Create upload orchestrator package with commands, results, and exceptions
- Add UploadFilesCommand, UploadFilesResult, and file change tracking
- Add UPLOAD stage type to StageType enum
- Implement file upload validation, deduplication, and multi-location storage
- Add UploadFilesResponse schema with summary and file details
* Add upload files use case with validation, change detection, and multi-location storage. Adding tests.
- Add UploadFilesUseCase with filename whitelist validation and size limit checks
- Add change detection logic comparing SHA256 digests against previous uploads
- Add multi-location storage: artifact store, build stream input dir, and playbook queue
* Fix upload_files to use standard Omnia playbook input directory path
* Add stage state tracking to upload files use case with PENDING→IN_PROGRESS→COMPLETED transitions
- Inject stage_repository, audit_repository, and uuid_generator into UploadFilesUseCase
- Add _get_upload_stage, _mark_stage_started, and _mark_stage_completed methods
- Transition upload stage from PENDING to IN_PROGRESS on first file upload
- Transition upload stage to COMPLETED after all files processed
- Update DevContainer and ProdContainer to wire new dependencies
- Update upload API dependencies to provide stage
* Add audit event emissions for stage transitions
* Refactor upload dependencies to use container-provided artifact_store, uuid_generator, and config instead of manual initialization
* pylint fixes
* Pylint fixes.
* fix: Allow file uploads when stage is already COMPLETED and always emit audit events.
* Adding additional files into allow list.
* renaming ha to high_availability_config.yml
Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com>
---------
Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com>
* Minimal OS-only functional group enablement (#4267) * Minimal OS-only functional group enablement for x86_64 and aarch64 * Update image_package_collector.py * Update provision_validation.py * Minimal OS functional group updates in provision * Minimal OS functional group upgrade * Fix os_* package cross-contamination and remove stale discovery templates * csi defect fix * nvidia dcgm install * Add 'provision' tag to omnia_run_tags (#4276) Signed-off-by: balajikumaran.cs <balajikumaran.c.s@gmail.com> --------- Signed-off-by: balajikumaran.cs <balajikumaran.c.s@gmail.com> Co-authored-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Co-authored-by: mcas <sakshi.s@dell.com> Co-authored-by: Super User <root@oim.omnia.test> Co-authored-by: snarthan <narthan.s@dell.com> Co-authored-by: balajikumaran.cs <balajikumaran.c.s@gmail.com>
Signed-off-by: Rajeshkumar-s2 <rajeshkumar.s2@dell.com>
* Fix all the integration tests * S1-3: Implement ImageGroup/Image data model for Release 2 Domain layer (core/image_group/): - ImageGroupId value object (String(128), non-UUID catalog identifier) - ImageGroupStatus enum (BUILT→DEPLOYING→DEPLOYED→RESTARTING→RESTARTED→ VALIDATING→PASSED/FAILED/CLEANED) with is_terminal() - PipelinePhase enum (BUILD, DEPLOY) - ImageGroup/Image domain entities with lifecycle state machine - Repository interfaces (ImageGroupRepository, ImageRepository) using ABC - Domain exceptions: DuplicateImageGroupError, ImageGroupNotFoundError, ImageGroupMismatchError, InvalidStateTransitionError - State machine guard functions with ALLOWED_TRANSITIONS and STATUS_FLOW ORM model changes (infra/db/models.py): - ImageGroupModel: String(128) PK, UNIQUE job_id FK (1:1 mapping), status with CHECK constraint, timestamps, cascade relationships - ImageModel: UUID PK, FK to image_groups, role+image_name, UNIQUE(image_group_id, role) constraint - JobModel: Added pipeline_phase column (nullable String(10)), singular image_group relationship (uselist=False) - StageModel: Added result_detail JSONB column Infrastructure: - ImageGroupMapper/ImageMapper in infra/db/mappers.py - SqlImageGroupRepository/SqlImageRepository in infra/db/repositories.py (save, find_by_id, find_by_job_id, find_by_job_id_for_update, update_status, list_by_status with pagination, exists, save_batch) - InMemoryImageGroupRepository/InMemoryImageRepository for dev/test - StageType enum extended with DEPLOY, PXE_BOOT, VALIDATE stages - Alembic migration 006: image_groups + images tables, pipeline_phase, result_detail columns - DI container updated (Dev + Prod) with new repository providers - API dependencies: get_image_group_repo, get_image_repo factories Tests (99 new tests, all passing): - Unit: value objects, entities, exceptions, state machine guards - Unit: in-memory repository operations and pagination - Integration: ORM round-trip with SQLite, relationships, constraints, cascade deletes, SQL repository operations, mapper conversions, StageModel.result_detail, JobModel.pipeline_phase Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * S1:4 Align Parse-Catalog and Image-Build API to new Images and ImageGroup data models * S1-5:S1-6: Implementation of Images and Deploy API * Avoid sharing any code between deploy and validate-image-on-test * Fix the issues from testing * Address review comments --------- Signed-off-by: Rajeshkumar-s2 <rajeshkumar.s2@dell.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…h_ucx_openmpi_json directory
Feature branch sync - pub/q2_dev to pub/build_stream
Signed-off-by: Abhishek S A <abhishek.sa3@dell.com>
Feature branch sync - pub/q2_dev to pub/build_stream
Implement Automation Framework Setup and Cleanup for Validate Stage
Signed-off-by: balajikumaran.cs <balajikumaran.c.s@gmail.com>
Signed-off-by: balajikumaran.cs <balajikumaran.c.s@gmail.com>
Fix undefined variable build_stream_root_dir in configure_automation.yml
…ult polling with database update. (#4391) * Update validate stage creation to prevent duplicates by checking for existing stage * Add exception handling for StageAlreadyCompletedError in local repository creation endpoint * Fix: State transition check fix * Add molecule test automation execution support to playbook watcher service - Add test_automation command type with molecule-specific field validation - Implement execute_molecule() function to run molecule verify commands - Validate scenario_names list and artifact/config paths for security - Parse test_report.json for test summary statistics - Route requests to execute_molecule() or execute_playbook() based on command_type - Add build_stream_root_dir variable for artifacts and shared data * Add result_detail JSONB field to Stage entity and populate test summary for validate stage - Add result_detail field to Stage entity and database mapper - Extend PlaybookResult with correlation_id, test_summary, and artifact_dir fields - Populate result_detail before complete()/fail() to avoid version conflicts - Transition ImageGroup to PASSED/FAILED on validate success/failure - Clear error fields and result_detail when resetting validate stage
…oad (#4384) * s3cmd configurations * update image tags * fix: Resolve FK constraint violation and catalog metadata persistence issues - Fix FK constraint by using shared session for image_group_repo and image_repo in ResultPoller - Concatenate multiple S3 paths with semicolon delimiter to respect unique constraint uq_images_image_group_id_role - Update cleanup_job.py to split semicolon-delimited paths and delete each individually - Widen image_name column from VARCHAR(256) to VARCHAR(512) to accommodate semicolon-delimited paths - Add logging to FileArtifactStore.store() to debug file write failures - Fix URI construction in parse_catalog.py for existing artifacts (construct file:// URI directly) - Update build_stream container image tag to 1.1 Tested with job 7da54d1f-ed26-41dd-b3ff-0386104e644d (image-build19) - ImageGroup and Images created successfully * Fix lint error * Modify the charlimit to 512 in DB * Fix the image_groups name and stages in summary * Updating Catalog changes aligning to recent updates * Fix the pylint issues * Catalog Updates to support powerscale drivers * Support Resume & Retry * Fix issues found during testing
* Update automation framework repository branch to v2.2.0.0 * removing duplicate variable
Add validate stage in pipeline
Signed-off-by: balajikumaran.cs <balajikumaran.c.s@gmail.com>
Signed-off-by: Rajeshkumar-s2 <rajeshkumar.s2@dell.com>
Merge pub/q2_dev to pub/build_stream
Signed-off-by: Rajeshkumar S <rajeshkumar.s2@dell.com>
Signed-off-by: Rajeshkumar S <rajeshkumar.s2@dell.com>
Signed-off-by: Rajeshkumar S <rajeshkumar.s2@dell.com>
Merege build_Stet status
Merge pub/build_stream to pub/q2_dev
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feature branch sync - pub/q2_dev to pub/q2_upgrade