feat: adding testing workflow to orchestrator tests by y-first · Pull Request #3738 · redhat-developer/rhdh

y-first · 2025-11-24T21:27:39Z

Description

Summary

This PR adds comprehensive E2E testing for the orchestrator plugin using the failswitch workflow and updates the deployment utilities to support it.

Key Changes

New E2E test suite (failswitch-workflow.spec.ts): Added comprehensive Playwright tests for the orchestrator plugin covering:
- Basic workflow execution with different input states (OK, KO, Wait)
- Workflow abort functionality
- Workflow tab validation and run details
- Re-running workflows from failure points (currently skipped)
Enhanced Orchestrator page object (orchestrator.ts): Added new methods to support failswitch workflow testing:
- selectFailSwitchWorkflowItem() - Navigate to failswitch workflow
- runFailSwitchWorkflow() - Execute workflow with configurable switch input
- reRunFailSwitchWorkflow() - Re-run workflow with new parameters
- reRunOnFailure() - Re-run from failure point or entire workflow
- validateWorkflowStatus() - Verify workflow execution status
Deployment script updates (.ibm/pipelines/utils.sh):
- Migrated from experimental user-onboarding workflow to failswitch workflow
- Updated workflow repository from rhdh-orchestrator-test to rhdhorchestrator
- Changed manifest paths to point to failswitch workflow resources
- Added explicit namespace parameter to oc apply command
- Removed user-onboarding specific secret configuration logic
- Updated deployment wait conditions for failswitch workflow
- Adds some updated configurations for deploying with OSL 1.37

Test Coverage

The new test suite validates:

✅ Successful workflow execution (OK state)
✅ Failed workflow execution (KO state)
✅ Running workflow state (Wait state)
✅ Workflow abort functionality
✅ Workflow re-run capabilities

Which issue(s) does this PR fix

Fixes #?

PR acceptance criteria

Please make sure that the following steps are complete:

GitHub Actions are completed and successful
Unit Tests are updated and passing
E2E Tests are updated and passing
Documentation is updated if necessary (requirement for new features)
Add a screenshot if the change is UX/UI related

How to test changes / Special notes to the reviewer

github-actions · 2025-11-24T22:21:49Z

The image is available at:

y-first · 2025-11-25T15:19:12Z

/ok-to-test

y-first · 2025-11-25T16:39:19Z

/test e2e-ocp-helm

rhdh-qodo-merge · 2025-11-25T16:39:22Z

You are above your monthly Qodo Merge usage quota. For more information, please visit here.

y-first · 2025-11-25T18:04:27Z

/test e2e-ocp-helm

rhdh-qodo-merge · 2025-11-25T18:04:31Z

You are above your monthly Qodo Merge usage quota. For more information, please visit here.

y-first · 2025-11-25T18:37:16Z

/test e2e-ocp-helm

rhdh-qodo-merge · 2025-11-25T18:37:19Z

You are above your monthly Qodo Merge usage quota. For more information, please visit here.

y-first · 2025-11-25T21:44:12Z

/test e2e-ocp-helm

rhdh-qodo-merge · 2025-11-25T21:44:16Z

You are above your monthly Qodo Merge usage quota. For more information, please visit here.

github-actions · 2025-11-25T22:40:30Z

The image is available at:

y-first · 2025-11-26T15:49:35Z

/test e2e-ocp-helm

The condition `isPrOcpHelmJob` was incorrectly matching the nightly job `pull-ci-...-e2e-ocp-helm-nightly`, causing orchestrator tests to be skipped. Added check for `e2e-ocp-helm-nightly` specifically to ensure orchestrator tests run only in that nightly job. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Orchestrator infrastructure is not installed on OSD-GCP, so tests should be skipped there as well. Added isOsdGcpJob check alongside the existing isPrOcpHelmJob condition. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Move orchestrator-related functions to a dedicated module following the new modular architecture introduced in PR redhat-developer#3817. This includes: - orchestrator::should_skip - determines if orchestrator should be skipped - orchestrator::disable_plugins_in_values - disables orchestrator plugins - orchestrator::install_infra_chart - deploys orchestrator-infra Helm chart - orchestrator::deploy_workflows - deploys workflows for Helm testing - orchestrator::deploy_workflows_operator - deploys workflows for Operator testing - orchestrator::enable_plugins_operator - enables plugins for operator deployment Existing function names are preserved as delegation functions for backwards compatibility. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Extract common code into private helper functions: - _orchestrator::clone_workflows - clone the workflow repo - _orchestrator::apply_manifests - apply workflow manifests - _orchestrator::wait_for_sonataflow_resources - wait for sf resources - _orchestrator::patch_workflow_postgres - patch workflow with pg config - _orchestrator::wait_for_workflow_deployments - wait for deployments Also: - Add constants for workflow repo URL and workflow names - Standardize logging across functions - Improve code readability and maintainability Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

github-actions · 2026-01-29T14:36:54Z

The image is available at:

ShellCheck fixes: - Remove unused $@ from functions that take no arguments (SC2120) SonarQube fixes: - Add explicit return statements at end of all functions - Use [[ instead of [ for conditional tests in orchestrator.sh - Refactor orchestrator::should_skip for clarity Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

github-actions · 2026-01-29T14:40:24Z

🚫 Image Push Skipped.

The container image push was skipped because the build was skipped (either due to [skip-build] tag or no relevant changes with existing image)

y-first · 2026-01-29T14:47:46Z

/test e2e-ocp-helm-nightly

qodo-code-review · 2026-01-29T14:47:50Z

ⓘ Your monthly quota for Qodo has expired. Upgrade your plan

ⓘ Paying users. Check that your Qodo account is linked with this Git user account

rhdh-qodo-merge · 2026-01-29T14:47:50Z

ⓘ Your monthly quota for Qodo has expired. Upgrade your plan

ⓘ Paying users. Check that your Qodo account is linked with this Git user account

github-actions · 2026-01-29T18:01:51Z

🚫 Image Push Skipped.

The container image push was skipped because the build was skipped (either due to [skip-build] tag or no relevant changes with existing image)

Add conditional check before declaring readonly constants to prevent errors when the script is sourced multiple times. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

github-actions · 2026-01-29T18:03:33Z

🚫 Image Push Skipped.

The container image push was skipped because the build was skipped (either due to [skip-build] tag or no relevant changes with existing image)

gustavolira · 2026-01-29T18:18:15Z

/test e2e-ocp-helm-nightly

Add _orchestrator::wait_for_sonataflow_reconciliation() to wait for the SonataFlow operator to reconcile after CR patch before checking rollout status. This fixes a race condition where oc rollout status was called before the operator had updated the Deployment. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

github-actions · 2026-01-29T19:38:13Z

🚫 Image Push Skipped.

The container image push was skipped because the build was skipped (either due to [skip-build] tag or no relevant changes with existing image)

gustavolira · 2026-01-29T19:38:19Z

/test e2e-ocp-helm-nightly

sonarqubecloud · 2026-01-29T19:39:21Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

gustavolira · 2026-01-29T21:19:22Z

/lgtm

openshift-ci · 2026-01-29T21:19:30Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gustavolira

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~.ibm/OWNERS~~ [gustavolira]
~~e2e-tests/OWNERS~~ [gustavolira]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

y-first · 2026-02-12T15:34:44Z

/cherry-pick release-1.9

openshift-cherrypick-robot · 2026-02-12T15:34:55Z

@y-first: only redhat-developer org members may request cherry picks. If you are already part of the org, make sure to change your membership to public. Otherwise you can still do the cherry-pick manually.

Details

In response to this:

/cherry-pick release-1.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

y-first · 2026-02-12T15:36:53Z

/cherry-pick release-1.9

openshift-cherrypick-robot · 2026-02-12T15:37:45Z

@y-first: #3738 failed to apply on top of branch "release-1.9":

Applying: feat: adding testing workflow to orchestrator tests
.git/rebase-apply/patch:273: trailing whitespace.
    await this.page.getByRole("button", { name: "Abort" }).click();    
warning: 1 line adds whitespace errors.
Using index info to reconstruct a base tree...
M	.ibm/pipelines/utils.sh
Falling back to patching base and 3-way merge...
Auto-merging .ibm/pipelines/utils.sh
CONFLICT (content): Merge conflict in .ibm/pipelines/utils.sh
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 feat: adding testing workflow to orchestrator tests

Details

In response to this:

/cherry-pick release-1.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci Bot requested review from albarbaro and subhashkhileri November 24, 2025 21:27

y-first temporarily deployed to external November 24, 2025 21:27 — with GitHub Actions Inactive

y-first marked this pull request as draft November 24, 2025 21:28

openshift-ci Bot added the do-not-merge/work-in-progress label Nov 24, 2025

openshift-ci Bot added the ok-to-test label Nov 25, 2025

y-first force-pushed the add-testing-workflow branch from bdec3d5 to 9139757 Compare November 25, 2025 16:30

y-first had a problem deploying to external November 25, 2025 16:30 — with GitHub Actions Error

y-first force-pushed the add-testing-workflow branch from 9139757 to 621733b Compare November 25, 2025 18:04

y-first had a problem deploying to external November 25, 2025 18:04 — with GitHub Actions Error

y-first force-pushed the add-testing-workflow branch from 621733b to 53ad152 Compare November 25, 2025 18:36

y-first had a problem deploying to external November 25, 2025 18:37 — with GitHub Actions Error

y-first force-pushed the add-testing-workflow branch from 53ad152 to 3498ef1 Compare November 25, 2025 21:42

y-first had a problem deploying to external November 25, 2025 21:42 — with GitHub Actions Error

y-first force-pushed the add-testing-workflow branch from 3498ef1 to d397fd4 Compare November 25, 2025 21:43

y-first had a problem deploying to external November 25, 2025 21:43 — with GitHub Actions Error

y-first force-pushed the add-testing-workflow branch from d397fd4 to c603489 Compare November 26, 2025 15:49

y-first had a problem deploying to external November 26, 2025 15:49 — with GitHub Actions Error

gustavolira and others added 6 commits January 29, 2026 10:50

fix: skip orchestrator tests for osd-gcp jobs

e7f6e08

Orchestrator infrastructure is not installed on OSD-GCP, so tests should be skipped there as well. Added isOsdGcpJob check alongside the existing isPrOcpHelmJob condition. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

style: fix prettier formatting in utils.sh

4eec41f

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

style: fix prettier formatting in orchestrator.ts

2f02a90

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

y-first commented Jan 29, 2026

View reviewed changes

Comment thread .ibm/pipelines/lib/orchestrator.sh

fix: guard readonly constants to allow multiple sourcing

aa0bce8

Add conditional check before declaring readonly constants to prevent errors when the script is sourced multiple times. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

gustavolira approved these changes Jan 29, 2026

View reviewed changes

gustavolira mentioned this pull request Feb 12, 2026

feat: backport orchestrator failswitch workflow tests to release-1.9 #4246

Merged

5 tasks

rhdh-qodo-merge Bot mentioned this pull request Apr 22, 2026

feat(ci): parallelize base and RBAC helm deployments #4679

Open

5 tasks

Conversation

y-first commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Which issue(s) does this PR fix

PR acceptance criteria

How to test changes / Special notes to the reviewer

Uh oh!

github-actions Bot commented Nov 24, 2025

Uh oh!

y-first commented Nov 25, 2025

Uh oh!

y-first commented Nov 25, 2025

Uh oh!

rhdh-qodo-merge Bot commented Nov 25, 2025

Uh oh!

y-first commented Nov 25, 2025

Uh oh!

rhdh-qodo-merge Bot commented Nov 25, 2025

Uh oh!

y-first commented Nov 25, 2025

Uh oh!

rhdh-qodo-merge Bot commented Nov 25, 2025

Uh oh!

y-first commented Nov 25, 2025

Uh oh!

rhdh-qodo-merge Bot commented Nov 25, 2025

Uh oh!

github-actions Bot commented Nov 25, 2025

Uh oh!

y-first commented Nov 26, 2025

Uh oh!

github-actions Bot commented Jan 29, 2026

Uh oh!

github-actions Bot commented Jan 29, 2026

Uh oh!

y-first commented Jan 29, 2026

Uh oh!

qodo-code-review Bot commented Jan 29, 2026

Uh oh!

rhdh-qodo-merge Bot commented Jan 29, 2026

Uh oh!

Uh oh!

github-actions Bot commented Jan 29, 2026

Uh oh!

github-actions Bot commented Jan 29, 2026

Uh oh!

gustavolira commented Jan 29, 2026

Uh oh!

github-actions Bot commented Jan 29, 2026

Uh oh!

gustavolira commented Jan 29, 2026

Uh oh!

sonarqubecloud Bot commented Jan 29, 2026

Quality Gate passed

Uh oh!

gustavolira commented Jan 29, 2026

Uh oh!

openshift-ci Bot commented Jan 29, 2026

Uh oh!

y-first commented Feb 12, 2026

Uh oh!

openshift-cherrypick-robot commented Feb 12, 2026

Uh oh!

y-first commented Feb 12, 2026

Uh oh!

openshift-cherrypick-robot commented Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

y-first commented Nov 24, 2025 •

edited

Loading