Skip to content

Increase racreset retries to handle iDRAC instability after clear-jobs#778

Merged
openshift-merge-bot[bot] merged 1 commit intoredhat-performance:mainfrom
mcornea:wait_jobs
Mar 3, 2026
Merged

Increase racreset retries to handle iDRAC instability after clear-jobs#778
openshift-merge-bot[bot] merged 1 commit intoredhat-performance:mainfrom
mcornea:wait_jobs

Conversation

@mcornea
Copy link
Copy Markdown
Collaborator

@mcornea mcornea commented Mar 3, 2026

After --clear-jobs --force, the iDRAC can take up to ~2 minutes before its Redfish Systems endpoint responds again, causing --racreset to fail with "ComputerSystem's Members array is either empty or missing". Add retries (15) and delay (10s) to allow the iDRAC time to stabilize.

- WARNING  - Passing secrets via command line arguments can be unsafe. Consider using environment variables (BADFISH_USERNAME, BADFISH_PASSWORD).
- INFO     - Job queue for iDRAC mgmt-f19-h02-000-r640.rdu2.scalelab.redhat.com successfully cleared.
- WARNING  - Passing secrets via command line arguments can be unsafe. Consider using environment variables (BADFISH_USERNAME, BADFISH_PASSWORD).
- ERROR    - ComputerSystem's Members array is either empty or missing
Tue Mar  3 10:45:33 AM UTC 2026
Waiting for iDRAC to stabilize...
- WARNING  - Passing secrets via command line arguments can be unsafe. Consider using environment variables (BADFISH_USERNAME, BADFISH_PASSWORD).
- ERROR    - ComputerSystem's Members array is either empty or missing
Tue Mar  3 10:45:47 AM UTC 2026
Waiting for iDRAC to stabilize...
- WARNING  - Passing secrets via command line arguments can be unsafe. Consider using environment variables (BADFISH_USERNAME, BADFISH_PASSWORD).
- ERROR    - ComputerSystem's Members array is either empty or missing
Tue Mar  3 10:46:00 AM UTC 2026
Waiting for iDRAC to stabilize...
- WARNING  - Passing secrets via command line arguments can be unsafe. Consider using environment variables (BADFISH_USERNAME, BADFISH_PASSWORD).
- ERROR    - ComputerSystem's Members array is either empty or missing
Tue Mar  3 10:46:14 AM UTC 2026
Waiting for iDRAC to stabilize...
- WARNING  - Passing secrets via command line arguments can be unsafe. Consider using environment variables (BADFISH_USERNAME, BADFISH_PASSWORD).
- ERROR    - ComputerSystem's Members array is either empty or missing
Tue Mar  3 10:46:27 AM UTC 2026
Waiting for iDRAC to stabilize...
- WARNING  - Passing secrets via command line arguments can be unsafe. Consider using environment variables (BADFISH_USERNAME, BADFISH_PASSWORD).
- ERROR    - ComputerSystem's Members array is either empty or missing
Tue Mar  3 10:46:42 AM UTC 2026
Waiting for iDRAC to stabilize...
- WARNING  - Passing secrets via command line arguments can be unsafe. Consider using environment variables (BADFISH_USERNAME, BADFISH_PASSWORD).
- INFO     - Status code 204 returned for POST command to reset iDRAC.
- INFO     - iDRAC will now reset and be back online within a few minutes.

After --clear-jobs --force, the iDRAC can take up to ~2 minutes before
its Redfish Systems endpoint responds again, causing --racreset to fail
with "ComputerSystem's Members array is either empty or missing". Add
retries (15) and delay (10s) to allow the iDRAC time to stabilize.
@openshift-ci openshift-ci Bot requested review from akrzos and jtaleric March 3, 2026 10:55
@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Mar 3, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akrzos

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot Bot merged commit 7143305 into redhat-performance:main Mar 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants