Improve/ add retries at the functional test level #7174
Comments
willdavsmith
added
the
maintenance
|
👋 @willdavsmith Thanks for filing this issue. A project maintainer will review this issue and get back to you soon. We also welcome community contributions! If you would like to pick this item up sooner and submit a pull request, please visit our contribution guidelines and assign this to yourself by commenting "/assign" on this issue. For more information on our triage process please visit our triage overview |
|
👍 We've reviewed this issue and have agreed to add it to our backlog. Please subscribe to this issue for notifications, we'll provide updates when we pick it up. We also welcome community contributions! If you would like to pick this item up sooner and submit a pull request, please visit our contribution guidelines and assign this to yourself by commenting "/assign" on this issue. For more information on our triage process please visit our triage overview |
# Description This is the test workflow to run functional test with retry every hour. It will help to understand how frequently the failure happens and confirm whether retry on test works as expected. ## Type of change <!-- Please select **one** of the following options that describes your change and delete the others. Clearly identifying the type of change you are making will help us review your PR faster, and is used in authoring release notes. If you are making a bug fix or functionality change to Radius and do not have an associated issue link please create one now. --> - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional). <!-- Please update the following to link the associated issue. This is required for some kinds of changes (see above). --> #7174 --------- Signed-off-by: Young Bu Park <youngp@microsoft.com>
# Description This will not configure the aggressive retry which can hide the bug. So I set 2 retries for the first iteration. * Enable retry on functional test to resolve the intermittent failures. (#7174) * Ignore the failures of `clean up cluster` step - Based on the investigation, timeout happens by deleting resources. It is ok to skip for failure because long-running test always clean up cluster first before running functional test. ( * Enable retry on `Configure Radius test workspace` to avoid [the intermittent 503 failure](#7401) ## Type of change <!-- Please select **one** of the following options that describes your change and delete the others. Clearly identifying the type of change you are making will help us review your PR faster, and is used in authoring release notes. If you are making a bug fix or functionality change to Radius and do not have an associated issue link please create one now. --> - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional). <!-- Please update the following to link the associated issue. This is required for some kinds of changes (see above). --> #7174 Signed-off-by: Young Bu Park <youngp@microsoft.com>
Area for Improvement
Today, we have retries at the verification level, but not at the functional test level. We could add functional test retries in the case of network failure and so on. We need to make sure to clean up test resources though.
Observed behavior
No response
Desired behavior
functional test failures
Proposed Fix
No response
rad Version
n/a
Operating system
No response
Additional context
No response
Would you like to support us?
AB#11172
The text was updated successfully, but these errors were encountered: