Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable retry on functional test #7430

Merged
merged 4 commits into from
Apr 3, 2024
Merged

Enable retry on functional test #7430

merged 4 commits into from
Apr 3, 2024

Conversation

youngbupark
Copy link

@youngbupark youngbupark commented Apr 2, 2024

Description

This will not configure the aggressive retry which can hide the bug. So I set 2 retries for the first iteration.

  • Enable retry on functional test to resolve the intermittent failures. (Improve/ add retries at the functional test level #7174)
  • Ignore the failures of clean up cluster step - Based on the investigation, timeout happens by deleting resources. It is ok to skip for failure because long-running test always clean up cluster first before running functional test. (
  • Enable retry on Configure Radius test workspace to avoid the intermittent 503 failure

Type of change

  • This pull request fixes a bug in Radius and has an approved issue (issue link required).
  • This pull request adds or changes features of Radius and has an approved issue (issue link required).
  • This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

#7174

Signed-off-by: Young Bu Park <youngp@microsoft.com>
Copy link

github-actions bot commented Apr 2, 2024

Test Results

3 136 tests  ±0   3 129 ✅ ±0   3m 31s ⏱️ +2s
  261 suites ±0       7 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 3f9a1e8. ± Comparison against base commit c92ce94.

♻️ This comment has been updated with latest results.

Copy link

codecov bot commented Apr 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.79%. Comparing base (186994d) to head (a9d61d9).
Report is 2 commits behind head on main.

❗ Current head a9d61d9 differs from pull request most recent head 3f9a1e8. Consider uploading reports for the commit 3f9a1e8 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7430      +/-   ##
==========================================
- Coverage   63.81%   63.79%   -0.02%     
==========================================
  Files         416      416              
  Lines       23440    23440              
==========================================
- Hits        14958    14954       -4     
- Misses       7223     7225       +2     
- Partials     1259     1261       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@radius-functional-tests
Copy link

radius-functional-tests bot commented Apr 3, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref a9d61d9
Unique ID funcd0f8ad3b07
Image tag pr-funcd0f8ad3b07
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcd0f8ad3b07
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcd0f8ad3b07
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funcd0f8ad3b07
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcd0f8ad3b07
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting kubernetes functional tests...
⌛ Starting ucp functional tests...
⌛ Starting samples functional tests...
⌛ Starting shared functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting cli functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting daprrp functional tests...
✅ kubernetes functional tests succeeded
✅ msgrp functional tests succeeded
✅ ucp functional tests succeeded
✅ samples functional tests succeeded
✅ daprrp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ cli functional tests succeeded
❌ shared functional test failed. Please check the logs for more details

@radius-functional-tests
Copy link

radius-functional-tests bot commented Apr 3, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref 3f9a1e8
Unique ID func3a40edc42b
Image tag pr-func3a40edc42b
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func3a40edc42b
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func3a40edc42b
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func3a40edc42b
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func3a40edc42b
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting daprrp functional tests...
⌛ Starting cli functional tests...
⌛ Starting ucp functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting shared functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting datastoresrp functional tests...
✅ msgrp functional tests succeeded
✅ samples functional tests succeeded
✅ ucp functional tests succeeded
✅ kubernetes functional tests succeeded
✅ daprrp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ cli functional tests succeeded
✅ shared functional tests succeeded

@youngbupark youngbupark merged commit bef1134 into main Apr 3, 2024
18 checks passed
@youngbupark youngbupark deleted the youngp/retry branch April 3, 2024 23:47
youngbupark pushed a commit that referenced this pull request Apr 4, 2024
youngbupark pushed a commit that referenced this pull request Apr 4, 2024
This reverts commit bef1134.

Signed-off-by: Young Bu Park <youngp@microsoft.com>
youngbupark pushed a commit that referenced this pull request Apr 4, 2024
This reverts commit bef1134.

# Description

Reverting the changes because retrying the entire test doesn't work as
expected.

## Type of change

<!--

Please select **one** of the following options that describes your
change and delete the others. Clearly identifying the type of change you
are making will help us review your PR faster, and is used in authoring
release notes.

If you are making a bug fix or functionality change to Radius and do not
have an associated issue link please create one now.

-->

- This pull request fixes a bug in Radius and has an approved issue
(issue link required).
- This pull request adds or changes features of Radius and has an
approved issue (issue link required).
- This pull request is a minor refactor, code cleanup, test improvement,
or other maintenance task and doesn't change the functionality of Radius
(issue link optional).

<!--

Please update the following to link the associated issue. This is
required for some kinds of changes (see above).

-->

Fixes: #issue_number

---------

Signed-off-by: Young Bu Park <youngp@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants