-
Notifications
You must be signed in to change notification settings - Fork 1.8k
C#: Increase attempts to reach nuget.org in integration tests to reduce flakiness. #20396
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
C#: Increase attempts to reach nuget.org in integration tests to reduce flakiness. #20396
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR increases timeout and retry attempts for reaching nuget.org in C# integration tests to reduce flakiness caused by network connectivity issues. The changes set environment variables to configure a higher timeout (1000ms) and more retry attempts (5) for the NuGet fallback feed reachability check.
Key Changes
- Added environment variable configuration for NuGet fallback feed timeout and retry limits
- Applied consistent timeout and retry settings across multiple integration test files
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| csharp/ql/integration-tests/posix/standalone_dependencies_nuget_config_error/test.py | Added import for os module and configured NuGet fallback feed timeout/retry environment variables |
| csharp/ql/integration-tests/all-platforms/standalone_winforms/test.py | Added import for os module and configured NuGet fallback feed timeout/retry environment variables |
| csharp/ql/integration-tests/all-platforms/standalone_resx/test.py | Added NuGet fallback feed timeout/retry environment variable configuration (os import already existed) |
|
|
||
| @runs_on.posix | ||
| def test(codeql, csharp): | ||
| # Making sure the reachability test of `nuget.org` succeeds: | ||
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000" | ||
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5" |
Copilot
AI
Sep 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] Consider extracting these magic numbers (1000, 5) into named constants at the module level to improve maintainability and ensure consistency across test files.
| @runs_on.posix | |
| def test(codeql, csharp): | |
| # Making sure the reachability test of `nuget.org` succeeds: | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000" | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5" | |
| # Constants for NuGet feeds check fallback configuration | |
| NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS = "1000" | |
| NUGET_FEEDS_CHECK_FALLBACK_LIMIT = "5" | |
| @runs_on.posix | |
| def test(codeql, csharp): | |
| # Making sure the reachability test of `nuget.org` succeeds: | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = NUGET_FEEDS_CHECK_FALLBACK_LIMIT |
| def test(codeql, csharp): | ||
| # Making sure the reachability test of `nuget.org` succeeds: | ||
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000" | ||
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5" |
Copilot
AI
Sep 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] Consider extracting these magic numbers (1000, 5) into named constants at the module level to improve maintainability and ensure consistency across test files.
| def test(codeql, csharp): | |
| # Making sure the reachability test of `nuget.org` succeeds: | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000" | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5" | |
| NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS = "1000" | |
| NUGET_FEEDS_CHECK_FALLBACK_LIMIT = "5" | |
| def test(codeql, csharp): | |
| # Making sure the reachability test of `nuget.org` succeeds: | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = NUGET_FEEDS_CHECK_FALLBACK_LIMIT |
| def test(codeql, csharp): | ||
| # Making sure the reachability test of `nuget.org` succeeds: | ||
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000" | ||
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5" |
Copilot
AI
Sep 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] Consider extracting these magic numbers (1000, 5) into named constants at the module level to improve maintainability and ensure consistency across test files.
| def test(codeql, csharp): | |
| # Making sure the reachability test of `nuget.org` succeeds: | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000" | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5" | |
| # Constants for NuGet feed fallback configuration | |
| NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS = "1000" | |
| NUGET_FEEDS_CHECK_FALLBACK_LIMIT = "5" | |
| def test(codeql, csharp): | |
| # Making sure the reachability test of `nuget.org` succeeds: | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS | |
| os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = NUGET_FEEDS_CHECK_FALLBACK_LIMIT |
In this PR we increase the number of attempts (and thus the timeout) for reaching
nuget.org(which is the default nuget fallback feed) to reduce integration test flakiness. If we still see an unacceptable level of flakiness we could consider to remove the nuget fallback reachability check from some of the tests to reduce network impact on the integration tests.