Skip to content

Conversation

@michaelnebel
Copy link
Contributor

@michaelnebel michaelnebel commented Sep 10, 2025

In this PR we increase the number of attempts (and thus the timeout) for reaching nuget.org (which is the default nuget fallback feed) to reduce integration test flakiness. If we still see an unacceptable level of flakiness we could consider to remove the nuget fallback reachability check from some of the tests to reduce network impact on the integration tests.

@github-actions github-actions bot added the C# label Sep 10, 2025
@michaelnebel michaelnebel marked this pull request as ready for review September 10, 2025 07:06
@michaelnebel michaelnebel requested a review from a team as a code owner September 10, 2025 07:06
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR increases timeout and retry attempts for reaching nuget.org in C# integration tests to reduce flakiness caused by network connectivity issues. The changes set environment variables to configure a higher timeout (1000ms) and more retry attempts (5) for the NuGet fallback feed reachability check.

Key Changes

  • Added environment variable configuration for NuGet fallback feed timeout and retry limits
  • Applied consistent timeout and retry settings across multiple integration test files

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
csharp/ql/integration-tests/posix/standalone_dependencies_nuget_config_error/test.py Added import for os module and configured NuGet fallback feed timeout/retry environment variables
csharp/ql/integration-tests/all-platforms/standalone_winforms/test.py Added import for os module and configured NuGet fallback feed timeout/retry environment variables
csharp/ql/integration-tests/all-platforms/standalone_resx/test.py Added NuGet fallback feed timeout/retry environment variable configuration (os import already existed)

Comment on lines 4 to +9

@runs_on.posix
def test(codeql, csharp):
# Making sure the reachability test of `nuget.org` succeeds:
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000"
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5"
Copy link

Copilot AI Sep 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider extracting these magic numbers (1000, 5) into named constants at the module level to improve maintainability and ensure consistency across test files.

Suggested change
@runs_on.posix
def test(codeql, csharp):
# Making sure the reachability test of `nuget.org` succeeds:
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000"
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5"
# Constants for NuGet feeds check fallback configuration
NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS = "1000"
NUGET_FEEDS_CHECK_FALLBACK_LIMIT = "5"
@runs_on.posix
def test(codeql, csharp):
# Making sure the reachability test of `nuget.org` succeeds:
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = NUGET_FEEDS_CHECK_FALLBACK_LIMIT

Copilot uses AI. Check for mistakes.
Comment on lines 4 to +7
def test(codeql, csharp):
# Making sure the reachability test of `nuget.org` succeeds:
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000"
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5"
Copy link

Copilot AI Sep 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider extracting these magic numbers (1000, 5) into named constants at the module level to improve maintainability and ensure consistency across test files.

Suggested change
def test(codeql, csharp):
# Making sure the reachability test of `nuget.org` succeeds:
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000"
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5"
NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS = "1000"
NUGET_FEEDS_CHECK_FALLBACK_LIMIT = "5"
def test(codeql, csharp):
# Making sure the reachability test of `nuget.org` succeeds:
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = NUGET_FEEDS_CHECK_FALLBACK_LIMIT

Copilot uses AI. Check for mistakes.
Comment on lines 4 to +7
def test(codeql, csharp):
# Making sure the reachability test of `nuget.org` succeeds:
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000"
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5"
Copy link

Copilot AI Sep 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider extracting these magic numbers (1000, 5) into named constants at the module level to improve maintainability and ensure consistency across test files.

Suggested change
def test(codeql, csharp):
# Making sure the reachability test of `nuget.org` succeeds:
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = "1000"
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = "5"
# Constants for NuGet feed fallback configuration
NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS = "1000"
NUGET_FEEDS_CHECK_FALLBACK_LIMIT = "5"
def test(codeql, csharp):
# Making sure the reachability test of `nuget.org` succeeds:
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT"] = NUGET_FEEDS_CHECK_FALLBACK_TIMEOUT_MS
os.environ["CODEQL_EXTRACTOR_CSHARP_BUILDLESS_NUGET_FEEDS_CHECK_FALLBACK_LIMIT"] = NUGET_FEEDS_CHECK_FALLBACK_LIMIT

Copilot uses AI. Check for mistakes.
@michaelnebel michaelnebel merged commit 046d0d4 into github:main Sep 10, 2025
19 checks passed
@michaelnebel michaelnebel deleted the csharp/improveintegrationteststability branch September 10, 2025 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants