Skip to content

chore: add resilience-test suite and remove individual test#6807

Open
PeterSchafer wants to merge 1 commit into
mainfrom
chore/CLI-1494_fault_injection
Open

chore: add resilience-test suite and remove individual test#6807
PeterSchafer wants to merge 1 commit into
mainfrom
chore/CLI-1494_fault_injection

Conversation

@PeterSchafer
Copy link
Copy Markdown
Contributor

Pull Request Submission Checklist

  • Follows CONTRIBUTING guidelines
  • Commit messages
    are release-note ready, emphasizing
    what was changed, not how.
  • Includes detailed description of changes
  • Contains risk assessment (Low | Medium | High)
  • Highlights breaking API changes (if applicable)
  • Links to automated tests covering new functionality
  • Includes manual testing instructions (if necessary)
  • Updates relevant GitBook documentation (PR link: ___)
  • Includes product update to be announced in the next stable release notes

What does this PR do?

This PR moves existing separate tests into a single test suite for easier overview and maintenance.

Where should the reviewer start?

How should this be manually tested?

What's the product update that needs to be communicated to CLI users?

N/A

Risk assessment (Low | Medium | High)?

Low, changes tests only

@PeterSchafer PeterSchafer requested review from a team as code owners May 14, 2026 16:39
@snyk-io
Copy link
Copy Markdown

snyk-io Bot commented May 14, 2026

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues
Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@snyk-pr-review-bot
Copy link
Copy Markdown

PR Reviewer Guide 🔍

🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Global Side Effect 🟠 [major]

Enabling isSecretsEnabled by default in featureFlagDefaults affects all acceptance tests in the repository that use the fakeServer. The existing comment in this file explicitly states that features should be enabled specifically by tests that target them. By making this a global default, other tests that do not expect or mock secrets-related API calls may fail when the CLI attempts to communicate with those endpoints.

['isSecretsEnabled', true],
Brittle Test Scenario 🟡 [minor]

In Scenario 4 (mid-execution-maintenance), the test sets the next status code to 200 for 4 requests without providing mock response bodies. CLI commands like test or monitor typically expect valid JSON from their initial API calls (e.g., for organization or user verification). If these calls return an empty 200 response, the CLI will fail with a JSON parsing error before reaching the maintenance failure state, causing the test to fail with an incorrect exit code.

server.setNextStatusCode(200);
server.setNextStatusCode(200);
server.setNextStatusCode(200);
server.setNextStatusCode(200);
📚 Repository Context Analyzed

This review considered 5 relevant code sections from 4 files (average relevance: 1.00)

['sbomTestReachability', false],
['useTestShimForOSCliTest', false],
['cliDotnetRuntimeResolution', false],
['isSecretsEnabled', true],
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we want to enable this only in the tests, to avoid doing this globally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant