Skip to content

chore: make MSK tests non-blocking #3170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 16, 2023
Merged

chore: make MSK tests non-blocking #3170

merged 3 commits into from
May 16, 2023

Conversation

hoffa
Copy link
Contributor

@hoffa hoffa commented May 16, 2023

Issue #, if available

Description of changes

Encountering flakiness every blue moon with this tests; added equivalent transform tests and marking as non-blocking since we've already tested it.

For reference, the guidelines for marking tests as @nonblocking are:

# In general, we should only mark integration tests as @nonblocking if
# 1. The test succeeded every region (this ensures the transformed output makes sense)
# 2. The test resources are defined in a single template without Parameters or other
# CloudFormation macros (this ensures we can represent transform using a transform test)
# 3. An identical transform test exists for the integration test template (this ensures we
# don't break the working template)
nonblocking = pytest.mark.xfail

In this case we do use Parameters. Specifically, the tests use these subnets:

MyMskCluster:
Type: AWS::MSK::Cluster
Properties:
BrokerNodeGroupInfo:
ClientSubnets:
- Ref: PreCreatedSubnetOne
- Ref: PreCreatedSubnetTwo

def get_parameters(self, dictionary):
parameters = []
parameters.append(self.generate_parameter("PreCreatedSubnetOne", dictionary["PreCreatedSubnetOne"]))
parameters.append(self.generate_parameter("PreCreatedSubnetTwo", dictionary["PreCreatedSubnetTwo"]))
return parameters

Created by the companion stack:

PreCreatedSubnetTwo:
Description: 'Pre-created #2 subnet that can be used inside other tests'
Value:
Ref: PreCreatedSubnetTwo
PreCreatedSubnetOne:
Description: 'Pre-created #1 subnet that can be used inside other tests'
Value:
Ref: PreCreatedSubnetOne

However those parameters don't have any SAM-specific handling since they're passed straight to AWS::MSK::Cluster, so we can be more lax on (2).

For transform test had to replace CodeUri to make it work with local transform.

Description of how you validated changes

With outdated credentials:

pytest integration --no-cov -k msk
==== 188 deselected, 2 xfailed, 2 warnings in 1.79s ====

Checklist

Examples?

Please reach out in the comments if you want to add an example. Examples will be
added to sam init through aws/aws-sam-cli-app-templates.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@hoffa hoffa marked this pull request as ready for review May 16, 2023 20:48
@hoffa hoffa requested a review from a team as a code owner May 16, 2023 20:48
@hoffa hoffa changed the title chore: make MSK test non-blocking chore: make MSK tests non-blocking May 16, 2023
@hoffa hoffa merged commit c2d7e21 into aws:develop May 16, 2023
@hoffa hoffa deleted the nonblocking-msk branch May 16, 2023 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants