Prevent adding 100% AB tests without intermediate smaller test - #16623
Merged
Conversation
🚀 Image pushed to AWS ECRImage digest: 🐛 Run the image locallyThe following can be used to run the image locally: # Refer to image using the immutable digest. Find alternatives below.
IMAGE_IDENTIFIER="@sha256:25b46774e43c72fb0557e2f0b2e8fd73ccc2b1d4230188ad54f834418c63e872"
# Refer to image using branch tag
# IMAGE_IDENTIFIER=":branch-jlk-prevent-100-perc-test"
# Refer to image using build tag
# IMAGE_IDENTIFIER=":build-30502"
# Refer to image via the GitHub commit SHA tag
# IMAGE_IDENTIFIER=":sha-34b1bf51d71a397117fc7126aba6bff75d3a4aad"
# Set environment variables for the AWS CLI
AWS_PROFILE="<A_PROFILE_FROM_JANUS>"
AWS_DEFAULT_REGION="eu-west-1"
IMAGE_ACCOUNT_ID=$(aws ssm get-parameter --name /organisation/accounts/deployTools --query "Parameter.Value" --output text)
REGISTRY="${IMAGE_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com"
IMAGE="${REGISTRY}/guardian/dotcom-rendering${IMAGE_IDENTIFIER}"
# Login to AWS ECR https://docs.aws.amazon.com/AmazonECR/latest/userguide/registry_auth.html
aws ecr get-login-password | docker login --username AWS --password-stdin $REGISTRY
# Pull the image
docker pull $IMAGE
# Run the image. You'll likely need to set additional flags. See https://docs.docker.com/reference/cli/docker/container/run.
docker run $IMAGE |
arelra
approved these changes
Aug 31, 2026
arelra
left a comment
arelra
left a comment
Member
There was a problem hiding this comment.
Great to have this as an automated check
gustavo-olmedo
pushed a commit
that referenced
this pull request
Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this change?
Prevent new 100% tests that didn't start at a smaller size, with a link to documentation on why
Why?
This has come up a few times with 100% tests, it's mentioned in the docs but it's easy to miss.
The link explains that this can cause a massive cache invalidation, which would spike traffic to origin, this should prevent it happening in the future.
I've added this in the build step rather than the validation step as the validation step doesn't have access to the current/previews state of the tests.
How has this change been tested?
Added unit tests