Skip to content

improvement(compat testing): Automatically load a second container when running cross-client compat e2e tests #24889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

scottn12
Copy link
Contributor

@scottn12 scottn12 commented Jun 23, 2025

This is a WIP and is not yet intended to be merged

Description

This PR ensures that a second container is always loaded when running cross-client compat tests. Additionally, it adds skipSecondContainerLoad to ITestContainerConfig which allows tests authors to disable this functionality.
This PR leverages skipSecondContainerLoad in 4 test suites. Without that option, those test suites would fail since they were written with the assumption that only a single contianer would be loaded.

Test Runtime Increase

On a codespaces instance, I ran time pnpm test from the test-end-to-end-tests package three times with and without the changes in this PR.

With second container
Run 1: 13m3.357s
Run 2: 14m18.576s
Run 3: 12m44.915s

Without
Run 1: 12m38.526s
Run 2: 12m23.861s
Run 3: 12m41.466s

We would need to re-test for more accurate/consistent results, but we can expect a small, but noticeable, increase in runtime.

Misc

AB#6826

@github-actions github-actions bot added area: tests Tests to add, test infrastructure improvements, etc base: main PRs targeted against main branch labels Jun 23, 2025
@scottn12 scottn12 changed the title improvement(compat testing): Always load a second container when running cross-client compat e2e tests improvement(compat testing): Automatically load a second container when running cross-client compat e2e tests Jun 23, 2025
@scottn12 scottn12 requested a review from agarwal-navin June 23, 2025 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: tests Tests to add, test infrastructure improvements, etc base: main PRs targeted against main branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant