Skip to content

Unskip sagemaker serve integration tests#5810

Merged
mujtaba1747 merged 9 commits intoaws:masterfrom
mujtaba1747:fix-optimize-integration-test
May 4, 2026
Merged

Unskip sagemaker serve integration tests#5810
mujtaba1747 merged 9 commits intoaws:masterfrom
mujtaba1747:fix-optimize-integration-test

Conversation

@mujtaba1747
Copy link
Copy Markdown
Collaborator

@mujtaba1747 mujtaba1747 commented Apr 30, 2026

Issue #, if available: #5802

Description of changes: Fix skipped tests in sagemaker-serve

test_model_customization_deployment.py

Problem:
cleanup_e2e_endpoints is a session-scoped fixture, but under pytest-xdist each worker gets its own session. This means multiple workers independently call Endpoint.get_all() and delete every e2e-* endpoint — including endpoints that another worker just created and is actively using.

Observed failure: Test running on worker gw2 deploys e2e-1777420521-4224, its base inference component goes InService, then a different worker's cleanup deletes it. The test on gw2 then fails when it tries to Endpoint.get() the endpoint it just created.

The teardown path had the same problem — it ran the same Endpoint.get_all() + delete loop after tests, risking a second round of collisions.

Changes:

Only delete e2e endpoints older than 24 hrs.

test_optimize_integration.py

Problem: Test kept failing because it used to run in us-east-2. And the test failed due to race condition: Other tests initializing the singleton SageMakerClient class in different region causing resource not found errors in test_optimize_integration,

Changes: Ensure tests run in a single region. Parameterized image uri in test_optimize_integration so it can run in us-west-2 and ported to other regions in future.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@mujtaba1747 mujtaba1747 changed the title Fix optimize integration test Fix sagemaker serve integration tests May 4, 2026
@mujtaba1747 mujtaba1747 changed the title Fix sagemaker serve integration tests Unskip sagemaker serve integration tests May 4, 2026
@mujtaba1747 mujtaba1747 merged commit ad9458c into aws:master May 4, 2026
15 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants