Skip to content

fix(tests): Fix flaky container and scientific notation tests#20650

Merged
ishaan-jaff merged 2 commits intomainfrom
litellm_fix_container_test_mock
Feb 7, 2026
Merged

fix(tests): Fix flaky container and scientific notation tests#20650
ishaan-jaff merged 2 commits intomainfrom
litellm_fix_container_test_mock

Conversation

@shin-bot-litellm
Copy link
Collaborator

Test Fixes

1. test_router_acreate_container_without_model

Problem: The test mocked container_create_handler (sync) but router.acreate_container uses _is_async=True which calls async_container_create_handler (async). This caused the test to hit the real OpenAI API with Authorization: Bearer None.

Fix: Use AsyncMock on async_container_create_handler instead.

2. test_register_model_with_scientific_notation

Problem: The test used a static "unique" model name (test-scientific-notation-model-unique-12345). When tests run in parallel with -n 16 in CI, this can cause conflicts between test workers accessing the shared litellm.model_cost dictionary.

Fix: Use uuid.uuid4() to generate truly unique model names per test run.


Fixes for CI stability

Shin added 2 commits February 7, 2026 17:44
The test was mocking container_create_handler (sync), but
router.acreate_container uses _is_async=True which calls
async_container_create_handler. This caused the test to hit
the real OpenAI API.

Fixed by using AsyncMock on async_container_create_handler.
The test was using a static "unique" model name which could cause
conflicts when running tests in parallel (-n 16 in CI). Using uuid
ensures truly unique names to prevent test pollution.
@vercel
Copy link

vercel bot commented Feb 7, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 7, 2026 5:48pm

Request Review

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Shin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 7, 2026

Greptile Overview

Greptile Summary

Fixed two flaky tests that were causing CI instability when running with parallel test workers.

  • test_router_acreate_container_without_model: Corrected mock from sync container_create_handler to async async_container_create_handler using AsyncMock, preventing the test from hitting the real OpenAI API with invalid credentials
  • test_register_model_with_scientific_notation: Replaced static model name with uuid-based unique names to prevent race conditions when tests run in parallel with -n 16
  • Removed unused imports (_cached_get_model_info_helper, get_model_info) that are no longer needed after cleanup

Both fixes are targeted and correct, addressing the root causes of test flakiness without introducing new issues.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • Both changes correctly fix test flakiness issues with targeted, well-documented fixes. The async mock change properly aligns with the code path execution, and the uuid approach definitively solves parallel test conflicts. No logic changes to production code.
  • No files require special attention

Important Files Changed

Filename Overview
tests/test_litellm/containers/test_container_api.py Fixed async mock to match router.acreate_container's async call path
tests/test_litellm/test_utils.py Used uuid for unique model names to prevent parallel test conflicts

Sequence Diagram

sequenceDiagram
    participant Test as test_router_acreate_container
    participant Router as router.acreate_container
    participant CreateContainer as create_container
    participant Handler as base_llm_http_handler
    participant AsyncHandler as async_container_create_handler
    
    Test->>Router: await router.acreate_container(name="Test Container")
    Router->>CreateContainer: create_container(..., async_call=True)
    Note over CreateContainer: Sets _is_async=True from kwargs
    CreateContainer->>Handler: container_create_handler(..., _is_async=True)
    Note over Handler: Checks _is_async flag
    Handler->>AsyncHandler: async_container_create_handler(...)
    Note over Test: Mock intercepts here (AsyncMock)
    AsyncHandler-->>Handler: ContainerObject
    Handler-->>CreateContainer: ContainerObject
    CreateContainer-->>Router: ContainerObject
    Router-->>Test: ContainerObject
Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@ishaan-jaff ishaan-jaff merged commit df299d3 into main Feb 7, 2026
11 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants