Fix test isolation for test_watsonx_gpt_oss_prompt_transformation

[shin-bot-litellm]

Regression Fix

Failing Job: litellm_mapped_tests_llms
Caused By: Test isolation issue introduced in 1017c3a
Author: @ishaan-jaff

What Broke

Test test_watsonx_gpt_oss_prompt_transformation fails intermittently when running with parallel test execution (-n 16).

Error:

AssertionError: Prompt transformation failed - the template should have been applied to transform messages into the correct format for gpt-oss-120b.
assert None is not None

Original Commit LOC

The test originally:

1. Cleared the litellm.known_tokenizer_config cache
2. Mocked only async tokenizer functions (_aget_tokenizer_config, _aget_chat_template_file)

This Fix

The issue was a race condition with parallel tests:

• Another test could populate the cache between clearing and usage
• When cached, the sync code path is used instead of async
• The sync path wasn't mocked, causing the template fetch to fail

Fix:

1. Set cache directly instead of clearing it (avoids race condition)
2. Also mock sync versions _get_tokenizer_config and _get_chat_template_file

This ensures the test is deterministic regardless of test execution order.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 5, 2026 6:30am

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Overview

Greptile Summary

This PR makes test_watsonx_gpt_oss_prompt_transformation deterministic under pytest-xdist by avoiding a cache-clear race and by mocking both async and sync HuggingFace template fetch helpers. The test now pre-populates litellm.known_tokenizer_config for openai/gpt-oss-120b and patches _get_* / _aget_* functions so prompt transformation always uses the mocked template regardless of which code path is hit.

One issue to address before merge: the test writes into the global litellm.known_tokenizer_config cache but never restores the prior value, which can leak mocked state into later tests within the same worker and reintroduce order dependence.

Confidence Score: 4/5

• Mostly safe to merge, but test isolation needs one more fix to avoid leaking global cache state across tests.
• Change is small and targeted (mocks sync+async paths to eliminate xdist flake). Main remaining risk is the new mutation of litellm.known_tokenizer_config without restoration, which can cause order-dependent failures in the same worker.
• tests/test_litellm/llms/watsonx/test_watsonx.py

Important Files Changed

Filename	Overview
tests/test_litellm/llms/watsonx/test_watsonx.py	Fixes intermittent async test by pre-populating litellm.known_tokenizer_config and mocking sync/async HF template fetch; however, it now mutates global cache without restoring, which can leak into later tests.

Sequence Diagram

sequenceDiagram
  participant T as pytest (xdist worker)
  participant Test as test_watsonx_gpt_oss_prompt_transformation
  participant L as litellm.acompletion
  participant HF as huggingface_template_handler
  participant C as litellm.known_tokenizer_config
  participant W as AsyncHTTPHandler.post (mock)

  T->>Test: run test
  Test->>C: set C["openai/gpt-oss-120b"] = mock_tokenizer_config
  Note over Test,HF: Patch async and sync HF helpers
  Test->>L: acompletion(model=watsonx_text/openai/gpt-oss-120b, messages)
  L->>HF: resolve chat template
  alt cache hit / sync path
    HF->>C: read C[hf_model]
    HF-->>L: tokenizer_config with chat_template
  else async path
    HF->>HF: _aget_tokenizer_config (patched)
    HF->>HF: _aget_chat_template_file (patched -> failure)
    HF-->>L: tokenizer_config with chat_template
  end
  L->>W: POST completion (mock)
  W-->>L: mock_completion_response
  L-->>Test: response
  Test->>Test: assert request input contains template tags

Loading

[greptile-apps]

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Global cache not restored

This test now writes to the module-level litellm.known_tokenizer_config (litellm.known_tokenizer_config[hf_model] = ...) but never restores the previous value. That makes the test order-dependent: subsequent tests in the same worker will see the mocked tokenizer config and may skip their intended code paths. Consider snapshotting the previous entry (or absence) and restoring it in a try/finally (or using a fixture) so the global cache is returned to its prior state when the test completes.

Prompt To Fix With AI

This is a comment left during a code review.
Path: tests/test_litellm/llms/watsonx/test_watsonx.py
Line: 286:291

Comment:
**Global cache not restored**

This test now writes to the module-level `litellm.known_tokenizer_config` (`litellm.known_tokenizer_config[hf_model] = ...`) but never restores the previous value. That makes the test order-dependent: subsequent tests in the same worker will see the mocked tokenizer config and may skip their intended code paths. Consider snapshotting the previous entry (or absence) and restoring it in a `try/finally` (or using a fixture) so the global cache is returned to its prior state when the test completes.

How can I resolve this? If you propose a fix, please make it concise.