Skip to content

[Fix] Migrate unittest parameterization to PyTorch - #2002

Closed
jayhenry wants to merge 3 commits into
InternLM:mainfrom
jayhenry:fix/torch-unittest-parametrize
Closed

[Fix] Migrate unittest parameterization to PyTorch#2002
jayhenry wants to merge 3 commits into
InternLM:mainfrom
jayhenry:fix/torch-unittest-parametrize

Conversation

@jayhenry

@jayhenry jayhenry commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • replace the third-party parametrize decorator in 28 test files
  • use PyTorch parametrize plus instantiate_parametrized_tests for unittest-based classes
  • use native pytest.mark.parametrize for the one plain pytest class
  • remove the third-party dependency and document why scanning module globals is unsafe with Transformers lazy modules
  • align Qwen3/Qwen3.5 behavior and regression expectations with Transformers 5.14.1

Root cause

The third-party decorator recursively scans decoration-frame globals to count stacked decorators. Transformers lazy modules can materialize attributes while that dictionary is being iterated, which raises RuntimeError: dictionary changed size during iteration. The upgrade exposed the latent incompatibility by changing the lazy-module surface imported during test collection.

PyTorch unittest parameterization records metadata on the test method and materializes cases from the decorated class. It does not recursively inspect imported module dictionaries.

Why parameterization is split

pytest.mark.parametrize cannot be used directly on unittest.TestCase subclasses. Pytest delegates those methods to the unittest runner, which invokes them without injected arguments. A minimal reproduction under pt29_glm2 fails with TypeError: test_value() missing 1 required positional argument.

Therefore:

  • unittest-based classes use PyTorch parametrize plus instantiate_parametrized_tests
  • plain pytest classes use pytest.mark.parametrize

This keeps each test on the parameterization mechanism supported by its runner without changing existing test base classes.

Transformers 5.14.1 compatibility

The first Action run proved that 547 tests still passed and isolated 12 remaining regressions. This PR also addresses them:

  • call the current Qwen3.5 dense decoder with keyword-only position_embeddings and seq_ctx
  • preserve the outer vision boundary required by current processors when a video token expands into per-frame placeholders, for both Qwen3-VL and Qwen3.5-VL
  • use get_vision_bilinear_indices_and_weights, the shared Transformers 5.14 vision interpolation API, while retaining FP32 accumulation and returning the model dtype before LayerNorm
  • update Qwen3.5 MTP loss references for the 5.14.1 token sequence and fused kernels
  • use an image-only tolerance for the bounded numerical drift introduced by the current fused GatedDeltaNet/MoE language path; instrumentation verified that the vision output and language-model input remain bitwise equal before the drift accumulates across 40 language layers
  • raise the affected Qwen3.5 test version guards to Transformers 5.14

Validation

Using pt29_glm2, variables from zdev/env.sh, and gpu_lock.sh for GPU runs:

  • minimal reproduction: third-party decorator fails during collection; PyTorch replacement collects successfully
  • minimal reproduction: pytest.mark.parametrize on unittest.TestCase fails because unittest does not inject parameters
  • all 28 changed test files: 212 tests collected
  • static audit: every PyTorch-parameterized method has instantiate_parametrized_tests
  • MoE fixture regression plus both dense decoder cases: 3 passed
  • Qwen3.5 standard VL, SP=1/4: 2 passed
  • Qwen3.5 MTP, SP=1/4: 2 passed
  • Qwen3.5 dense vision tower bitwise parity: 1 passed
  • production-file Ruff checks and git diff --check: passed
  • PR lint Action: passed

The follow-up Action completed with 558 passed, 18 skipped, and one stale Qwen3.5 chat-template expectation. The four original HF video-reference failures passed. Commit cde72f58 updates both remaining video-template expectations for the Transformers 5.14 outer boundary; the exact failed node now passes locally.

@jayhenry jayhenry closed this Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant