Skip to content

Modular: three tiny test fixtures cannot serve the new workflow tests #14389

Description

@yiyixuxu

Opened by Claude (Opus 5) on behalf of @yiyixuxu.

Describe the bug

#14355 adds three shared modular tests — test_from_pretrained_workflow, test_load_components_workflow and test_unload_components — that build a pipeline from the test class's pretrained_model_name_or_path and compare it against the blocks the test class declares:

pipe = ModularPipeline.from_pretrained(self.pretrained_model_name_or_path, workflow=workflow_name)
ref_pipe = blocks.get_workflow(workflow_name).init_pipeline(self.pretrained_model_name_or_path)
assert set(pipe.component_names) == set(ref_pipe.component_names)

Three hf-internal-testing fixtures cannot answer that, in three different ways. None of them is a problem with the pipelines' code — they are the first tests to go through from_pretrained for these classes at all, since ModularPipelineTesterMixin.get_pipeline builds from self.pipeline_blocks_class() directly. They are skipped with a TODO each on that branch; this issue tracks the fixtures.

1. hf-internal-testing/tiny-anima-modular-pipe does not carry a modular_model_index.json

OSError: Failed to load config from 'hf-internal-testing/tiny-anima-modular-pipe'.
Could not find or load 'modular_model_index.json' or 'model_index.json'.

Anima's test classes override get_pipeline to assemble dummy components in-process:

def get_pipeline(self, components_manager=None, dtype=torch.float32):
    pipe = self.pipeline_blocks_class().init_pipeline(components_manager=components_manager)
    pipe.update_components(**get_dummy_components())

so pretrained_model_name_or_path was never loaded from and the repository was never needed. Fix: publish a tiny Anima modular repository under that name. Affects TestAnimaModularPipelineFast and TestAnimaImg2ImgModularPipelineFast, three tests each.

2. hf-internal-testing/tiny-flux2-klein-modular names the base blocks for a distilled checkpoint

{
  "_blocks_class_name": "Flux2KleinBaseAutoBlocks",
  "_class_name": "Flux2KleinModularPipeline",
  "is_distilled": true
}

from_pretrained honours _blocks_class_name, so it builds the base blocks while the test classes declare Flux2KleinAutoBlocks. The two differ by exactly the guider:

Flux2KleinAutoBlocks      text2image  ['image_processor', 'scheduler', 'text_encoder', 'tokenizer', 'transformer', 'vae']
Flux2KleinBaseAutoBlocks  text2image  ['guider', ...same...]

The route reads identically by block name on both sides, which is why only the component set shows it; the classes differ at text_encoder (Flux2KleinBaseTextEncoderStep) and denoise.denoise.denoiser (Flux2KleinBaseLoopDenoiser).

Fix: set _blocks_class_name to Flux2KleinAutoBlocks, since the other two fields both say distilled — or, if the fixture is meant to be the base model, set _class_name to Flux2KleinBaseModularPipeline, is_distilled to false, and have the test classes declare Flux2KleinBaseAutoBlocks. Affects TestFlux2KleinModularPipelineFast and TestFlux2KleinImageConditionedModularPipelineFast.

3. hf-internal-testing/tiny-qwenimage-edit-modular names the base pipeline class

{
  "_blocks_class_name": "SequentialPipelineBlocks",
  "_class_name": "QwenImageModularPipeline"
}

_blocks_class_name is a saved generic sequence, so resolution falls back to _class_name's default_blocks_name, i.e. QwenImageAutoBlocks:

ValueError: Workflow 'image_conditioned' not found in QwenImageAutoBlocks.
Available workflows: ['text2image', 'image2image', 'inpainting', 'controlnet_text2image',
                     'controlnet_image2image', 'controlnet_inpainting']

Fix: record the Edit classes. hf-internal-testing/tiny-qwenimage-edit-plus-modular carries the same _class_name: QwenImageModularPipeline and is worth correcting at the same time — it does not fail today only because QwenImageEditPlusAutoBlocks declares no workflow map, so the test skips itself. Affects TestQwenImageEditModularPipelineFast.

Reproduction

On the branch of #14355:

pytest tests/modular_pipelines/ -k "test_from_pretrained_workflow or test_load_components_workflow or test_unload_components"

Before the skips: 10 failed, 79 passed.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions