Skip to content

[inductor][cpu]fp32 inductor_max_autotune dynamic shape multiple thread accuracy & performance TypeError in 2024-12-09 nightly releaseΒ #143102

@zxd1997066

Description

@zxd1997066

πŸ› Describe the bug

fp32 inductor_max_autotune dynamic shape multiple thread failures

suite name thread accuracy perf reason(reference only)
huggingface AllenaiLongformerBase multiple √ X AllenaiLongformerBase, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
torchbench BERT_pytorch multiple X X
huggingface BlenderbotForCausalLM multiple pass_due_to_skip X BlenderbotForCausalLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface BlenderbotSmallForCausalLM multiple √ X BlenderbotSmallForCausalLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface BlenderbotSmallForConditionalGeneration multiple √ X BlenderbotSmallForConditionalGeneration, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface DebertaForMaskedLM multiple √ X DebertaForMaskedLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface DebertaForQuestionAnswering multiple √ X DebertaForQuestionAnswering, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface DebertaV2ForMaskedLM multiple pass_due_to_skip X DebertaV2ForMaskedLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface DistilBertForMaskedLM multiple √ X DistilBertForMaskedLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface DistilBertForQuestionAnswering multiple √ X DistilBertForQuestionAnswering, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface M2M100ForConditionalGeneration multiple √ X M2M100ForConditionalGeneration, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface MBartForCausalLM multiple √ X MBartForCausalLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface MBartForConditionalGeneration multiple √ X MBartForConditionalGeneration, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface MT5ForConditionalGeneration multiple √ X MT5ForConditionalGeneration, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface OPTForCausalLM multiple √ X OPTForCausalLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface PLBartForCausalLM multiple √ X PLBartForCausalLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface PLBartForConditionalGeneration multiple √ X PLBartForConditionalGeneration, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface PegasusForCausalLM multiple √ X PegasusForCausalLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface PegasusForConditionalGeneration multiple √ X PegasusForConditionalGeneration, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface Speech2Text2ForCausalLM multiple √ X Speech2Text2ForCausalLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface T5ForConditionalGeneration multiple √ X T5ForConditionalGeneration, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface T5Small multiple √ X T5Small, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface TrOCRForCausalLM multiple √ X TrOCRForCausalLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface XGLMForCausalLM multiple √ X XGLMForCausalLM, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
huggingface XLNetLMHeadModel multiple √ X XLNetLMHeadModel, LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints but found element of type Mul at pos 0
timm_models botnet26t_256 multiple X X
timm_models cait_m36_384 multiple X X
timm_models coat_lite_mini multiple X X
timm_models convit_base multiple X X
timm_models eca_botnext26ts_256 multiple X X
timm_models eca_halonext26ts multiple X X
torchbench hf_DistilBert multiple X √
torchbench hf_T5 multiple X √
torchbench hf_T5_base multiple X √ hf_T5_base, TIMEOUT
timm_models jx_nest_base multiple X X
timm_models levit_128 multiple X X
torchbench moondream multiple X √
timm_models sebotnet33ts_256 multiple X X
timm_models swin_base_patch4_window7_224 multiple X X
timm_models tnt_s_patch16_224 multiple X X
timm_models visformer_small multiple X X
timm_models volo_d1_224 multiple X X
timm_models xcit_large_24_p8_224 multiple X X
  File "/workspace/pytorch/torch/_inductor/select_algorithm.py", line 1701, in benchmark_in_current_process
    raise e from None
  File "/workspace/pytorch/torch/_inductor/select_algorithm.py", line 1665, in benchmark_in_current_process
    timing = benchmark_choice_in_current_process(choice, inputs)
  File "/workspace/pytorch/torch/_inductor/select_algorithm.py", line 1651, in benchmark_choice_in_current_process
    result = choice.benchmark(*inpts, out=output)
  File "/workspace/pytorch/torch/_inductor/select_algorithm.py", line 1225, in benchmark
    new_args, new_out = self._preprocessor(args, out)
  File "/workspace/pytorch/torch/_inductor/codegen/cpp_gemm_template.py", line 900, in preprocessor
    new_inputs, new_layout = normalize_shapes(
  File "/workspace/pytorch/torch/_inductor/codegen/cpp_gemm_template.py", line 861, in normalize_shapes
    new_inputs[1] = new_inputs[1].as_strided(
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
LoweringException: TypeError: as_strided(): argument 'size' (position 1) must be tuple of ints, but found element of type Mul at pos 0

Versions

SW info

name target_branch target_commit refer_branch refer_commit
torchbench main 766a5e3a main 766a5e3a
torch main 2682e5e main e29dabb
torchvision main 0.19.0a0+d23a6e1 main 0.19.0a0+d23a6e1
torchtext main 0.16.0a0+b0ebddc main 0.16.0a0+b0ebddc
torchaudio main 2.5.0a0+332760d main 2.5.0a0+332760d
torchdata main 0.7.0a0+11bb5b8 main 0.7.0a0+11bb5b8
dynamo_benchmarks main nightly main nightly

Repro:
inductor_single_run.sh
bash inductor_single_run.sh multiple inference accuracy timm_models cait_m36_384 float32 first dynamic default 0 inductor_max_autotune
Suspected guilty commit: 20f24e3
timm_models-eca_botnext26ts_256-inference-float32-dynamic-default-multiple-accuracy-crash_guilty_commit.log
cc @ezyang @gchanan @zou3519 @kadeng @msaroufim @chauhang @penguinwu @chuanqi129

Metadata

Metadata

Assignees

Labels

high priorityoncall: cpu inductorCPU Inductor issues for Intel team to triageoncall: pt2triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions