[inductor][cpu] [dynamic shapes][cppwrapper] performance regression

[yudongsi]

new_perf_regression 2023-09-17 compare with 2023-09-10 nightly release

Note: multi threads secnario for models above first *, single thread for models between two *

new_perf_regression

name	batch_size_new	speed_up_new	inductor_new	eager_new	compilation_latency_new	batch_size_old	speed_up_old	inductor_old	eager_old	compilation_latency_old	Ratio Speedup(New/old)	Eager Ratio(old/new)	Inductor Ratio(old/new)	Compilation_latency_Ratio(old/new)
hf_T5_large	1	0.956146	0.495255559	0.47353662171561406	329.334921	1	1.176139	0.40203819399999996	0.472852799452966	340.740606	0.81	1.0	0.81	1.03
*	*	*	*	*	*	*	*	*	*	*	*	*	*	*
*	*	*	*	*	*	*	*	*	*	*	*	*	*	*

SW info

SW	Nightly commit	Main commit
Pytorch	0de2555	d860313
Torchbench	/	ffbbebb9
torchaudio	475b6ae	ede4309
torchtext	142d029	45e4b8c
torchvision	8636bf3	4ac707a
torchdata	eb9bf61	d76d92c
dynamo_benchmarks	0200b11	/

Repro

inductor_single_run.sh bash inductor_single_run.sh multiple/single inference performance torchbench/huggingface/timm_models model float32 first dynamic cpp 0

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305

[leslie-fang-intel]

Suspect same root-cause as #109700 and #109285
@Valentine233 please help to take a look.

[Valentine233]

Test hf_T5_large with 0917 nightly: speedup=1.117x.
Script: bash inductor_single_test.sh multiple inference performance torchbench hf_T5_large float32 first dynamic cpp 0.
Could not reproduce.