fx2ait low performance

[Oldpan]

When I run AITemplate/fx2ait/fx2ait/example/02_vision_model/test_vision_model.py

the performace is strange:

BS: 1, PT Eager time per iter: 0.0008954061126708984ms, PT Eager QPS: 1116.81, FX2AIT time per iter: 0.0008138240051269531ms, FX2AIT Eager QPS: 1228.77, Speedup: 1.10, 
== Benchmark Result for: TestResNet
BS: 8, PT Eager time per iter: 0.002293975067138672ms, PT Eager QPS: 3487.40, FX2AIT time per iter: 0.002814176635742188ms, FX2AIT Eager QPS: 2842.75, Speedup: 0.82, 
== Benchmark Result for: TestResNet
BS: 16, PT Eager time per iter: 0.003973939208984375ms, PT Eager QPS: 4026.23, FX2AIT time per iter: 0.00559388671875ms, FX2AIT Eager QPS: 2860.27, Speedup: 0.71, 
== Benchmark Result for: TestResNet
BS: 32, PT Eager time per iter: 0.006892441406250001ms, PT Eager QPS: 4642.77, FX2AIT time per iter: 0.011519876708984376ms, FX2AIT Eager QPS: 2777.81, Speedup: 0.60, 
== Benchmark Result for: TestResNet
BS: 256, PT Eager time per iter: 0.0491755615234375ms, PT Eager QPS: 5205.84, FX2AIT time per iter: 0.084581982421875ms, FX2AIT Eager QPS: 3026.65, Speedup: 0.58, 
== Benchmark Result for: TestResNet
BS: 512, PT Eager time per iter: 0.098196552734375ms, PT Eager QPS: 5214.03, FX2AIT time per iter: 0.1877259375ms, FX2AIT Eager QPS: 2727.38, Speedup: 0.52, 
.
----------------------------------------------------------------------
Ran 1 test in 254.248s

OK

My env is:

• 3080
• cuda11.8

[ipiszy]

This is weird, more debugging is needed.
Probably you could try to dump some gpu traces by using Nsight Systems.