Make Qwen 3.5 MoE exportable on 4090 gpus

[Gasoonjia]

🚀 The feature, motivation and pitch

Exporting Qwen 3.5 MoE on x090 (using 4090 as experiement) got OOM issue.

One of the core thing is aoti_compile duplicate some of the tensor in GPU for keeping their original value during the optimization.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

[Gasoonjia]

Did some hacky changes including moving the back up data onto cpu memory, using original tensor not back-uped tensor's metadata for source of truth, aggressive gpu memory clean up, etc.

Now output is expected, perf: (p = 512 and d = 128)
4090: prefill: 1,954.9 tok/s decode: 203.2 tok/s
A100: prefill: ~1600 tok/s decode: 170 tok/s

about 15% improve.

However compare with llama.cpp on 4090:
4090: prefill 7,708.55 tok/s; decode (128 tokens) 183.10 tok/s
A100: prefill 3000 tok/s, decode: 100 tok/s

140% perf improve on prefill and 80% improve on decode.

seems we are overfitting on A100 😂
Also checking if some of my hacky changing leads to the perf issue.