Dynamo does a lot of unnecessary input cloning for real_value_cache, even if cache is not used

[ezyang]

🐛 Describe the bug

            proxy.tracer.real_value_cache[proxy.node] = _clone_input(example_value)

in torch/_dynamo/variables/builder.py is ALWAYS run, even if the real_value_cache is not used at all

Error logs

No response

Minified repro

No response

[voznesenskym]

Yes we talked about making this whole cache a lot more lazy.