TorchBench issues with optimization pass for inference

[csarofeen]

Reported by Jason Ansel

Repo:
https://github.com/jansel/torchdynamo/blob/main/torchbench.py

Repro:
./torchbench.py --no-skip -d cuda -n 200 --speedup-ts --nvfuser
Can be compared with
./torchbench.py --no-skip -d cuda -n 200 --speedup-ts

Error:

File "/fsx/users/jansel/conda/envs/torchbenchmark/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
RuntimeError: Graph::copy() encountered a use of a value input.84 not in scope. Run lint!

[jjsjann123]

Hmmmm. let me kick off a container to run through those. 🤞

[jjsjann123]

Took me a while to get something to run with my setup.

Not seeing the exact same error, but close enough. -> I'm hitting an out of bound access to hash table.. Trying my luck with a debug build now...

[jjsjann123]

hmmmm, the alias pass in our devel is still failing. But that code is actually not in upstream master yet....

The graph is very big and I don't yet have a good idea what's causing the error yet. I'm gonna pretend that doesn't exist for the time being and test the code base on upstream master instead.

[Chillee]

@jjsjann123 I could try running it through my minifier if you can dump the graph :)

[jjsjann123]

I think we are good with running the benchmark @kevinstephano Let me know if you run into more issues 🤞