Add dumping IR facility to the benchmarks

[penguinwu]

Add the script to dump graph IRs for all the benchmarks

[bertmaher]

Looks good, but do consider my suggestion about warming up rather than disabling the profiling executor.

[bertmaher]

I'm a little wary of this because the non-profiling executor isn't a default mode anymore. I don't know how different the graphs would look but I don't know that it's a good idea to collect stats from a different mode than we run with.

To get rid of the profile nodes, I'd suggest adding a "warmup" before dumping stats:

for _ in range(torch._C._jit_num_profiled_runs()):
  model(*example_inputs)

[penguinwu]

Good suggestion. I made the default collection to use the default setting (i.e., profile executor enabled and warmed up). Bu I added additional options to dump graphs as inlined_graph and also dump graph w/o profiling executor. Sometimes we need to see the graph before profiling executor to check if TS IR level optimizations can handle them (e.g., for symbolic shape analysis or alias analysis).

usage: collect_graph_ir.py [-h] [--benchmark BENCHMARK] [--no_profiling] [--inlined_graph]

dump last_executed graph for all benchmarks with JIT implementation

optional arguments:
  -h, --help            show this help message and exit
  --benchmark BENCHMARK, -b BENCHMARK
                        dump graph for <BENCHMARK>
  --no_profiling        dump last_executed graphs w/o profiling executor
  --inlined_graph       dump graphs dumped by module.inlined_graph

[wconstab]

lgtm other than comments from Bert. Thanks for adding this!