Export example with dynamo=True and onnxscript fusion

[justinchuby]

Describe your changes

Checklist before requesting a review

• Add unit tests for this change.
• Make sure all tests can pass.
• Update documents if necessary.
• Lint and apply fixes to your code by running lintrunner -a
• Is this a user-facing change? If yes, give a description of this change to be included in the release notes.
• Is this PR including examples changes? If yes, please remember to update example documentation in a follow-up PR.

(Optional) Issue link

[justinchuby]

olive run --config examples/llama3_2_export/llama3_2.yaml    (onnx) 
[2025-04-17 18:08:00,565] [INFO] [run.py:142:run_engine] Running workflow default_workflow
[2025-04-17 18:08:00,583] [INFO] [cache.py:138:__init__] Using cache directory: /home/justinchu/dev/Olive/cache/default_workflow
[2025-04-17 18:08:00,585] [INFO] [accelerator_creator.py:40:normalize] No accelerators specified. Defaulting to cpu.
[2025-04-17 18:08:00,585] [INFO] [accelerator_creator.py:217:create_accelerators] Running workflow on accelerator specs: cpu-cpu
[2025-04-17 18:08:00,585] [INFO] [engine.py:223:run] Running Olive on accelerator: cpu-cpu
[2025-04-17 18:08:00,585] [INFO] [engine.py:864:_create_system] Creating target system ...
[2025-04-17 18:08:00,586] [INFO] [engine.py:867:_create_system] Target system created in 0.000566 seconds
[2025-04-17 18:08:00,586] [INFO] [engine.py:879:_create_system] Creating host system ...
[2025-04-17 18:08:00,586] [INFO] [engine.py:882:_create_system] Host system created in 0.000023 seconds
[2025-04-17 18:08:00,856] [INFO] [engine.py:683:_run_pass] Running pass convert:onnxconversion
/home/justinchu/dev/onnxscript/onnxscript/converter.py:816: FutureWarning: 'onnxscript.values.Op.param_schemas' is deprecated in version 0.1 and will be removed in the future. Please use '.op_signature' instead.
  param_schemas = callee.param_schemas()
/home/justinchu/dev/onnxscript/onnxscript/converter.py:816: FutureWarning: 'onnxscript.values.OnnxFunction.param_schemas' is deprecated in version 0.1 and will be removed in the future. Please use '.op_signature' instead.
  param_schemas = callee.param_schemas()
[torch.onnx] Obtain model graph for `LlamaForCausalLM([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `LlamaForCausalLM([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decomposition...
[torch.onnx] Run decomposition... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
/home/justinchu/anaconda3/envs/onnx/lib/python3.13/site-packages/torch/onnx/_internal/exporter/_dynamic_shapes.py:264: UserWarning: # The axis name: batch_size will not be used, since it shares the same shape constraints with another axis: batch_size.
  warnings.warn(
/home/justinchu/anaconda3/envs/onnx/lib/python3.13/site-packages/torch/onnx/_internal/exporter/_dynamic_shapes.py:264: UserWarning: # The axis name: sequence_length will not be used, since it shares the same shape constraints with another axis: sequence_length.
  warnings.warn(
/home/justinchu/anaconda3/envs/onnx/lib/python3.13/site-packages/torch/onnx/_internal/exporter/_dynamic_shapes.py:264: UserWarning: # The axis name: past_sequence_length will not be used, since it shares the same shape constraints with another axis: past_sequence_length.
  warnings.warn(
Applied 399 of general pattern rewrite rules.
[2025-04-17 18:08:52,885] [INFO] [engine.py:757:_run_pass] Pass convert:onnxconversion finished in 52.028721 seconds
[2025-04-17 18:08:52,887] [INFO] [engine.py:683:_run_pass] Running pass fusion:onnxscriptfusion
Applied 33 of general pattern rewrite rules.
[2025-04-17 18:09:33,519] [INFO] [engine.py:757:_run_pass] Pass fusion:onnxscriptfusion finished in 40.631930 seconds
[2025-04-17 18:09:33,522] [INFO] [engine.py:322:run_accelerator] Save footprint to /home/justinchu/dev/Olive/llama3_output_model/footprints.json.
[2025-04-17 18:09:33,523] [INFO] [engine.py:241:run] Run history for cpu-cpu:
[2025-04-17 18:09:33,530] [INFO] [engine.py:497:dump_run_history] run history:
+------------+-------------------+------------------+----------------+-----------+
| model_id   | parent_model_id   | from_pass        |   duration_sec | metrics   |
+============+===================+==================+================+===========+
| 2c124960   |                   |                  |                |           |
+------------+-------------------+------------------+----------------+-----------+
| c303e660   | 2c124960          | onnxconversion   |        52.0287 |           |
+------------+-------------------+------------------+----------------+-----------+
| 592d7abb   | c303e660          | onnxscriptfusion |        40.6319 |           |
+------------+-------------------+------------------+----------------+-----------+
[2025-04-17 18:09:33,530] [INFO] [cache.py:195:load_model] Loading model 592d7abb from cache.
[2025-04-17 18:09:36,066] [INFO] [engine.py:266:run] Saved output model to /home/justinchu/dev/Olive/llama3_output_model

[justinchuby]

Remove cache?

[devang-ml]

yes. There is a default location of cache. Cache is useful while working on a model. For example, it'd help if the user is rerunning the flow by just changing parameters of fusion pass then Olive will skip the conversion step and pick it up from the cache.

[devang-ml]

Do you see GQA fusion happening here?

[justinchuby]

Do you see GQA fusion happening here?

I didn't see it. I am checking with @shubhambhokare1 to see what's going on