Pass for Nvidia ModelOpt graph surgery framework

[hthadicherla]

Describe your changes

Add NVModelOptGraphSurgery pass to integrate NVIDIA ModelOpt graph surgeries into Olive. Supports all existing surgeries in ModelOpt like GQA fusion, DQ-Transpose and all future surgeries that will be added in ModelOpt

Changes:

• New pass: olive/passes/onnx/nvmo_graph_surgery.py
• Pass registration in olive_config.json
• Unit tests: test/passes/onnx/test_nvmo_graph_surgery.py
• Documentation in pass.rst and onnx-transformations.md

Usage

Example

{
    "type": "NVModelOptGraphSurgery",
    "config": {
        "surgery_type": "replace-gqa" # Surgery-key in Modelopt,
        "surgery_params": {
             # Surgery specific parameters of that particular surgery, example below
            "hf_model_id": "meta-llama/Llama-2-7b-hf",
            "io_dtype": "float16"
        }
    }
}

Checklist before requesting a review

• Add unit tests for this change.
• Make sure all tests can pass.
• Update documents if necessary.
• Lint and apply fixes to your code by running lintrunner -a
• Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

Release note: Added NVModelOptGraphSurgery pass for running NVIDIA ModelOpt graph surgeries on ONNX models.

[hthadicherla]

@jambayk Can you review this PR? This is graph surgery pass for NVIDIA stack. The implementation of the surgeries will be in modelopt and essentially we are calling them through the olive pass.

Pipeline is failing because graph_surgeon dependency is using onnx.helper.bfloat.float32_to_bfloat16 which has been deprecated in onnx since version 1.18 and removed in 1.20.

[hthadicherla]

This is a conflict between onnx_graphsurgeon and onnx and needs to be resolved within onnx_graphsurgeon. I have raised a PR in ModelOpt for a workaround temporarily. NVIDIA/Model-Optimizer#1204.

After it is merged, this failure should resolve itself

[hthadicherla]

The PR has been merged. Can you re review it again @jambayk ?

tests need to be skipped until new release of modelopt due to incompatibility with latest onnx versions

[xiaoyu-work]

@hthadicherla can you update this PR? we plan to release new Olive version this Friday and this PR will be in the new release

[hthadicherla]

@xiaoyu-work @jambayk I have added skip to the test. Can you reapprove it now ?