Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

onnxruntime-genai based metadata and packaging #1073

Merged
merged 1 commit into from
Apr 19, 2024
Merged

Conversation

shaahji
Copy link
Contributor

@shaahji shaahji commented Apr 11, 2024

onnxruntime-genai based metadata and packaging

  • Added support for packaging models (and additional files) generated by the GenAIModelExporter. Also, updated the pass configuration to include search parameters that are forwarded to the generated genai_config file.
  • Added support for carrying "additional files" from one pass to next. These files will end up in the generated models output folder and will be packaged.
  • Two new packaging configuration options -
    • include_sample_code
    • inlcude_runtime_packages

Release Note:

Packaging config include two new options -

  • include_sample_code - whether or not to include sample code in zip archive. Defaults to true.
  • include_runtime_packages - whether or not to include runtime packages (like onnxruntime) in zip archive. Defaults to true.

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary.
  • Lint and apply fixes to your code by running lintrunner -a
  • Is this a user-facing change? If yes, give a description of this change to be included in the release notes.
  • Is this PR including examples changes? If yes, please remember to update example documentation in a follow-up PR.

(Optional) Issue link

examples/llama2/llama2_genai.json Outdated Show resolved Hide resolved
olive/engine/packaging/packaging_config.py Outdated Show resolved Hide resolved
@shaahji shaahji force-pushed the shaahji/ortgenai branch 2 times, most recently from 2d3728a to 7b161ed Compare April 12, 2024 10:29
@devang-ml devang-ml requested a review from jambayk April 12, 2024 19:10
olive/passes/olive_pass.py Outdated Show resolved Hide resolved
olive/passes/olive_pass.py Outdated Show resolved Hide resolved
olive/passes/olive_pass.py Outdated Show resolved Hide resolved
@shaahji shaahji changed the title Packaging for OrtGenAI exported models onnxruntime-genai based metadata and packaging Apr 12, 2024
test/unit_test/passes/test_pass.py Fixed Show fixed Hide fixed
test/unit_test/passes/test_pass.py Fixed Show fixed Hide fixed
test/unit_test/passes/test_pass.py Fixed Show fixed Hide fixed
olive/passes/olive_pass.py Outdated Show resolved Hide resolved
@shaahji shaahji force-pushed the shaahji/ortgenai branch 2 times, most recently from b8f9a27 to f1a30a3 Compare April 17, 2024 19:37
olive/passes/olive_pass.py Outdated Show resolved Hide resolved
* Added support for packaging models (and additional files) generated by
  the GenAIModelExporter. Also, updated the pass configuration to include
  search parameters that are forwarded to the generated genai_config file.
* Added support for carrying "additional files" from one pass to next. These
  files will end up in the generated models output folder and will be
  packaged.
* Two new packaging configuration options -
  ** include_sample_code
  ** inlcude_runtime_packages
Copy link
Contributor

@jambayk jambayk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good to me (with the note that we will revisit the folder structure later). Please feel free to merge if Mike has no other comments.

@shaahji shaahji merged commit 04b4b2c into main Apr 19, 2024
35 checks passed
@shaahji shaahji deleted the shaahji/ortgenai branch April 19, 2024 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants