Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Additional ModelBuilder pass options #1112

Merged
merged 1 commit into from
Apr 25, 2024
Merged

Additional ModelBuilder pass options #1112

merged 1 commit into from
Apr 25, 2024

Conversation

shaahji
Copy link
Contributor

@shaahji shaahji commented Apr 25, 2024

Additional ModelBuilder pass options

Adding the following new options for model builder pass

  • int4_block_size
  • int4_accuracy_level
  • exclude_embeds
  • exclude_lm_head
  • enable_cuda_graph

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary.
  • Lint and apply fixes to your code by running lintrunner -a
  • Is this a user-facing change? If yes, give a description of this change to be included in the release notes.
  • Is this PR including examples changes? If yes, please remember to update example documentation in a follow-up PR.

(Optional) Issue link

@shaahji shaahji force-pushed the shaahji/task29744 branch 2 times, most recently from aa53f1e to 995eced Compare April 25, 2024 10:15
@shaahji shaahji marked this pull request as ready for review April 25, 2024 10:15
guotuofeng
guotuofeng reviewed Apr 25, 2024
View reviewed changes
olive/passes/onnx/model_builder.py
bf16 = 3
int8 = 4

def __str__(self) -> str:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need the __str__ method since it is int enum?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prints a simpler representation rather than the full name.

@shaahji
Additional ModelBuilder pass options
7618b22 
Adding the following new options for model builder pass
+ int4_block_size
+ int4_accuracy_level
+ exclude_embeds
+ exclude_lm_head
+ enable_cuda_graph
@jambayk jambayk merged commit 5e9e762 into main Apr 25, 2024
35 checks passed
@jambayk jambayk deleted the shaahji/task29744 branch April 25, 2024 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@guotuofeng guotuofeng guotuofeng approved these changes

@jambayk jambayk jambayk approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@shaahji @guotuofeng @jambayk