Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling of extra_args for ModelBuilder pass #1118

Merged
merged 1 commit into from
Apr 27, 2024
Merged

Conversation

shaahji
Copy link
Contributor

@shaahji shaahji commented Apr 27, 2024

Fix handling of extra_args for ModelBuilder pass

GenAI model builder's extra_args works based on existence of a key rather then the value of the key itself. So, Olive cannot pass a key with a value of true/false. The key has to explicitly be added/removed based on the value of the key itself in the user config.

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary.
  • Lint and apply fixes to your code by running lintrunner -a
  • Is this a user-facing change? If yes, give a description of this change to be included in the release notes.
  • Is this PR including examples changes? If yes, please remember to update example documentation in a follow-up PR.

(Optional) Issue link

GenAI model builder's extra_args works based on existence of a key
rather then the value of the key itself. So, Olive cannot pass a key
with a value of true/false. The key has to explicitly be added/removed
based on the value of the key itself in the user config.
if config["exclude_lm_head"]:
extra_args["exclude_lm_head"] = config["exclude_lm_head"]

if config["enable_cuda_graph"] is not None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this condition is not needed since the value is always a bool and this condition is always true.

@shaahji shaahji merged commit b9deebb into main Apr 27, 2024
35 checks passed
@shaahji shaahji deleted the shaahji/fixes branch April 27, 2024 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants