Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Mistral fp16 config #980

Merged
merged 2 commits into from
Mar 6, 2024
Merged

Add Mistral fp16 config #980

merged 2 commits into from
Mar 6, 2024

Conversation

apsonawane
Copy link
Contributor

@apsonawane apsonawane commented Mar 1, 2024

Describe your changes

Add float16 configuration to Mistral.

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary.
  • Lint and apply fixes to your code by running lintrunner -a
  • Is this a user-facing change? If yes, give a description of this change to be included in the release notes.
  • Is this PR including examples changes? If yes, please remember to update example documentation in a follow-up PR.

(Optional) Issue link

@apsonawane apsonawane requested a review from jambayk March 1, 2024 18:19
jambayk
jambayk previously approved these changes Mar 1, 2024
Copy link
Contributor

@jambayk jambayk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this example!

Copy link
Contributor

@devang-ml devang-ml left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add e2e example that uses the output model produced using mistral_fp16_optimize.json

Thanks!

devang-ml
devang-ml previously approved these changes Mar 6, 2024
examples/mistral/mistral.py Outdated Show resolved Hide resolved
examples/mistral/mistral.py Outdated Show resolved Hide resolved
@apsonawane apsonawane force-pushed the asonawane/mistral-float16 branch 2 times, most recently from a7cb54e to d9e7c51 Compare March 6, 2024 20:39
Copy link
Contributor

@jambayk jambayk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@apsonawane apsonawane merged commit eb94a1d into main Mar 6, 2024
33 checks passed
@apsonawane apsonawane deleted the asonawane/mistral-float16 branch March 6, 2024 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants