Skip to content
This repository has been archived by the owner on Aug 30, 2024. It is now read-only.

[BesTLA] Support fp16 for compute_dtype and scale_dtype #292

Merged
merged 19 commits into from
Jun 21, 2024
Merged

[BesTLA] Support fp16 for compute_dtype and scale_dtype #292

merged 19 commits into from
Jun 21, 2024

Conversation

luoyu-intel
Copy link
Contributor

@luoyu-intel luoyu-intel commented Jun 13, 2024 •
edited
Loading

Type of Change

  1. support scale_dtype=fp16
  2. support compute_dtype=fp16, only available for GNR with AMX_FP16
  3. split kernels into multiple ISA files
  4. move intrinsic codes from mha to bestla kernel
  5. change the default scale_dtype to fp16, default compute_dtype to int8

github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 13, 2024
View reviewed changes
bestla/bestla/bestla_prologue_b.h Fixed Show fixed Hide fixed
bestla/bestla/bestla_prologue_b.h Fixed Show fixed Hide fixed
bestla/bestla/bestla_prologue_b.h Fixed Show fixed Hide fixed
zhewang1-intc
zhewang1-intc reviewed Jun 21, 2024
View reviewed changes
Copy link
Contributor

@zhewang1-intc zhewang1-intc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what' meaning of move intrinsic code from mha to bestla kernel? I didn't notice any mha related ut

@zhewang1-intc
Copy link
Contributor

will this pr support gemv with bf16 activation?

@luoyu-intel
Copy link
Contributor Author

what' meaning of move intrinsic code from mha to bestla kernel? I didn't notice any mha related ut

Use the unified ISA control

@luoyu-intel
Copy link
Contributor Author

will this pr support gemv with bf16 activation?

Not in this pr.

@luoyu-intel luoyu-intel merged commit 9652017 into main Jun 21, 2024
15 of 16 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@zhewang1-intc zhewang1-intc zhewang1-intc approved these changes

@airMeng airMeng airMeng approved these changes

Assignees
No one assigned
Labels
BesTLA ready to merge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@luoyu-intel @zhewang1-intc @airMeng