Skip to content

[Dev] Support Numeric Precision BFloat16 as activation type #148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 212 commits into from
Aug 23, 2024
Merged

[Dev] Support Numeric Precision BFloat16 as activation type #148

merged 212 commits into from
Aug 23, 2024

Conversation

@LeiWang1999
Copy link
Contributor

@LeiWang1999 LeiWang1999 commented Aug 23, 2024 •
edited
Loading

This pull request includes a variety of changes to update dependencies, add support for bfloat16 data type, and improve functionality in several modules. Below are the most important changes grouped by theme:

Dependency Updates:

  • Updated the tvm submodule to a new commit (3c6317a1ea614b7277ffe0b4ede18b4652afad1c).
  • Incremented the version number to 0.0.1.dev14 in VERSION and bitblas/__init__.py [1] [2].

Support for bfloat16 Data Type:

  • Added bfloat16 support in various functions and modules, including bitblas/base/utils.py, bitblas/gpu/gemv_dequantize.py, and bitblas/ops/general_matmul/__init__.py [1] [2] [3].
  • Modified type mappings and assertions to include bfloat16 in bitblas/builder/wrapper/tir.py, bitblas/gpu/matmul_analysis.py, and bitblas/wrapper/general.py [1] [2] [3].

Future TODO Items

  • Update Supported Matrix in README.md

LeiWang1999 added 30 commits July 5, 2024 08:54
@LeiWang1999
Refactor BatchMatMulEmitter and BatchMatMulSelector for improved read…
d8884e6 
…ability and maintainability
@LeiWang1999
Refactor import statements for improved readability and maintainability
fc84173
@LeiWang1999
Refactor import statements for improved readability and maintainability
02f64de
@LeiWang1999
disable failure email for ci
397eee6
@LeiWang1999
remove email notifications.
20f6ad1
@LeiWang1999
move relax pass from testing to mlc_llm
b93c394
@LeiWang1999
Merge branch 'main' of https://github.com/Microsoft/BitBLAS into main
ba6a6df
@LeiWang1999
Refactor scripts with se check_eual_ref_scripts_with_emitter function
257693a
@LeiWang1999
Lint Fix
9bb7f49
@LeiWang1999
Merge branch 'main' of https://github.com/Microsoft/BitBLAS into main
39e7614
@LeiWang1999
Refactor scripts with se check_eual_ref_scripts_with_emitter function
93eb5a5
@LeiWang1999
bug fix in test
aa66a90
@LeiWang1999
Merge branch 'main' of https://github.com/Microsoft/BitBLAS into dev
ae14a53
@LeiWang1999
lint fix.
79b08e4
@LeiWang1999
test cuda i4 kernel
86fd036
@LeiWang1999
Refactor copyright notice in i4matmul.hpp
6b73a21
@LeiWang1999
Merge branch 'main' of https://github.com/Microsoft/BitBLAS into dev
0ba90c1
@LeiWang1999
Refactor BitBLASLinear test module for improved readability and maint…
086d208 
…ainability
@LeiWang1999
refactor test as version below python 3.9 cannot handle int32 overflow.
47a3abd
@LeiWang1999
format lint for test
024b247
@LeiWang1999
Refactor test_int4b_fp16_convert.py for improved readability and main…
bfedeaa 
…tainability
@LeiWang1999
remove unused design file
e672a23
@LeiWang1999
move tile device from package to base
21e5430
@LeiWang1999
dummy impl for codegen
fd11940
@LeiWang1999
Refactor file structure for ladder_permutate module
9ccfa85
@LeiWang1999
Refactor backend class and fix typos in comments
7c7d73e
@LeiWang1999
Deep refactor Lib related code.
47d5fc5
@LeiWang1999
remove ci pull.
53dd0dd
@LeiWang1999
LintFix
d58ac43
@LeiWang1999
refactor builder for whl build
37cb07c
@LeiWang1999
Merge branch 'main' of https://github.com/Microsoft/BitBLAS into upda…
9cde970 
…te_transform
@LeiWang1999
bugfix for bitnet
1c89c60
@LeiWang1999
annotatte todo.
cfda648
@LeiWang1999
lint fix
83b99c4
@LeiWang1999
regist fast_decode for int8xint4
0b6b4a3
@LeiWang1999
Refactor CUDA code to use sm architecture instead of compute architec…
e93d1de 
…ture
@LeiWang1999
compress qkv and gate up for bitnet
84b8267
@LeiWang1999
improve elementwise schedule
c21874c
@LeiWang1999
Refactor BitNet model checkpoint generation scripts
e9b0769
@LeiWang1999
cross thread reduce for tl
8905a57
@LeiWang1999
Merge branch 'main' of https://github.com/Microsoft/BitBLAS into tl_tmac
95c93b7
@LeiWang1999
fix scale only lop3 tensorize instructions.
67ad761
@LeiWang1999
bug fix for scale only case
374065e
@LeiWang1999
fix scale for warp memory dequantize
b83e8fb
@LeiWang1999
lint fix
fc63748
@LeiWang1999
bug fox
0ae3953
@LeiWang1999
format
cdd2244
@LeiWang1999
fix repack from gptqv2
e72b6d0
@LeiWang1999
chore: Enable large files for Hugging Face models
89e0508
@LeiWang1999
Merge branch 'main' of https://github.com/Microsoft/BitBLAS into gptqv2
87dfc18
@LeiWang1999
bump version to dev14
6087ce0
@LeiWang1999
BF 16 Update
b7a40bc
@LeiWang1999
lint fix
cb281cd
@LeiWang1999
chore: Update BitBLAS benchmark scripts and fix typos
760d715
@LeiWang1999
chore: Add gptqmodel to test requirements
2c94c82
@LeiWang1999 LeiWang1999 changed the title [Dev] Support Numeric Precision BFloat16 as activation type. [Dev] Support Numeric Precision BFloat16 as activation type Aug 23, 2024
@LeiWang1999 LeiWang1999 merged commit 673290b into microsoft:main Aug 23, 2024
@LeiWang1999 LeiWang1999 mentioned this pull request Aug 23, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@LeiWang1999