Skip to content

Conversation

@LeiWang1999
Copy link
Contributor

This pull request includes several changes across multiple files to improve functionality, fix issues, and update dependencies. The most important changes include updating a submodule, modifying conditions for specific operations, and updating script behavior for model generation.

Submodule Update:

  • 3rdparty/tvm: Updated submodule commit to the latest version. ([3rdparty/tvmL1-R1](https://github.com/microsoft/BitBLAS/pull/141/files#diff-fa909c93fe94e9aa04c9e7f19e5754a2bb274678ad5c6275ee4bf54c6f9b1066L1-R1))

Code Enhancements:

  • bitblas/gpu/matmul_analysis.py: Modified the condition to check for dequantize_info to ensure it only supports float16 data type. ([bitblas/gpu/matmul_analysis.pyL626-R627](https://github.com/microsoft/BitBLAS/pull/141/files#diff-6f289ae4794dea10c7718ec09a9e100a564894c8a93c36559c9fa6d0688553e3L626-R627))
  • bitblas/ops/general_matmul/__init__.py: Added a temporary solution to legalize the dynamic symbolic, with a note to potentially remove it in the future. ([bitblas/ops/general_matmul/__init__.pyR88-R89](https://github.com/microsoft/BitBLAS/pull/141/files#diff-74fe5dd2824cb03a0fb2b0a913a2fc5caeb9c08e5368c318cd32b3af7e6f52edR88-R89))

Script Updates:

  • integration/BitNet/eval_correctness.py: Disabled flash_attention_2 and added functionality to quantize the model and compare generated text before and after quantization. ([integration/BitNet/eval_correctness.pyL75-R88](https://github.com/microsoft/BitBLAS/pull/141/files#diff-24610ce0fe8c0977dec03f95f17f709d384085cdb8ff49f86f563dba7dfa5ec4L75-R88))
  • integration/BitNet/maint/create_bitblas_ckpt.py: Disabled flash_attention_2 in the model configuration. ([integration/BitNet/maint/create_bitblas_ckpt.pyL71-R71](https://github.com/microsoft/BitBLAS/pull/141/files#diff-8b1a01ecd38f6ed3a87e4974faf672137b4a044a6137bc3ff7854a0d515689efL71-R71))
  • integration/BitNet/maint/generate_bitnet_model_bitblas_format.sh: Added a check to create the SAVED_MODEL_DIR directory if it does not exist. ([integration/BitNet/maint/generate_bitnet_model_bitblas_format.shR21-R23](https://github.com/microsoft/BitBLAS/pull/141/files#diff-d6095b66d1b0492e71833fb5a0f99593bf2261720a96ab43811a1b32f136b6c1R21-R23))

Dependency Update:

  • integration/BitNet/requirements.txt: Added transformers==4.40 to the requirements. ([integration/BitNet/requirements.txtR3](https://github.com/microsoft/BitBLAS/pull/141/files#diff-c274b4a63b1dc11eb5c71badfa85865c12befc3563dfa52d57f5c4655809c155R3))

@LeiWang1999 LeiWang1999 merged commit af697de into microsoft:main Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant