Skip to content

Conversation

@LeiWang1999
Copy link
Contributor

During ci testing with with_scaling=False, some test case may fail due to the range of uint4/int4 is too wide. This can lead to some mismatch elements when we leverage torch.testing.assert_close, even though the numbers are correct and make sense.

  File "/root/BitBLAS/testing/python/operators/test_general_matmul_bf16.py", line 193, in <module>
    matmul_torch_forward_weight_dequantize(1024, 1024, 1024, "bfloat16", "uint4", "float32",
  File "/root/BitBLAS/testing/python/operators/test_general_matmul_bf16.py", line 176, in matmul_torch_forward_weight_dequantize
    torch.testing.assert_close(permuted_inputs[-1], ref_result, rtol=1e2, atol=1e0)
  File "/opt/conda/lib/python3.10/site-packages/torch/testing/_comparison.py", line 1520, in assert_close
    raise error_metas[0].to_error(msg)
AssertionError: Tensor-likes are not close!

Mismatched elements: 2 / 1048576 (0.0%)

To allow few mismatch elements in some case, this pull request introduce our own assert_close function, which allow few tolerance for few mismatched arguments.

if not with_scaling:
    # when scaling is not enabled, we should have some mismatch due to the scaling factor
    bitblas.testing.torch_assert_close(permuted_inputs[-1], ref_result, rtol=1e2, atol=1e0)

@LeiWang1999 LeiWang1999 merged commit 6895496 into microsoft:main Nov 28, 2024
6 checks passed
@LeiWang1999 LeiWang1999 mentioned this pull request Nov 28, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant