Skip to content

Handle non-contiguous inputs in CPU blockwise quantize/dequantize#1996

Merged
matthewdouglas merged 1 commit into
bitsandbytes-foundation:mainfrom
egeozkoc:feature/cpu-noncontiguous-blockwise
Jul 9, 2026
Merged

Handle non-contiguous inputs in CPU blockwise quantize/dequantize#1996
matthewdouglas merged 1 commit into
bitsandbytes-foundation:mainfrom
egeozkoc:feature/cpu-noncontiguous-blockwise

Conversation

@egeozkoc

@egeozkoc egeozkoc commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #1995. The CPU quantize_blockwise / dequantize_blockwise kernels pass raw pointers (get_ptr) to the native library, which assumes contiguous memory. Non-contiguous inputs (e.g. strided slices) were read in physical order and produced silently incorrect results — the same bug class as #1342 / #1690, which #1859 fixed for CUDA only.

Per the maintainer's guidance on #1995, this applies the same approach as the CUDA backend: convert the input with A = A.contiguous() at kernel entry.

Changes

  • bitsandbytes/backends/cpu/ops.py: add A = A.contiguous() at the start of the CPU quantize_blockwise and dequantize_blockwise kernels (in the latter, before torch.empty_like so the output is contiguous as well).
  • tests/test_ops.py: enable the previously CPU-skipped TestNonContiguousInputs blockwise tests.

Verification

Run locally on the CPU backend (with the compiled native library):

  • The two enabled blockwise non-contiguous tests now pass — 18 cases (fp16/bf16/fp32 × blocksize 64/128/256); these were previously silently incorrect.
  • All CPU blockwise tests pass with no regression on the contiguous path (.contiguous() is a no-op there): 289 passed, 0 failed.

Minimal repro before/after:

import torch
import bitsandbytes.functional as F

A = torch.randn(64, 128)
A_nc = A[::2, :]                       # non-contiguous
qn, sn = F.quantize_blockwise(A_nc, blocksize=64)
qc, sc = F.quantize_blockwise(A_nc.contiguous(), blocksize=64)
print(torch.equal(qn, qc))             # before: False, after: True

🤖 Generated with Claude Code

The CPU quantize_blockwise / dequantize_blockwise kernels pass raw
pointers (get_ptr) to the native library, which assumes contiguous
memory. Non-contiguous inputs (e.g. strided slices) were read in
physical order and produced silently incorrect results.

Add `A = A.contiguous()` at kernel entry, matching the CUDA backend
fix (bitsandbytes-foundation#1859). Enable the previously CPU-skipped TestNonContiguousInputs
blockwise regression tests, which now pass (18 cases: fp16/bf16/fp32 x
blocksize 64/128/256).

Fixes bitsandbytes-foundation#1995

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@egeozkoc egeozkoc force-pushed the feature/cpu-noncontiguous-blockwise branch from f056fd2 to 317a6d5 Compare July 9, 2026 16:16
@matthewdouglas matthewdouglas added this to the v0.50.0 milestone Jul 9, 2026
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@matthewdouglas matthewdouglas merged commit 771db27 into bitsandbytes-foundation:main Jul 9, 2026
84 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

quantize_blockwise / dequantize_blockwise produce incorrect results for non-contiguous CPU inputs (CPU follow-up to #1690)

2 participants