[QST] NF4 Offset

[jeromeku]

Feature request

How is the offset derived in the NormalFloat4 type?

[TimDettmers]

Closing as duplicate of #938. The offset in create_normal_map determines the outermost quantile boundary — scipy.stats.norm.ppf(offset) gives the largest quantization bin edge. It was empirically optimized for minimal quantization error on normally-distributed weights. See the closing comment on #938 for more detail.

[TimDettmers]

The NF4 offset parameter is now documented in commit d7c179c — see the docstring on create_normal_map in bitsandbytes/functional.py.