Calibration dataset sample size and its sequence length are always fixed. Why?

[Sakusakumura]

I've noticed in the implementation of AwqQuantizer that the number of samples and sequence length for the calibration dataset are fixed at 128 and 512, respectively. Is this a deliberate choice due to certain constraints or optimizations? I believe that making these values variable may have been overlooked when the change "Allow user to use custom calibration data for quantization #27" enabled the use of custom datasets.

AutoAWQ/awq/quantize/quantizer.py

Line 27 in 299c460

self.modules, self.module_kwargs, self.inps = self.init_quant()

AutoAWQ/awq/quantize/quantizer.py

Line 293 in 299c460

def init_quant(self, n_samples=128, seqlen=512):

[casper-hansen]

This is how the original AWQ was implemented. I allowed custom datasets but I can see there are some restrictions now that I should look into

[Sakusakumura]

Understood. Thank you!