Configuring group_size > 128

[hzfantasy]

To meet the requirements of the hardware, I need to quantize with group_size >= 512. Is it possible to do this in GPTQModel?

[jeyapradhaj]

Group size is configurable in 'QuantizeConfig'. For example,
quant_config = QuantizeConfig(bits=4, group_size=256)