Skip to content

Conversation

@Qubitium
Copy link
Collaborator

@Qubitium Qubitium commented Sep 30, 2025

zoom zoom

Signed-off-by: Qubitium <Qubitium@modelcloud.ai>
Signed-off-by: Qubitium <Qubitium@modelcloud.ai>
Signed-off-by: Qubitium <Qubitium@modelcloud.ai>
Signed-off-by: Qubitium <Qubitium@modelcloud.ai>
@Qubitium Qubitium marked this pull request as ready for review September 30, 2025 09:52
@Qubitium
Copy link
Collaborator Author

@codex Check this PR for bugs

@Qubitium Qubitium merged commit 10f3d1f into main Sep 30, 2025
5 checks passed
@Qubitium Qubitium deleted the meta-dedup branch September 30, 2025 09:54
@avtc
Copy link
Contributor

avtc commented Oct 1, 2025

@Qubitium Please check error during save, can be related to this PR:
main branch hash: 3da0344

INFO  Format: Converting GPTQ v2 to v1                                                                                     
Traceback (most recent call last):
  File "/home/ubuntu/Documents/Quantize/quantize-glm4.5-air-gptqmodel-clean.py", line 63, in <module>
    model.save(OUTPUT_DIR)
    ~~~~~~~~~~^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/models/base.py", line 832, in save
    self.save_quantized(
    ~~~~~~~~~~~~~~~~~~~^
        save_dir=save_dir,
        ^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        meta_quantizer=meta_quantizer,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        eora_path=eora_path)
        ^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/models/writer.py", line 231, in save_quantized
    model = convert_gptq_v2_to_v1_format(
        model, quantize_config=quantize_config, qlinear_kernel=self.qlinear_kernel
    )
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/utils/model.py", line 669, in convert_gptq_v2_to_v1_format
    convert_gptq_v2_to_v1_format_module(module=submodule, quantize_config=quantize_config)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/utils/model.py", line 635, in convert_gptq_v2_to_v1_format_module
    module.qzeros.data[:, range(0, module.qzeros.data.shape[1], 3)] -= (
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Inplace update to inference tensor outside InferenceMode is not allowed.You can make a clone to get a normal tensor before doing inplace update.See https://github.com/pytorch/rfcs/pull/17 for more details.

@Qubitium
Copy link
Collaborator Author

Qubitium commented Oct 1, 2025

@Qubitium Please check error during save, can be related to this PR: main branch hash: 3da0344

INFO  Format: Converting GPTQ v2 to v1                                                                                     
Traceback (most recent call last):
  File "/home/ubuntu/Documents/Quantize/quantize-glm4.5-air-gptqmodel-clean.py", line 63, in <module>
    model.save(OUTPUT_DIR)
    ~~~~~~~~~~^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/models/base.py", line 832, in save
    self.save_quantized(
    ~~~~~~~~~~~~~~~~~~~^
        save_dir=save_dir,
        ^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        meta_quantizer=meta_quantizer,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        eora_path=eora_path)
        ^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/models/writer.py", line 231, in save_quantized
    model = convert_gptq_v2_to_v1_format(
        model, quantize_config=quantize_config, qlinear_kernel=self.qlinear_kernel
    )
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/utils/model.py", line 669, in convert_gptq_v2_to_v1_format
    convert_gptq_v2_to_v1_format_module(module=submodule, quantize_config=quantize_config)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/utils/model.py", line 635, in convert_gptq_v2_to_v1_format_module
    module.qzeros.data[:, range(0, module.qzeros.data.shape[1], 3)] -= (
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Inplace update to inference tensor outside InferenceMode is not allowed.You can make a clone to get a normal tensor before doing inplace update.See https://github.com/pytorch/rfcs/pull/17 for more details.

Ok. this look to be an easy fix. Looks unrelated to the PR but new code caused different timing so you are hitting more/different thread state bugs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants