[BUG] Vram on cuda:0 usage vs 4.2.5

[avtc]

Trying to quantize with gptqmodel commit hash d8f3c78
with mock_quantization=False, and got an error on first layer with experts (layer 1) (GLM-4.5-Air):

Quantizing mlp.experts.32.gate_proj in layer  [1 of 45] ████-------------------------------------------------------------------------------------------------| 0:13:41 / 5:14:43 [2/46] 4.3%Traceback (most recent call last):
  File "/home/ubuntu/Documents/Quantize/quantize-glm4.5-Air-gptqmodel-moe-prune-smart-4.py", line 489, in <module>
    model.quantize(
    ~~~~~~~~~~~~~~^
        calibration_dataset,
        ^^^^^^^^^^^^^^^^^^^^
        batch_size=BATCH_SIZE,
        ^^^^^^^^^^^^^^^^^^^^^^
        )
        ^
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/models/base.py", line 717, in quantize
    return module_looper.loop(
           ~~~~~~~~~~~~~~~~~~^
        backend=backend,
        ^^^^^^^^^^^^^^^^
        fail_safe=self.quantize_config.fail_safe,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/module_looper.py", line 850, in loop
    name, m = fut.result()
              ~~~~~~~~~~^^
  File "/home/ubuntu/.pyenv/versions/3.13.7t/lib/python3.13t/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/ubuntu/.pyenv/versions/3.13.7t/lib/python3.13t/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/utils/threadx.py", line 360, in _run
    result = fn(*args, **kwargs)
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/module_looper.py", line 842, in _process_on_worker
    proc.process(module=nm)
    ~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/gptq_processor.py", line 123, in process
    wq, q_scales, q_zeros, q_g_idx, duration, avg_loss, damp_percent, nsamples = g.quantize()
                                                                                 ~~~~~~~~~~^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/quantization/gptq.py", line 354, in quantize
    Hinv, damp = self.hessian_inverse(self.H)
                 ~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/quantization/gptq.py", line 257, in hessian_inverse
    H2 = torch.linalg.cholesky(H2)
RuntimeError: cusolver error: CUSOLVER_STATUS_INTERNAL_ERROR, when calling `cusolverDnCreate(handle)`. If you keep seeing this error, you may use `torch.backends.cuda.preferred_linalg_library()` to try linear algebra operators with other supported backends. See https://pytorch.org/docs/stable/backends.html#torch.backends.cuda.preferred_linalg_library

[Qubitium]

Another threading bug. This one is wild. So it just suddently stopped working. Looks like linalg ops are flaky under threading.

[Qubitium]

@avtc Both issue should be fixed for good. Please try it now. Will reopen if you still get errors.

[avtc]

INFO  ModuleLooper: forward start (processor=`gptq`, layer=`model.layers.1`, subset=3/7, batches=1057)                     %
Quantizing mlp.experts.32.gate_proj in layer  [1 of 45] █------------------------------------| 0:12:39 / 4:50:57 [2/46] 4.3%Traceback (most recent call last):
  File "/home/ubuntu/Documents/Quantize/quantize-glm4.5-Air-gptqmodel-moe-prune-smart-4.py", line 489, in <module>
    model.quantize(
    ~~~~~~~~~~~~~~^
        calibration_dataset,
        ^^^^^^^^^^^^^^^^^^^^
        batch_size=BATCH_SIZE,
        ^^^^^^^^^^^^^^^^^^^^^^
        )
        ^
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/models/base.py", line 946, in quantize
    return module_looper.loop(
           ~~~~~~~~~~~~~~~~~~^
        backend=backend,
        ^^^^^^^^^^^^^^^^
        fail_safe=self.quantize_config.fail_safe,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/module_looper.py", line 860, in loop
    name, m = fut.result()
              ~~~~~~~~~~^^
  File "/home/ubuntu/.pyenv/versions/3.13.7t/lib/python3.13t/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/ubuntu/.pyenv/versions/3.13.7t/lib/python3.13t/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/utils/threadx.py", line 367, in _run
    result = fn(*args, **kwargs)
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/module_looper.py", line 852, in _process_on_worker
    proc.process(module=nm)
    ~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/gptq_processor.py", line 123, in process
    wq, q_scales, q_zeros, q_g_idx, duration, avg_loss, damp_percent, nsamples = g.quantize()
                                                                                 ~~~~~~~~~~^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/quantization/gptq.py", line 354, in quantize
    Hinv, damp = self.hessian_inverse(self.H)
                 ~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/quantization/gptq.py", line 250, in hessian_inverse
    H2 = TORCH_LINALG.cholesky(H2)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/utils/safe.py", line 47, in locked
    return attr(*args, **kwargs)
RuntimeError: cusolver error: CUSOLVER_STATUS_INTERNAL_ERROR, when calling `cusolverDnCreate(handle)`. If you keep seeing this error, you may use `torch.backends.cuda.preferred_linalg_library()` to try linear algebra operators with other supported backends. See https://pytorch.org/docs/stable/backends.html#torch.backends.cuda.preferred_linalg_library
terminate called without an active exception
Aborted (core dumped)

Could be a memory issue? With 10 samples instead of 1084 - it does not throw.

The dataset to repro with GLM-4.5-Air:

import random

# Set seed for reproducibility
random.seed(42)

# 1. General Language
c4 = load_dataset(
    "allenai/c4",
    data_files="en/c4-train.00001-of-01024.json.gz",
    split="train"
).shuffle(seed=42).select(range(300)) # 300

# 2. Reasoning
gsm8k = load_dataset("gsm8k", "main", split="train").shuffle(seed=42).select(range(300)) # 300
arc = load_dataset("ai2_arc", "ARC-Challenge", split="train").shuffle(seed=42).select(range(300)) # 300

# 3. Technical/Development
humaneval = load_dataset("openai_humaneval", split="test").shuffle(seed=42).select(range(164)) # 164

# 4. Instruction-Following
alpaca = load_dataset("tat41s0u-lab/alpaca", split="train").shuffle(seed=42).select(range(20)) # 20

# Process each dataset and extract text
calibration_texts = []

# Process C4 dataset
for item in c4:
    calibration_texts.append(item["text"])

# Process GSM8K dataset
for item in gsm8k:
    calibration_texts.append(f"Question: {item['question']}\nAnswer: {item['answer']}")

# Process ARC dataset
for item in arc:
    calibration_texts.append(f"{item['question']}")

# Process HumanEval dataset
for item in humaneval:
    calibration_texts.append(item["prompt"])

# Process Alpaca dataset
for item in alpaca:
    input_text = f"\nInput: {item['input']}" if item.get("input") else ""
    calibration_texts.append(f"Instruction: {item['instruction']}{input_text}\nOutput: {item['output']}")

# Final shuffle to mix domains
random.shuffle(calibration_texts)

# Verify length
print(f"Total samples: {len(calibration_texts)}")

# Use with GPTQModel
calibration_dataset = calibration_texts  # This is your final calibration dataset

[Qubitium]

@avtc What! This looks to be a torch internal bug inferfacing with low level cusolver. I got the same crash as yours and solved with the PR fixes. Let me triple check your stacktrace to see how the heck it is still happening. There is a gigantic global lock on the linalg ops now so there is not possible for two thrads to execute anything torch.linalg related.

Can you send me your full running quant script and I will replicted on my end.

So I don't think it has any absolute relationship with your calibration dataset size or the memory usage of the ops. The dataset size only changed the timing of the calls.

[avtc]

@Qubitium
The full script.
quantize-glm4.5-Air-gptqmodel-moe-prune-smart-4.py

I have lowered number of samples in:

c4 = load_dataset(
    "allenai/c4",
    data_files="en/c4-train.00001-of-01024.json.gz",
    split="train"
).shuffle(seed=42).select(range(100)) # 300

and got another error, look like memory related:

Quantizing mlp.experts.33.gate_proj in layer  [1 of 45] █------------------------------------| 0:11:43 / 4:29:29 [2/46] 4.3%Traceback (most recent call last):
  File "/home/ubuntu/Documents/Quantize/quantize-glm4.5-Air-gptqmodel-moe-prune-smart-4.py", line 489, in <module>
    model.quantize(
    ~~~~~~~~~~~~~~^
        calibration_dataset,
        ^^^^^^^^^^^^^^^^^^^^
        batch_size=BATCH_SIZE,
        ^^^^^^^^^^^^^^^^^^^^^^
        )
        ^
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/models/base.py", line 946, in quantize
    return module_looper.loop(
           ~~~~~~~~~~~~~~~~~~^
        backend=backend,
        ^^^^^^^^^^^^^^^^
        fail_safe=self.quantize_config.fail_safe,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/module_looper.py", line 860, in loop
    name, m = fut.result()
              ~~~~~~~~~~^^
  File "/home/ubuntu/.pyenv/versions/3.13.7t/lib/python3.13t/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/ubuntu/.pyenv/versions/3.13.7t/lib/python3.13t/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/utils/threadx.py", line 367, in _run
    result = fn(*args, **kwargs)
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/module_looper.py", line 852, in _process_on_worker
    proc.process(module=nm)
    ~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/gptq_processor.py", line 123, in process
    wq, q_scales, q_zeros, q_g_idx, duration, avg_loss, damp_percent, nsamples = g.quantize()
                                                                                 ~~~~~~~~~~^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/quantization/gptq.py", line 494, in quantize
    W1[:, i:] -= err1.unsqueeze(1).matmul(Hinv1[i, i:].unsqueeze(0))
                 ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)`
terminate called without an active exception
Aborted (core dumped)

[avtc]

It there a switch to turn off data-parallelism to check without it?

[avtc]

After lowering samples more it passes the layer 1, will check how it goes.

# 1. General Language (40% - 410 samples)
c4 = load_dataset(
    "allenai/c4",
    data_files="en/c4-train.00001-of-01024.json.gz",
    split="train"
).shuffle(seed=42).select(range(100)) # 300

# 2. Reasoning (30% - 307 samples)
gsm8k = load_dataset("gsm8k", "main", split="train").shuffle(seed=42).select(range(100)) # 300
arc = load_dataset("ai2_arc", "ARC-Challenge", split="train").shuffle(seed=42).select(range(300)) # 300

# 3. Technical/Development (20% - 205 samples)
humaneval = load_dataset("openai_humaneval", split="test").shuffle(seed=42).select(range(164)) # 164

# 4. Instruction-Following (10% - 102 samples)
alpaca = load_dataset("tatsu-lab/alpaca", split="train").shuffle(seed=42).select(range(20)) # 20

[Qubitium]

Can you make it crash again and show me your gptqmodel quant logs on cli before the crash? I need to see your gpu memory usage per device which is now printed per completed quant module

It could be memory pressure.

[avtc]

@Qubitium
Set back dataset size, the issue reproduced.
It does not log experts before the exception.

NFO  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
INFO  | process     | layer     | module               | loss             | samples     | damp        | time      | fwd_time     | (v)ram                                                                                                              | dynamic     |
INFO  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
INFO  | gptq        | 1         | self_attn.k_proj     | 0.0000091407 | 247943      | 0.05000     | 0.960     | 17.797       | cuda:0=14.4GB, cuda:1=9.0GB, cuda:2=6.3GB, cuda:3=7.4GB, cuda:4=6.9GB, cuda:5=8.8GB, cuda:6=7.0GB, cuda:7=7.5GB     | None        | 
INFO  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
INFO  | gptq        | 1         | self_attn.v_proj     | 0.0000003122 | 247943      | 0.05000     | 1.007     | 17.797       | cuda:0=14.4GB, cuda:1=9.0GB, cuda:2=6.3GB, cuda:3=7.4GB, cuda:4=6.9GB, cuda:5=8.8GB, cuda:6=7.0GB, cuda:7=7.5GB     | None        | 
INFO  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
INFO  | gptq        | 1         | self_attn.q_proj     | 0.0000121996 | 247943      | 0.05000     | 1.558     | 17.797       | cuda:0=14.4GB, cuda:1=9.0GB, cuda:2=6.3GB, cuda:3=7.4GB, cuda:4=6.9GB, cuda:5=8.8GB, cuda:6=7.0GB, cuda:7=7.5GB     | None        | 
INFO  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
INFO  ModuleLooper: forward start (processor=`gptq`, layer=`model.layers.1`, subset=2/7, batches=1057)                     %
INFO  GC completed in 13.993s (pass #1).                                                                                   %
INFO  | gptq        | 1         | self_attn.o_proj     | 0.0000000035 | 247943      | 0.05000     | 2.590     | 66.516       | cuda:0=5.9GB, cuda:1=3.5GB, cuda:2=6.7GB, cuda:3=3.5GB, cuda:4=4.1GB, cuda:5=4.1GB, cuda:6=3.5GB, cuda:7=4.1GB      | None        | 
INFO  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
INFO  ModuleLooper: forward start (processor=`gptq`, layer=`model.layers.1`, subset=3/7, batches=1057)                     %
Quantizing mlp.experts.32.gate_proj in layer  [1 of 45] █------------------------------------| 0:14:30 / 5:33:30 [2/46] 4.3%Traceback (most recent call last):
  File "/home/ubuntu/Documents/Quantize/quantize-glm4.5-Air-gptqmodel-moe-prune-smart-4.py", line 489, in <module>
    model.quantize(
    ~~~~~~~~~~~~~~^
        calibration_dataset,
        ^^^^^^^^^^^^^^^^^^^^
        batch_size=BATCH_SIZE,
        ^^^^^^^^^^^^^^^^^^^^^^
        )
        ^
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/models/base.py", line 946, in quantize
    return module_looper.loop(
           ~~~~~~~~~~~~~~~~~~^
        backend=backend,
        ^^^^^^^^^^^^^^^^
        fail_safe=self.quantize_config.fail_safe,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/module_looper.py", line 860, in loop
    name, m = fut.result()
              ~~~~~~~~~~^^
  File "/home/ubuntu/.pyenv/versions/3.13.7t/lib/python3.13t/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/ubuntu/.pyenv/versions/3.13.7t/lib/python3.13t/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/utils/threadx.py", line 367, in _run
    result = fn(*args, **kwargs)
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/module_looper.py", line 852, in _process_on_worker
    proc.process(module=nm)
    ~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/looper/gptq_processor.py", line 123, in process
    wq, q_scales, q_zeros, q_g_idx, duration, avg_loss, damp_percent, nsamples = g.quantize()
                                                                                 ~~~~~~~~~~^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/quantization/gptq.py", line 354, in quantize
    Hinv, damp = self.hessian_inverse(self.H)
                 ~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/quantization/gptq.py", line 250, in hessian_inverse
    H2 = TORCH_LINALG.cholesky(H2)
  File "/home/ubuntu/git/avtc/GPTQModel/gptqmodel/utils/safe.py", line 47, in locked
    return attr(*args, **kwargs)
RuntimeError: cusolver error: CUSOLVER_STATUS_INTERNAL_ERROR, when calling `cusolverDnCreate(handle)`. If you keep seeing this error, you may use `torch.backends.cuda.preferred_linalg_library()` to try linear algebra operators with other supported backends. See https://pytorch.org/docs/stable/backends.html#torch.backends.cuda.preferred_linalg_library
terminate called without an active exception
terminate called recursively
terminate called recursively
Aborted (core dumped)

gptq_log_infatuation_time_10_04_2025_10h_30m_21s.log

[avtc]

The issue can be closed as related to low vram or large dataset. (btw same dataset worked for me before data parallel)